DD3-LoRa-Bridge-MultiSender/Requirements.md

# Firmware Requirements (Rust Port Preparation)

## 1. Scope

This document defines the behavior that must be preserved when recreating this firmware in another language (target: Rust).  
It is based on the current `lora-refactor` code state and captures:
- functional behavior
- protocol/data contracts
- module and function responsibilities
- runtime state-machine requirements

Function names below are C++ references. Rust naming/layout may differ, but the behavior must remain equivalent.

## 2. System-Level Requirements

- Role selection:
  - `Sender` when `GPIO14` reads HIGH.
  - `Receiver` when `GPIO14` reads LOW.
- Device identity:
  - derive `short_id` from MAC bytes 4/5.
  - canonical `device_id` format: `dd3-XXXX` uppercase hex.
- LoRa transport:
  - frame format: `[msg_kind][short_id_be][payload][crc16_ccitt]`.
  - reject invalid CRC/msg-kind/length.
- Payload codec:
  - schema `3` with `present_mask` (30-bit sparse second map).
  - support `n==0` sync-request packets.
- Time bootstrap guardrail:
  - sender must not run normal sampling/transmit until valid ACK time received.
  - accept ACK time only if `time_valid=1` and `epoch >= MIN_ACCEPTED_EPOCH_UTC`.
- Sampling/transmit cadence:
  - sender sample cadence 1 Hz.
  - sender batch cadence 30 s.
  - sync-request cadence 15 s while unsynced.
- Receiver behavior:
  - decode/reconstruct sparse timestamps.
  - ACK each decoded batch promptly.
  - update MQTT, web status, SD logging.
- Persistence:
  - Wi-Fi/MQTT/NTP/web credentials in Preferences namespace `dd3cfg`.
- Web and display time rendering:
  - local timezone from `TIMEZONE_TZ`.
- SD logging:
  - CSV columns include both `ts_utc` and `ts_hms_utc`.
  - history parser expects this current layout.

## 3. Protocol and Data Contracts

- `LoraMsgKind`:
  - `BatchUp=0`
  - `AckDown=1`
- `AckDown` payload fixed length `7` bytes:
  - `[flags:1][batch_id_be:2][epoch_utc_be:4]`
  - `flags bit0 = time_valid`
- `BatchInput`:
  - fixed arrays length `30` (`energy_wh`, `p1_w`, `p2_w`, `p3_w`)
  - `present_mask` must satisfy: only low 30 bits used and `bit_count == n`
- Timestamp constraints:
  - receiver rejects decoded data whose timestamps are below `MIN_ACCEPTED_EPOCH_UTC`
- CSV header (current required layout):
  - `ts_utc,ts_hms_utc,p_w,p1_w,p2_w,p3_w,e_kwh,bat_v,bat_pct,rssi,snr,err_m,err_d,err_tx,err_last`

## 4. Module and Function Requirements

## `src/config.cpp`

- `DeviceRole detect_role()`
  - configure role pin input pulldown and map to sender/receiver role.

## `src/data_model.cpp`

- `void init_device_ids(uint16_t&, char*, size_t)`
  - read MAC, derive short ID, format canonical device ID.
- `const char *rx_reject_reason_text(RxRejectReason)`
  - stable mapping for diagnostics and payloads.

## `src/html_util.cpp`

- `String html_escape(const String&)`
  - escape `& < > " '`.
- `String url_encode_component(const String&)`
  - percent-encode non-safe characters.
- `bool sanitize_device_id(const String&, String&)`
  - accept `XXXX` or `dd3-XXXX`; reject path traversal, `%`, invalid hex.
- Internal helpers to preserve behavior:
  - `is_hex_char`
  - `to_upper_hex4`

## `src/meter_driver.cpp`

- `void meter_init()`
  - configure `Serial2` at `9600 7E1`, RX pin `PIN_METER_RX`, RX buffer size `8192` on ESP32.
- `bool meter_poll_frame(const char *&, size_t&)`
  - incremental frame collector with start `/`, end `!`, timeout, overflow handling.
- `bool meter_parse_frame(const char*, size_t, MeterData&)`
  - parse OBIS values and set meter data fields.
- `bool meter_read(MeterData&)`
  - compatibility wrapper around poll+parse.
- Internal parse helpers to preserve numeric behavior:
  - `parse_obis_ascii_value`
  - `parse_obis_ascii_unit_scale`
  - `hex_nibble`
  - `parse_obis_hex_u32`
  - `meter_debug_log`

## `src/power_manager.cpp`

- `void power_sender_init()`
  - sender low-power setup (CPU freq, Wi-Fi/BT off, ADC setup).
- `void power_receiver_init()`
  - receiver power setup.
- `void power_configure_unused_pins_sender()`
  - configure known-unused pins with pulldown.
- `void read_battery(MeterData&)`
  - averaged ADC conversion and voltage calibration.
- `uint8_t battery_percent_from_voltage(float)`
  - LUT + interpolation.
- `void light_sleep_ms(uint32_t)`
  - timer-based light sleep.
- `void go_to_deep_sleep(uint32_t)`
  - timer-based deep sleep.

## `src/time_manager.cpp`

- `void time_receiver_init(const char*, const char*)`
  - configure NTP servers and timezone env.
- `uint32_t time_get_utc()`
  - return epoch or `0` when not plausible.
- `bool time_is_synced()`
  - sync status helper.
- `void time_set_utc(uint32_t)`
  - set system time and sync flags.
- `void time_get_local_hhmm(char*, size_t)`
  - timezone-based local `HH:MM` output.
- `uint32_t time_get_last_sync_utc()`
- `uint32_t time_get_last_sync_age_sec()`
- Internal behavior-critical helpers:
  - `note_last_sync`
  - `ensure_timezone_set`

## `src/lora_transport.cpp`

- `void lora_init()`
  - initialize SX1276 with configured LoRa params.
- `bool lora_send(const LoraPacket&)`
  - frame pack + CRC append + transmit.
- `bool lora_receive(LoraPacket&, uint32_t timeout_ms)`
  - parse frame, validate, return metadata including RSSI/SNR.
- `RxRejectReason lora_get_last_rx_reject_reason()`
  - consume-and-clear reject reason.
- `bool lora_get_last_rx_signal(int16_t&, float&)`
  - access last RX signal snapshot.
- `void lora_idle()`
- `void lora_sleep()`
- `void lora_receive_continuous()`
- `bool lora_receive_window(LoraPacket&, uint32_t)`
- `uint32_t lora_airtime_ms(size_t)`
  - compute packet airtime from SF/BW/CR/preamble.
- Internal behavior-critical helpers:
  - `note_reject`
  - `crc16_ccitt`

## `src/payload_codec.cpp`

- `bool encode_batch(const BatchInput&, uint8_t*, size_t, size_t*)`
  - schema v3 encoder with metadata, sparse present mask, delta coding.
- `bool decode_batch(const uint8_t*, size_t, BatchInput*)`
  - strict schema/magic/flags decode + bounds checks.
- Varint primitives:
  - `uleb128_encode`, `uleb128_decode`
  - `zigzag32`, `unzigzag32`
  - `svarint_encode`, `svarint_decode`
- Internal helpers:
  - `write_u16_le`, `write_u32_le`
  - `read_u16_le`, `read_u32_le`
  - `ensure_capacity`
  - `bit_count32`
- Optional self-test:
  - `payload_codec_self_test` (when `PAYLOAD_CODEC_TEST`).

## `src/json_codec.cpp`

- `bool meterDataToJson(const MeterData&, String&)`
  - create MQTT state JSON with stable field semantics.
- Internal numeric formatting helpers:
  - `round2`
  - `round_to_i32`
  - `short_id_from_device_id`
  - `format_float_2`
  - `set_int_or_null`

## `src/mqtt_client.cpp`

- `void mqtt_init(const WifiMqttConfig&, const char*)`
- `void mqtt_loop()`
- `bool mqtt_is_connected()`
- `bool mqtt_publish_state(const MeterData&)`
- `bool mqtt_publish_faults(const char*, const FaultCounters&, FaultType, uint32_t)`
- `bool mqtt_publish_discovery(const char*)`
- `bool mqtt_publish_test(const char*, const String&)` (test mode only)
- Internal behavior-critical helpers:
  - `fault_text`
  - `mqtt_connect`
  - `publish_discovery_sensor`

## `src/wifi_manager.cpp`

- `void wifi_manager_init()`
- `bool wifi_load_config(WifiMqttConfig&)`
- `bool wifi_save_config(const WifiMqttConfig&)`
- `bool wifi_connect_sta(const WifiMqttConfig&, uint32_t timeout_ms)`
- `void wifi_start_ap(const char*, const char*)`
- `bool wifi_is_connected()`
- `String wifi_get_ssid()`

## `src/sd_logger.cpp`

- `void sd_logger_init()`
- `bool sd_logger_is_ready()`
- `void sd_logger_log_sample(const MeterData&, bool include_error_text)`
  - append/create per-day CSV under `/dd3/<device_id>/YYYY-MM-DD.csv`.
- Internal behavior-critical helpers:
  - `fault_text`
  - `ensure_dir`
  - `format_date_utc`
  - `format_hms_utc`

## `src/display_ui.cpp`

Public display API that must remain behavior-equivalent:
- `display_power_down`
- `display_init`
- `display_set_role`
- `display_set_self_ids`
- `display_set_sender_statuses`
- `display_set_last_meter`
- `display_set_last_read`
- `display_set_last_tx`
- `display_set_sender_queue`
- `display_set_sender_batches`
- `display_set_last_error`
- `display_set_receiver_status`
- `display_set_test_code` (test mode)
- `display_set_test_code_for_sender` (test mode)
- `display_tick`

Internal rendering helpers to preserve behavior:
- `oled_set_power`
- `age_seconds`
- `round_power_w`
- `render_last_error_line`
- `render_last_sync_line`
- `render_sender_status`
- `render_sender_measurement`
- `render_receiver_status`
- `render_receiver_sender`

## `src/web_server.cpp`

Public web API:
- `web_server_set_config`
- `web_server_set_sender_faults`
- `web_server_set_last_batch`
- `web_server_begin_ap`
- `web_server_begin_sta`
- `web_server_loop`

Internal route/state functions to preserve behavior:
- `format_local_hms`
- `format_epoch_local_hms`
- `timestamp_age_seconds`
- `round_power_w`
- `auth_required`
- `fault_text`
- `ensure_auth`
- `html_header`
- `html_footer`
- `format_faults`
- `sanitize_sd_download_path`
- `checkbox_checked`
- `sanitize_history_device_id`
- `sanitize_download_filename`
- `history_reset`
- `history_date_from_epoch`
- `history_open_next_file`
- `history_parse_line`
- `history_tick`
- `render_sender_block`
- `append_sd_listing`
- `handle_root`
- `handle_wifi_get`
- `handle_wifi_post`
- `handle_sender`
- `handle_manual`
- `handle_history_start`
- `handle_history_data`
- `handle_sd_download`

## `src/test_mode.cpp` (`ENABLE_TEST_MODE`)

- `test_sender_loop`
  - periodic JSON test frame transmit.
- `test_receiver_loop`
  - decode test JSON, update display test markers, publish MQTT test topic.

## `src/main.cpp` (Core Orchestration)

These functions define end-to-end firmware behavior and must have equivalents:

- Logging/utilities:
  - `serial_debug_printf`
  - `bit_count32`
  - `abs_diff_u32`

- Meter-time anchoring and ingest:
  - `meter_time_update_snapshot`
  - `set_last_meter_sample`
  - `parse_meter_frame_sample`
  - `meter_queue_push_latest`
  - `meter_reader_task_entry`
  - `meter_reader_start`
  - `meter_reader_pump`

- Sender/receiver state setup and shared state:
  - `init_sender_statuses`
  - `update_battery_cache`
  - `battery_sample_due`

- Queue and sample batching:
  - `batch_queue_drop_oldest`
  - `sender_note_rx_reject`
  - `batch_queue_peek`
  - `batch_queue_enqueue`
  - `reset_build_counters`
  - `append_meter_sample`
  - `last_sample_ts`

- Fault tracking/publish:
  - `note_fault`
  - `age_seconds`
  - `counters_changed`
  - `publish_faults_if_needed`

- Watchdog:
  - `watchdog_init`
  - `watchdog_kick`

- Binary helpers and ID conversion:
  - `write_u16_le`
  - `read_u16_le`
  - `write_u16_be`
  - `read_u16_be`
  - `write_u32_be`
  - `read_u32_be`
  - `sender_id_from_short_id`
  - `short_id_from_sender_id`

- Numeric normalization/sanitization:
  - `kwh_to_wh_from_float`
  - `float_to_i16_w`
  - `float_to_i16_w_clamped`
  - `battery_mv_from_voltage`

- Timeout and airtime-driven scheduling:
  - `compute_batch_rx_timeout_ms`
  - `compute_batch_ack_timeout_ms`

- LoRa TX pipeline:
  - `send_batch_payload`
  - `send_batch_ack`
  - `prepare_inflight_from_queue`
  - `send_inflight_batch`
  - `send_meter_batch`
  - `send_sync_request`
  - `resend_inflight_batch`
  - `finish_inflight_batch`

- LoRa RX reassembly/decode:
  - `reset_batch_rx`
  - `process_batch_packet`

- Role loop orchestration:
  - `setup`
  - `sender_loop`
  - `receiver_loop`
  - `loop`

## 5. Rust Porting Constraints and Recommendations

- Preserve wire compatibility first:
  - LoRa frame byte layout, CRC16, ACK format, payload schema v3.
- Preserve persistent storage keys:
  - Preferences keys (`ssid`, `pass`, `mqhost`, `mqport`, `mquser`, `mqpass`, `ntp1`, `ntp2`, `webuser`, `webpass`, `valid`).
- Preserve timing constants and acceptance thresholds:
  - bootstrap guardrail, retry counts, schedule intervals, min accepted epoch.
- Preserve CSV output layout exactly:
  - consumers (history parser and external tooling) depend on it.
- Preserve enum meanings:
  - `FaultType`, `RxRejectReason`, `LoraMsgKind`.

Suggested Rust module split:
- `config`, `ids`, `meter`, `power`, `time`, `lora_transport`, `payload_codec`, `batch`, `mqtt`, `wifi_cfg`, `sd_log`, `web`, `display`, `runtime`.

Suggested Rust primitives:
- async task for meter reader + bounded channel (drop-oldest behavior).
- explicit state structs for sender/receiver loops.
- serde-free/manual codec for wire compatibility where needed.

## 6. Port Validation Checklist

- Sender unsynced boot sends only sync requests.
- ACK time bootstrap unlocks normal sender sampling.
- Sparse present-mask encode/decode round-trip matches C++.
- Receiver reconstructs timestamps correctly for gaps.
- Duplicate batch handling updates counters and suppresses duplicate publish/log.
- Web UI shows `epoch (HH:MM:SS TZ)` local time.
- SD CSV header/fields match expected order.
- History endpoint reads only current CSV layout successfully.
- MQTT state/fault payload fields match existing names and semantics.