DD3-LoRa-Bridge-MultiSender/Requirements.md

Firmware Requirements (Rust Port Preparation)

1. Scope

This document defines the behavior that must be preserved when recreating this firmware in another language (target: Rust).
It is based on the current lora-refactor code state and captures:

• functional behavior
• protocol/data contracts
• module and function responsibilities
• runtime state-machine requirements

Function names below are C++ references. Rust naming/layout may differ, but the behavior must remain equivalent.

2. Refactored Architecture Baseline

The lora-refactor branch split role-specific runtime from the previous large main.cpp into dedicated modules while keeping a single firmware image:

• src/main.cpp is a thin coordinator that:
  • detects role and initializes shared platform subsystems,
  • prepares role module configuration,
  • calls begin() once,
  • delegates runtime in loop().
• sender runtime ownership:
  • src/sender_state_machine.h
  • src/sender_state_machine.cpp
• receiver runtime ownership:
  • src/receiver_pipeline.h
  • src/receiver_pipeline.cpp
• receiver shared mutable state used by setup wiring and runtime:
  • src/app_context.h (ReceiverSharedState)

Sender state machine invariants must remain behavior-equivalent:

• single inflight batch at a time,
• ACK acceptance only for matching batch_id,
• retry bounded by BATCH_MAX_RETRIES,
• queue depth bounded by BATCH_QUEUE_DEPTH.

3. System-Level Requirements

• Role selection:
  • Sender when GPIO14 reads HIGH.
  • Receiver when GPIO14 reads LOW.
• Device identity:
  • derive short_id from MAC bytes 4/5.
  • canonical device_id format: dd3-XXXX uppercase hex.
• LoRa transport:
  • frame format: [msg_kind][short_id_be][payload][crc16_ccitt].
  • reject invalid CRC/msg-kind/length.
• Payload codec:
  • schema 3 with present_mask (30-bit sparse second map).
  • support n==0 sync-request packets.
• Time bootstrap guardrail:
  • sender must not run normal sampling/transmit until valid ACK time received.
  • accept ACK time only if time_valid=1 and epoch >= MIN_ACCEPTED_EPOCH_UTC.
  • sender fault counters reset when first valid sync is accepted.
  • after first sync, sender fault counters reset again at each UTC hour boundary.
• Sampling/transmit cadence:
  • sender sample cadence 1 Hz.
  • sender batch cadence 30 s.
  • when sender backlog exists (batch_count > 1) and no ACK is pending, sender performs immediate catch-up sends (still stop-and-wait with one inflight batch).
  • sync-request cadence 15 s while unsynced.
  • sender retransmits reuse cached encoded payload bytes for same inflight batch.
  • sender ACK receive window is adaptive from airtime + observed ACK RTT, with expanded second window on miss.
• Receiver behavior:
  • decode/reconstruct sparse timestamps.
  • ACK accepted batches promptly.
  • reject unknown/mismatched sender identities before ACK and before SD/MQTT/web updates.
  • update MQTT, web status, SD logging.
• Persistence:
  • Wi-Fi/MQTT/NTP/web credentials in Preferences namespace dd3cfg.
• Web auth defaults:
  • WEB_AUTH_REQUIRE_STA=true
  • WEB_AUTH_REQUIRE_AP=true
• Web and display time rendering:
  • local timezone from TIMEZONE_TZ.
• Sender diagnostics:
  • structured sender diagnostics are emitted to serial debug output only.
  • diagnostics do not change LoRa payload schema or remap payload fields.
• SD logging:
  • CSV columns include both ts_utc and ts_hms_local.
  • per-day CSV file partitioning uses local date (TIMEZONE_TZ) under /dd3/<device_id>/YYYY-MM-DD.csv.
  • history day-file resolution prefers local-date filenames and falls back to legacy UTC-date filenames.
  • history parser supports both current (ts_utc,ts_hms_local,p_w,...) and legacy (ts_utc,p_w,...) layouts.

4. Protocol and Data Contracts

• LoraMsgKind:
  • BatchUp=0
  • AckDown=1
• AckDown payload fixed length 7 bytes:
  • [flags:1][batch_id_be:2][epoch_utc_be:4]
  • flags bit0 = time_valid
  • sender acceptance window is implementation-adaptive; payload format stays unchanged.
• BatchInput:
  • fixed arrays length 30 (energy_wh, p1_w, p2_w, p3_w)
  • present_mask must satisfy: only low 30 bits used and bit_count == n
• Timestamp constraints:
  • receiver rejects decoded data whose timestamps are below MIN_ACCEPTED_EPOCH_UTC
• CSV header (current required layout):
  • ts_utc,ts_hms_local,p_w,p1_w,p2_w,p3_w,e_kwh,bat_v,bat_pct,rssi,snr,err_m,err_d,err_tx,err_last
• Home Assistant discovery contract:
  • topic: homeassistant/sensor/<device_id>/<key>/config
  • unique_id: <device_id>_<key>
  • device.identifiers: ["<device_id>"]
  • device.name: <device_id>
  • device.model: DD3-LoRa-Bridge
  • device.manufacturer: AcidBurns
  • drift guards:
    • canonical value is HA_MANUFACTURER in include/config.h,
    • compile-time lock via static_assert in include/config.h,
    • script guard test/check_ha_manufacturer.ps1,
    • smoke test guard test/test_refactor_smoke/test_refactor_smoke.cpp.

5. Module and Function Requirements

src/config.cpp

• DeviceRole detect_role()
  • configure role pin input pulldown and map to sender/receiver role.

lib/dd3_legacy_core/src/data_model.cpp

• void init_device_ids(uint16_t&, char*, size_t)
  • read MAC, derive short ID, format canonical device ID.
• const char *rx_reject_reason_text(RxRejectReason)
  • stable mapping for diagnostics and payloads.

lib/dd3_legacy_core/src/html_util.cpp

• String html_escape(const String&)
  • escape & < > " '.
• String url_encode_component(const String&)
  • percent-encode non-safe characters.
• bool sanitize_device_id(const String&, String&)
  • accept XXXX or dd3-XXXX; reject path traversal, %, invalid hex.
• Internal helpers to preserve behavior:
  • is_hex_char
  • to_upper_hex4

src/meter_driver.cpp

• void meter_init()
  • configure Serial2 at 9600 7E1, RX pin PIN_METER_RX, RX buffer size 8192 on ESP32.
• bool meter_poll_frame(const char *&, size_t&)
  • incremental frame collector with start /, end !, timeout, overflow handling.
• bool meter_parse_frame(const char*, size_t, MeterData&)
  • parse OBIS values and set meter data fields.
• bool meter_read(MeterData&)
  • compatibility wrapper around poll+parse.
• void meter_get_stats(MeterDriverStats&)
  • expose parser/UART counters for sender-local diagnostics.
• Internal parse helpers to preserve numeric behavior:
  • detect_obis_field
  • parse_decimal_fixed
  • parse_obis_ascii_payload_value
  • parse_obis_ascii_unit_scale
  • hex_nibble
  • parse_obis_hex_payload_u32
  • meter_debug_log

src/power_manager.cpp

• void power_sender_init()
  • sender low-power setup (CPU freq, Wi-Fi/BT off, ADC setup).
• void power_receiver_init()
  • receiver power setup.
• void power_configure_unused_pins_sender()
  • configure known-unused pins with pulldown.
• void read_battery(MeterData&)
  • averaged ADC conversion and voltage calibration.
• uint8_t battery_percent_from_voltage(float)
  • LUT + interpolation.
• void light_sleep_ms(uint32_t)
  • timer-based light sleep.
• void go_to_deep_sleep(uint32_t)
  • timer-based deep sleep.

src/time_manager.cpp

• void time_receiver_init(const char*, const char*)
  • configure NTP servers and timezone env.
• uint32_t time_get_utc()
  • return epoch or 0 when not plausible.
  • updates "clock plausible" state independently from sync state.
• bool time_is_synced()
  • true only after explicit sync signals (NTP callback/status or trusted time_set_utc).
• void time_set_utc(uint32_t)
  • set system time and sync flags.
• void time_get_local_hhmm(char*, size_t)
  • timezone-based local HH:MM output.
• uint32_t time_get_last_sync_utc()
• uint32_t time_get_last_sync_age_sec()
• Internal behavior-critical helpers:
  • note_last_sync
  • mark_synced
  • ntp_sync_notification_cb
  • ensure_timezone_set

src/lora_transport.cpp

• void lora_init()
  • initialize SX1276 with configured LoRa params.
• bool lora_send(const LoraPacket&)
  • frame pack + CRC append + transmit.
• bool lora_receive(LoraPacket&, uint32_t timeout_ms)
  • parse frame, validate, return metadata including RSSI/SNR.
• RxRejectReason lora_get_last_rx_reject_reason()
  • consume-and-clear reject reason.
• bool lora_get_last_rx_signal(int16_t&, float&)
  • access last RX signal snapshot.
• void lora_idle()
• void lora_sleep()
• void lora_receive_continuous()
• bool lora_receive_window(LoraPacket&, uint32_t)
• uint32_t lora_airtime_ms(size_t)
  • compute packet airtime from SF/BW/CR/preamble.
• Internal behavior-critical helpers:
  • note_reject
  • lora_build_frame, lora_parse_frame, lora_crc16_ccitt (implemented in lib/dd3_transport_logic/src/lora_frame_logic.cpp)

lib/dd3_legacy_core/src/payload_codec.cpp

• bool encode_batch(const BatchInput&, uint8_t*, size_t, size_t*)
  • schema v3 encoder with metadata, sparse present mask, delta coding.
• bool decode_batch(const uint8_t*, size_t, BatchInput*)
  • strict schema/magic/flags decode + bounds checks.
• Varint primitives:
  • uleb128_encode, uleb128_decode
  • zigzag32, unzigzag32
  • svarint_encode, svarint_decode
• Internal helpers:
  • write_u16_le, write_u32_le
  • read_u16_le, read_u32_le
  • ensure_capacity
  • bit_count32
• Optional self-test:
  • payload_codec_self_test (when PAYLOAD_CODEC_TEST).

lib/dd3_legacy_core/src/json_codec.cpp

• bool meterDataToJson(const MeterData&, String&)
  • create MQTT state JSON with stable field semantics.
• Internal numeric formatting helpers:
  • round2
  • round_to_i32
  • short_id_from_device_id
  • format_float_2
  • set_int_or_null

src/mqtt_client.cpp

• void mqtt_init(const WifiMqttConfig&, const char*)
• void mqtt_loop()
• bool mqtt_is_connected()
• bool mqtt_publish_state(const MeterData&)
• bool mqtt_publish_faults(const char*, const FaultCounters&, FaultType, uint32_t)
• bool mqtt_publish_discovery(const char*)
• bool mqtt_publish_test(const char*, const String&) (test mode only)
• Internal behavior-critical helpers:
  • fault_text
  • mqtt_connect
  • publish_discovery_sensor
  • discovery payload uses canonical device identity fields and manufacturer=AcidBurns

src/wifi_manager.cpp

• void wifi_manager_init()
• bool wifi_load_config(WifiMqttConfig&)
• bool wifi_save_config(const WifiMqttConfig&)
  • returns false when any Preferences write/verify fails.
• bool wifi_connect_sta(const WifiMqttConfig&, uint32_t timeout_ms)
• void wifi_start_ap(const char*, const char*)
• bool wifi_is_connected()
• String wifi_get_ssid()

src/sd_logger.cpp

• void sd_logger_init()
• bool sd_logger_is_ready()
• void sd_logger_log_sample(const MeterData&, bool include_error_text)
  • append/create per-day CSV under /dd3/<device_id>/YYYY-MM-DD.csv using local calendar date from TIMEZONE_TZ.
• Internal behavior-critical helpers:
  • fault_text
  • ensure_dir
  • format_date_local
  • format_hms_local

src/display_ui.cpp

Public display API that must remain behavior-equivalent:

• display_power_down
• display_init
• display_set_role
• display_set_self_ids
• display_set_sender_statuses
• display_set_last_meter
• display_set_last_read
• display_set_last_tx
• display_set_sender_queue
• display_set_sender_batches
• display_set_last_error
• display_set_receiver_status
• display_set_test_code (test mode)
• display_set_test_code_for_sender (test mode)
• display_tick

Internal rendering helpers to preserve behavior:

• oled_set_power
• age_seconds
• round_power_w
• render_last_error_line
• render_last_sync_line
• render_sender_status
• render_sender_measurement
• render_receiver_status
• render_receiver_sender

src/web_server.cpp

Public web API:

• web_server_set_config
• web_server_set_sender_faults
• web_server_set_last_batch
• web_server_begin_ap
• web_server_begin_sta
• web_server_loop

Internal route/state functions to preserve behavior:

• format_local_hms
• format_epoch_local_hms
• timestamp_age_seconds
• round_power_w
• auth_required
• fault_text
• ensure_auth
• html_header
• html_footer
• format_faults
• sanitize_sd_download_path
• checkbox_checked
• sanitize_history_device_id
• sanitize_download_filename
• history_reset
• history_date_from_epoch_local
• history_date_from_epoch_utc (legacy fallback mapping)
• history_open_next_file
• history_parse_line
• history_tick
• render_sender_block
• append_sd_listing
• handle_root
• handle_wifi_get
• handle_wifi_post
• handle_sender
• handle_manual
• handle_history_start
• handle_history_data
• handle_sd_download

src/test_mode.cpp (ENABLE_TEST_MODE)

• test_sender_loop
  • periodic JSON test frame transmit.
• test_receiver_loop
  • decode test JSON, update display test markers, publish MQTT test topic.

src/app_context.h

• ReceiverSharedState
  • retains receiver-owned shared status/fault/discovery state used by setup wiring and runtime.

src/sender_state_machine.h/.cpp (Sender Runtime)

Public API:

• SenderStateMachineConfig
• SenderStats
• SenderStateMachine::begin(...)
• SenderStateMachine::loop()
• SenderStateMachine::stats()

Behavior-critical internals (migrated from pre-refactor main.cpp) that must remain equivalent:

• Logging/utilities:
  • serial_debug_printf
  • bit_count32
  • abs_diff_u32
• Meter-time anchoring and ingest:
  • meter_time_update_snapshot
  • set_last_meter_sample
  • parse_meter_frame_sample
  • meter_queue_push_latest
  • meter_reader_task_entry
  • meter_reader_start
  • meter_reader_pump
• Sender state/data handling:
  • update_battery_cache
  • battery_sample_due
  • batch_queue_drop_oldest
  • sender_note_rx_reject
  • sender_log_diagnostics
  • batch_queue_peek
  • batch_queue_enqueue
  • reset_build_counters
  • append_meter_sample
  • last_sample_ts
• Sender fault handling:
  • note_fault
  • clear_faults
  • sender_reset_fault_stats
  • sender_reset_fault_stats_on_first_sync
  • sender_reset_fault_stats_on_hour_boundary
• Sender-specific encoding/scheduling:
  • kwh_to_wh_from_float
  • float_to_i16_w
  • float_to_i16_w_clamped
  • battery_mv_from_voltage
  • compute_batch_ack_timeout_ms
  • send_batch_payload
  • invalidate_inflight_encode_cache
  • prepare_inflight_from_queue
  • send_inflight_batch
  • send_meter_batch
  • send_sync_request
  • resend_inflight_batch
  • finish_inflight_batch
  • sender_loop

src/receiver_pipeline.h/.cpp (Receiver Runtime)

Public API:

• ReceiverPipelineConfig
• ReceiverStats
• ReceiverPipeline::begin(...)
• ReceiverPipeline::loop()
• ReceiverPipeline::stats()

Behavior-critical internals (migrated from pre-refactor main.cpp) that must remain equivalent:

• Receiver setup/state:
  • init_sender_statuses
• Fault handling/publish:
  • note_fault
  • clear_faults
  • age_seconds
  • counters_changed
  • publish_faults_if_needed
• Binary helpers and ID conversion:
  • write_u16_le
  • read_u16_le
  • write_u16_be
  • read_u16_be
  • write_u32_be
  • read_u32_be
  • sender_id_from_short_id
  • short_id_from_sender_id
• LoRa RX/TX pipeline:
  • compute_batch_rx_timeout_ms
  • send_batch_ack
  • reset_batch_rx
  • process_batch_packet
  • receiver_loop

src/main.cpp (Thin Coordinator)

Current core orchestration requirements:

• setup
  • initialize shared subsystems once,
  • force-link dd3_legacy_core before first legacy-core symbol use (dd3_legacy_core_force_link()),
  • instantiate role config and call role begin,
  • keep role-specific runtime out of this file.
• loop
  • delegate to SenderStateMachine::loop() or ReceiverPipeline::loop() by role.
• Watchdog wrapper remains in coordinator:
  • watchdog_init
  • watchdog_kick

6. Rust Porting Constraints and Recommendations

• Preserve wire compatibility first:
  • LoRa frame byte layout, CRC16, ACK format, payload schema v3.
  • sender optimization changes must not alter payload field meanings.
• Preserve persistent storage keys:
  • Preferences keys (ssid, pass, mqhost, mqport, mquser, mqpass, ntp1, ntp2, webuser, webpass, valid).
• Preserve timing constants and acceptance thresholds:
  • bootstrap guardrail, retry counts, schedule intervals, min accepted epoch.
• Preserve CSV output layout exactly:
  • consumers (history parser and external tooling) depend on it.
  • preserve reader compatibility for both current and legacy layouts.
• Preserve enum meanings:
  • FaultType, RxRejectReason, LoraMsgKind.

Suggested Rust module split:

• config, ids, meter, power, time, lora_transport, payload_codec, sender_state_machine, receiver_pipeline, app_context, mqtt, wifi_cfg, sd_log, web, display, runtime.

Suggested Rust primitives:

• async task for meter reader + bounded channel (drop-oldest behavior).
• explicit state structs for sender/receiver loops.
• serde-free/manual codec for wire compatibility where needed.

7. Port Validation Checklist

• Sender unsynced boot sends only sync requests.
• ACK time bootstrap unlocks normal sender sampling.
• Sparse present-mask encode/decode round-trip matches C++.
• Receiver reconstructs timestamps correctly for gaps.
• Duplicate batch handling updates counters and suppresses duplicate publish/log.
• Web UI shows epoch (HH:MM:SS TZ) local time.
• SD CSV header/fields match expected order.
• SD daily files roll over at local midnight (TIMEZONE_TZ), not UTC midnight.
• History endpoint reads current and legacy CSV layouts successfully.
• History endpoint can read both local-date and legacy UTC-date day filenames.
• MQTT state/fault payload fields match existing names and semantics.

8. Port Readiness Audit (2026-02-20)

Evidence checked on lora-refactor:

• build verification:
  • pio run -e lilygo-t3-v1-6-1
  • pio run -e lilygo-t3-v1-6-1-test
• drift guard verification:
  • powershell -ExecutionPolicy Bypass -File test/check_ha_manufacturer.ps1
• refactor ownership verification:
  • sender state machine state/API present in src/sender_state_machine.h/.cpp,
  • receiver pipeline API present in src/receiver_pipeline.h/.cpp,
  • coordinator remains thin in src/main.cpp.

Findings:

• Requirements are functionally met by current C++ baseline from static/code-build checks.
• The old requirement ownership under src/main.cpp was stale; this document now maps that behavior to sender_state_machine and receiver_pipeline.
• No wire/protocol or persistence contract drift found in this audit.