Add firmware verification harness

2026-07-13 11:10:19 +02:00
parent def09160d0
commit 99c07ecda6
16 changed files with 1004 additions and 32 deletions
@@ -0,0 +1,117 @@
+# Current State
+
+Assessment date: 2026-07-13
+Branch assessed: `lora-refactor`
+Commit assessed: `def0916 refactor lora payload timing`
+
+## Architecture Verification
+
+| Assumption | Status | Evidence |
+|---|---:|---|
+| `src/main.cpp` is the thin top-level coordinator. | Verified | Initializes shared services, creates role configs, delegates `loop()` to role objects. |
+| Sender runtime is owned by `SenderStateMachine`. | Verified | `src/sender_state_machine.h/.cpp` own sender state, queueing, sync, ACK, retry, and sleep loop. |
+| Receiver runtime is owned by `ReceiverPipeline`. | Verified | `src/receiver_pipeline.h/.cpp` own receive, reassembly, ACK, duplicate, MQTT, SD, and web update flow. |
+| One firmware image supports both roles. | Verified | `detect_role()` selects role at boot; `loop()` dispatches sender or receiver in one image. |
+| PlatformIO is the canonical build system. | Verified | `platformio.ini` defines all firmware environments. |
+| Arduino remains the framework. | Verified | `[env] framework = arduino`. |
+| pioarduino Espressif platform remains pinned. | Verified | Platform URL is `https://github.com/pioarduino/platform-espressif32/releases/download/51.03.07/platform-espressif32.zip`. |
+| Existing environments include `production`, `debug`, and `test`. | Verified | `platformio.ini` has exactly those project environments. |
+| Payload schema v4 is current. | Verified | `lib/dd3_legacy_core/src/payload_codec.cpp` uses `kSchema = 4`; payload tests use schema v4 vectors. |
+| Existing golden vectors remain relevant. | Verified by inspection | `test/test_payload_codec` contains schema v4 golden vectors. |
+| Power behavior was changed by this task. | Not changed | This harness task did not edit firmware source. |
+
+## Current Build Configuration
+
+- PlatformIO Core observed locally: `6.1.18`.
+- Expected PlatformIO Core for CI and reproducible verification: `6.1.18`.
+- PlatformIO emits a local warning that multiple/obsolete cores may be installed: `Obsolete PIO Core v6.1.18 is used (previous was 6.1.19)`.
+- Board: `ttgo-lora32-v1`.
+- Framework: Arduino.
+- Environments:
+  - `production`: serial debug off, light sleep on.
+  - `debug`: serial diagnostics on, real meter and real LoRa.
+- `test`: `ENABLE_TEST_MODE` and `PAYLOAD_CODEC_TEST`.
+
+## Verification Snapshot
+
+- `pio run -e production`: passed; RAM `95588` bytes, flash `1182653` bytes.
+- `pio run -e debug`: passed; RAM `95628` bytes, flash `1189501` bytes.
+- `pio run -e test`: passed; RAM `95060` bytes, flash `1182177` bytes.
+- `pio test -e test --without-uploading --without-testing`: passed compile-only for all seven embedded Unity suites.
+- `python tools/check_size.py`: passed with current artifact baselines in `docs/engineering/size-baseline.json`.
+- `python tools/verify.py`: failed only at static analysis; `pio check -e production --fail-on-defect=high` exits 1 because cppcheck fails on the Xtensa C++ toolchain package header.
+- `pio check -e production --fail-on-defect=high --skip-packages`: also failed because cppcheck reports high defects in third-party ArduinoJson headers under `.pio/libdeps`.
+- The strict static-analysis failure is intentionally preserved until analyzer scope and third-party dependency policy are reviewed.
+
+## Resolved Dependency Baseline
+
+Observed with `pio pkg list` after forcing UTF-8 console output:
+
+| Dependency | Configured requirement | Resolved version |
+|---|---|---:|
+| pioarduino `platform-espressif32` | release URL `51.03.07` | `51.3.6` |
+| Arduino ESP32 framework | platform-provided | `3.0.7` |
+| ESP32 Arduino libs | platform-provided | `5.1.0+sha.632e0c2a9f` |
+| `sandeepmistry/LoRa` | `^0.8.0` | `0.8.0` |
+| `bblanchon/ArduinoJson` | `^6.21.5` | `6.21.6` |
+| `adafruit/Adafruit SSD1306` | `^2.5.9` | `2.5.17` |
+| `adafruit/Adafruit GFX Library` | `^1.11.9` | `1.12.6` |
+| `Adafruit BusIO` | transitive | `1.17.4` |
+| `knolleary/PubSubClient` | `^2.8` | `2.8.0` |
+| `throwtheswitch/Unity` | `^2.6.1` | `2.6.1` |
+
+Proposed exact direct pins for a separate dependency-only commit:
+
+```ini
+sandeepmistry/LoRa@0.8.0
+bblanchon/ArduinoJson@6.21.6
+adafruit/Adafruit SSD1306@2.5.17
+adafruit/Adafruit GFX Library@1.12.6
+knolleary/PubSubClient@2.8.0
+throwtheswitch/Unity@2.6.1
+```
+
+## Hardware and Persistence Contracts
+
+- Role pin: `GPIO14`; HIGH is sender, LOW is receiver.
+- OLED control pin: `GPIO13`.
+- LoRa defaults in `include/config.h`: 868 MHz, SF12, BW 125 kHz, coding rate 4/5, sync word `0x34`.
+- Configured sender short ID: `0x6540`; configured receiver short ID: `0x7EB4`.
+- Preferences namespace: `dd3cfg`.
+- Preferences keys: `ssid`, `pass`, `mqhost`, `mqport`, `mquser`, `mqpass`, `ntp1`, `ntp2`, `webuser`, `webpass`, `valid`.
+- CSV current header: `ts_utc,ts_hms_local,p_w,p1_w,p2_w,p3_w,e_kwh,bat_v,bat_pct,rssi,snr,err_m,err_d,err_tx,err_last`.
+
+## Protocol Contracts
+
+- LoRa frame: `[msg_kind:1][device_short_id_be:2][payload...][crc16_ccitt_be:2]`.
+- `LoraMsgKind`: `BatchUp=0`, `AckDown=1`.
+- `BatchUp` chunk header: `[batch_id_le:2][chunk_index:1][chunk_count:1][total_len_le:2][chunk_payload...]`.
+- Payload codec: schema v4, magic `0xDDB3`, 30-bit `present_mask`, `n == 0` sync-request support.
+- `AckDown` payload length: 7 bytes, `[flags:1][batch_id_be:2][epoch_utc_be:4]`.
+- ACK time is accepted only when `flags bit0` is set and epoch is at least `MIN_ACCEPTED_EPOCH_UTC`.
+- MQTT state topic: `smartmeter/<device_id>/state`.
+- MQTT fault topic: `smartmeter/<device_id>/faults`.
+- Test MQTT topic: `smartmeter/<device_id>/test`.
+- Home Assistant discovery topic: `homeassistant/sensor/<device_id>/<key>/config`.
+
+## Repository Shape
+
+- `lib/dd3_legacy_core`: shared legacy logic for data model, HTML utilities, JSON codec, and payload codec.
+- `lib/dd3_transport_logic`: pure-ish transport helpers for LoRa frame, HA discovery JSON, and batch reassembly.
+- `test/`: seven embedded Unity suites, compile-only runnable without a board.
+- `.vscode/`: generated PlatformIO files plus extension recommendations. `.vscode/extensions.json` had pre-existing local modifications before this task.
+- Existing Gitea configuration: none found before this task.
+
+## Security Assessment
+
+Confirmed current behavior:
+
+- AP password default is `changeme123`.
+- Web UI defaults are `admin/admin`.
+- `WEB_AUTH_REQUIRE_AP` and `WEB_AUTH_REQUIRE_STA` are both `true`.
+- Wi-Fi, MQTT, NTP, and web-auth credentials are stored through ESP32 Preferences in namespace `dd3cfg`.
+- The source does not implement LoRa cryptographic authentication; it uses CRC, configured sender/receiver IDs, batch IDs, and ACK rate limiting.
+- The source contains no OTA implementation or firmware signing path.
+- No project configuration for secure boot or flash encryption was found.
+
+Recommendations are in `docs/engineering/risk-register.md`; authentication, encryption, secure boot, flash encryption, OTA signing, and eFuse changes need a separate approved task.
@@ -0,0 +1,64 @@
+# HIL Strategy
+
+This document is design-only. Do not flash hardware or run destructive hardware operations from the non-hardware verifier.
+
+## Goals
+
+- Exercise one sender board and one receiver board as a pair.
+- Capture serial logs from both boards.
+- Configure serial ports, baud rates, reset controls, and power controls outside source control.
+- Simulate a DD3 smart meter over UART.
+- Use a test MQTT broker and controlled Wi-Fi conditions.
+- Optionally inject SD faults.
+
+## Proposed Stack
+
+- Python `pytest` for orchestration.
+- `pyserial` for sender, receiver, and smart-meter simulator UARTs.
+- A local MQTT broker such as Mosquitto in a test container.
+- Runner-side power/reset control through a safe abstraction, for example USB relay, programmable hub, or lab PSU API.
+- Local configuration in ignored files such as `hil_tests/config.local.yaml`.
+
+## Hardware Topologies
+
+| Topology | Purpose |
+|---|---|
+| One sender board | Sender boot, time bootstrap request, meter parsing, backlog behavior, power telemetry. |
+| One receiver board | AP/STA web behavior, MQTT publish behavior, SD behavior, malformed packet handling where injectable. |
+| Sender + receiver pair | Normal LoRa transfer, ACK behavior, duplicate handling, long soak. |
+| Receiver + smart-meter simulator | Receiver-only paths are limited; most meter simulation is sender-side. |
+
+## Test Scenarios
+
+| Scenario | Required equipment | Assertions |
+|---|---|---|
+| Time bootstrap | Sender + receiver | Sender stays sync-only before valid ACK and starts normal flow after valid ACK time. |
+| Normal batch transfer | Pair + meter simulator + MQTT broker | Receiver ACKs, publishes MQTT JSON, updates last batch, and logs CSV if SD enabled. |
+| ACK loss | Pair with RF shield/control or receiver ACK suppression build fixture | Sender retries within `BATCH_MAX_RETRIES` and preserves stop-and-wait behavior. |
+| Duplicate batches | Pair with induced ACK loss | Receiver ACKs duplicates but suppresses duplicate publish/log. |
+| Malformed and truncated LoRa packets | Receiver plus packet injector fixture | Receiver rejects without ACK, SD, MQTT, or web updates. |
+| Sender backlog | Sender + receiver + meter simulator | Backlog drains with single inflight batch and no queue overflow beyond policy. |
+| Receiver restart | Pair with reset control | Sender recovers after receiver reboot and resumes ACKed transfers. |
+| Meter failure | Sender + UART simulator | Fault counters increase according to stale/failure semantics. |
+| Meter timestamp jumps | Sender + UART simulator | Jump is detected, counted once per event, and timestamp anchor recovers. |
+| MQTT failure | Receiver + broker control | Receiver continues LoRa ACK path and reconnects MQTT without corrupting state. |
+| Wi-Fi failure and AP fallback | Receiver + Wi-Fi control | AP fallback starts, saved config retry returns to STA when Wi-Fi recovers. |
+| Missing or full SD | Receiver + SD fault fixture | Firmware continues without crashing; history endpoints report errors. |
+| Repeated resets | Pair + reset control | No persistent config corruption; role detection remains stable. |
+| Watchdog recovery | Pair + fault injection | Watchdog resets are visible and system returns to expected role behavior. |
+| Heap and stack regression | Pair with telemetry parsing | Minimum heap/stack stay above agreed thresholds during web/history/LoRa load. |
+| Long-duration soak | Pair + broker + meter simulator + SD | No hangs, reset storms, queue runaway, or publish/log drift over target duration. |
+
+## Configuration Rules
+
+- Keep local serial ports, Wi-Fi SSIDs, MQTT credentials, power-control endpoints, and board IDs out of source control.
+- Use ignored local config files under `hil_tests/`.
+- Commit only sanitized examples that contain fake ports, fake hosts, and no usable secrets.
+- HIL runners must be labeled separately from non-hardware CI runners and must not run untrusted pull-request code.
+
+## Safety Gates
+
+- Non-hardware CI must never flash boards.
+- HIL jobs must require explicit labels and a trusted branch or manual approval.
+- Power-cycle operations must validate configured device paths before acting.
+- Firmware flashing, eFuse changes, secure boot, flash encryption, and OTA signing are outside this harness task.
@@ -0,0 +1,60 @@
+# Risk Register
+
+Assessment date: 2026-07-13
+
+## Verified Defects
+
+| ID | Finding | Evidence | Impact | Next action |
+|---|---|---|---|---|
+| VD-001 | First-boot web credential behavior is documented inconsistently. | `include/config.h` says first-boot AP forces password change; `src/web_server.cpp` serves `/wifi` behind current credentials and does not enforce a first-boot password change. | Operators may deploy with default `admin/admin` longer than intended. | Design an explicit credential-onboarding flow in a separate security task. |
+| VD-002 | `Requirements.md` referenced payload schema v3 while code, README, and tests use schema v4. | `payload_codec.cpp` uses `kSchema = 4`; `test_payload_codec` validates schema v4 vectors. | Contract documentation could mislead future protocol work. | Updated schema references to v4 in this harness task. |
+| VD-003 | `docs/TESTS.md` referenced old `lilygo-t3...` environments. | Current `platformio.ini` has `production`, `debug`, and `test`. | Developers could run non-existent verification commands. | Replaced with current compile-only and on-device commands. |
+| VD-004 | PlatformIO static analysis currently fails outside repository-owned firmware logic. | Default `pio check -e production --fail-on-defect=high` fails when cppcheck reaches `toolchain-xtensa-esp32/.../bits/c++config.h:520`; `--skip-packages` avoids that header but still reports high defects under `.pio/libdeps/production/ArduinoJson/.../Variant/ConverterImpl.hpp`. | The canonical verification command is strict and correctly returns non-zero until analyzer scope/policy is explicitly decided. | Resolve in a separate tooling task; do not silently add skips, suppressions, or dependency changes just to obtain a passing check. |
+
+## Probable Risks
+
+| ID | Finding | Evidence | Impact | Next action |
+|---|---|---|---|---|
+| PR-001 | Direct `lib_deps` are version-ranged, not exact. | `platformio.ini` uses `^` ranges. | New installs may resolve different library versions. | Apply exact pins in a separate dependency-only commit after review. |
+| PR-002 | PlatformIO Core version is not locked by the repository. | Local `pio --version` is `6.1.18`; CI installs the expected version but local shells may differ. | Local and CI behavior can drift. | Keep expected core version documented and update only in a dedicated tooling task. |
+| PR-003 | Static analysis coverage depends on PlatformIO's available check tools. | `pio check` is available, but tool availability can vary by runner cache/network. | CI may fail for tool setup rather than firmware defects. | Keep output visible; add pinned analyzer packages only if needed later. |
+| PR-004 | `.vscode/extensions.json` had pre-existing local modifications. | `git status --short` reported it modified before this task. | Review may mix user-local editor changes with harness work if committed together. | Keep it out of this task unless the user explicitly asks. |
+
+## Missing Tests
+
+| ID | Gap | Existing partial coverage | Next action |
+|---|---|---|---|
+| MT-001 | No native host test environment yet. | Transport logic is close to pure logic but still includes `Arduino.h`. | Follow staged extraction plan in `docs/engineering/test-matrix.md`. |
+| MT-002 | ACK sender-side validation is not directly unit-tested as production code. | Some ACK-related behavior is documented and indirectly covered by compile checks. | Extract ACK parse/validation into a small pure helper, then add native or embedded tests. |
+| MT-003 | Retry timeout calculation is not directly unit-tested. | Sender stats and constants exist; behavior is runtime-coupled. | Extract timeout calculation before adding tests. |
+| MT-004 | SD history parser is not covered by a dedicated test suite. | Web input and CSV contracts are documented; compile checks cover buildability. | Extract current/legacy CSV line parsing to a pure helper. |
+| MT-005 | Web POST validation is not covered beyond helper sanitizers. | `test_html_escape` covers escaping, URL encoding, and device ID sanitization. | Add focused tests when web parsing is extracted. |
+
+## Missing Documentation
+
+| ID | Gap | Next action |
+|---|---|---|
+| MD-001 | No measured hardware power baseline is recorded in this branch. | Record current, light-sleep-off, and light-sleep-on measurements in a separate HIL/power task. |
+| MD-002 | No Gitea runner inventory is recorded. | Document runner labels and container prerequisites in CI operations notes once the runner exists. |
+| MD-003 | No sanitized HIL local config example exists yet. | Add `hil_tests/config.example.yaml` when pytest harness scaffolding starts. |
+
+## Security Decisions
+
+| ID | Current decision | Status | Recommendation |
+|---|---|---|---|
+| SD-001 | AP password default is `changeme123`. | Confirmed. | Replace with per-device provisioning in a separate approved task. |
+| SD-002 | Web defaults are `admin/admin`. | Confirmed. | Add forced first-boot credential change in a separate approved task. |
+| SD-003 | Credentials are stored in Preferences. | Confirmed. | Assess NVS encryption, flash encryption, and credential rotation separately. |
+| SD-004 | LoRa packets are not cryptographically authenticated. | Confirmed. | Design message authentication and replay handling separately; preserve current wire contract until approved. |
+| SD-005 | Replay protection is limited to batch IDs, duplicate tracking, sender IDs, and ACK rate limiting. | Confirmed. | Add explicit threat model before changing protocol. |
+| SD-006 | OTA, firmware signing, secure boot, and flash encryption are not configured in this project. | Confirmed by source/config search; hardware fuse state unknown. | Decide production boot-chain policy in a separate security task. |
+
+## Hardware-Dependent Questions
+
+| ID | Question | Why it matters | Proposed verification |
+|---|---|---|---|
+| HQ-001 | Does GPIO14 role selection remain stable with SD SCK reuse on receiver boot? | `main.cpp` releases the role pin before SD uses GPIO14. | HIL boot-role and SD mount tests. |
+| HQ-002 | Does chunked light sleep preserve 1 Hz meter UART capture on real meters? | Power behavior depends on UART timing and FreeRTOS scheduling. | Sender HIL with smart-meter UART simulator and real meter. |
+| HQ-003 | Are ACK windows sufficient at worst-case RSSI/SNR and duplicate retries? | LoRa airtime and receiver delay depend on RF environment. | Paired-board RF soak tests with ACK loss injection. |
+| HQ-004 | Is SD logging reliable across missing, full, and corrupt cards? | Runtime currently treats SD as optional but history depends on it. | HIL SD fault injection. |
+| HQ-005 | Are heap and stack margins acceptable during web history queries and long LoRa batches? | ESP32 memory pressure is hardware/runtime dependent. | Long-duration HIL with heap/stack telemetry. |
@@ -0,0 +1,22 @@
+{
+  "generated_on": "2026-07-13",
+  "platformio_core": "6.1.18",
+  "note": "Baseline captured after successful pio run -e production/debug/test. max_firmware_bin_bytes allows 32768 bytes of reviewable growth.",
+  "environments": {
+    "production": {
+      "firmware_bin_bytes": 1189248,
+      "firmware_elf_bytes": 14672908,
+      "max_firmware_bin_bytes": 1222016
+    },
+    "debug": {
+      "firmware_bin_bytes": 1196096,
+      "firmware_elf_bytes": 14698704,
+      "max_firmware_bin_bytes": 1228864
+    },
+    "test": {
+      "firmware_bin_bytes": 294464,
+      "firmware_elf_bytes": 4644536,
+      "max_firmware_bin_bytes": 327232
+    }
+  }
+}
@@ -0,0 +1,88 @@
+# Test Matrix
+
+Use `python tools/verify.py` as the canonical non-hardware verification command.
+
+## Non-Hardware Verification
+
+| Check | Command | Hardware | Required in CI |
+|---|---|---|---|
+| Repository, dependency, and PlatformIO policy | `python tools/check_dependencies.py` | None | Yes |
+| Text hygiene and Python syntax | `python tools/check_format.py` | None | Yes |
+| Production build | `python -m platformio run -e production` | None | Yes |
+| Debug build | `python -m platformio run -e debug` | None | Yes |
+| Test firmware build | `python -m platformio run -e test` | None | Yes |
+| Compile-only Unity suites | `python -m platformio test -e test --without-uploading --without-testing` | None | Yes |
+| Static analysis | `python -m platformio check -e production --fail-on-defect=high` | None | Yes |
+| Documentation consistency | `python tools/check_docs.py` | None | Yes |
+| Firmware size report | `python tools/check_size.py` | None after builds | Yes |
+
+## Existing Embedded Unity Suites
+
+| Suite | Primary coverage | Compile-only command | On-device command |
+|---|---|---|---|
+| `test_payload_codec` | Payload schema v4, sparse mask, varints, golden vectors | `python -m platformio test -e test --without-uploading --without-testing -f test_payload_codec` | `python -m platformio test -e test -f test_payload_codec` |
+| `test_lora_transport` | CRC16, LoRa frame build/parse, chunk reassembly | `python -m platformio test -e test --without-uploading --without-testing -f test_lora_transport` | `python -m platformio test -e test -f test_lora_transport` |
+| `test_json_codec` | MQTT JSON keys, optional keys, HA discovery fields | `python -m platformio test -e test --without-uploading --without-testing -f test_json_codec` | `python -m platformio test -e test -f test_json_codec` |
+| `test_html_escape` | HTML escaping, URL encoding, web device-ID validation | `python -m platformio test -e test --without-uploading --without-testing -f test_html_escape` | `python -m platformio test -e test -f test_html_escape` |
+| `test_refactor_smoke` | Public refactor headers and HA manufacturer guard | `python -m platformio test -e test --without-uploading --without-testing -f test_refactor_smoke` | `python -m platformio test -e test -f test_refactor_smoke` |
+| `test_meter_fault_count` | Stale-meter fault counter regression | `python -m platformio test -e test --without-uploading --without-testing -f test_meter_fault_count` | `python -m platformio test -e test -f test_meter_fault_count` |
+| `test_security_fuzz` | Malformed payload/frame/reassembly rejection | `python -m platformio test -e test --without-uploading --without-testing -f test_security_fuzz` | `python -m platformio test -e test -f test_security_fuzz` |
+
+## Hardware Requirements
+
+| Scenario | Boards | Smart meter | Wi-Fi | MQTT | SD |
+|---|---:|---:|---:|---:|---:|
+| Compile-only Unity tests | 0 | No | No | No | No |
+| On-device Unity test suite | 1 | No | No | No | No |
+| Test-mode sender or receiver smoke | 1 | No | Optional | Optional | Optional |
+| Normal sender boot and sampling | 1 sender | Yes or simulator | No | No | No |
+| Normal receiver boot and web/AP fallback | 1 receiver | No | Optional | Optional | Optional |
+| End-to-end batch transfer | 1 sender + 1 receiver | Yes or simulator | Yes | Yes | Optional |
+| SD logging and history | 1 receiver plus sender traffic | Optional | Optional | Optional | Yes |
+| Power measurement | 1 sender plus receiver ACK source | Yes or simulator | No | No | No |
+
+## Contract Traceability
+
+| Contract | Tests or checks |
+|---|---|
+| LoRa frame format | `test_lora_transport`, `test_security_fuzz`, `tools/check_docs.py` |
+| CRC validation | `test_lora_transport`, `test_security_fuzz` |
+| Chunk reassembly | `test_lora_transport`, `test_security_fuzz` |
+| Payload schema | `test_payload_codec`, `test_security_fuzz`, `tools/check_docs.py` |
+| Sender-ID validation | Compile coverage in `test_refactor_smoke`; runtime behavior documented in `README.md`; HIL planned |
+| ACK matching | Runtime behavior documented in `README.md`; HIL planned |
+| Time bootstrap | Payload sync vector in `test_payload_codec`; runtime behavior documented in `README.md`; HIL planned |
+| Retry bounds | Constants and state machine compile coverage; `test_meter_fault_count` covers one retry-adjacent regression; extraction planned |
+| Queue bounds | State machine compile coverage; extraction planned |
+| Duplicate handling | Runtime behavior documented in `README.md`; HIL planned |
+| MQTT JSON keys | `test_json_codec` |
+| Home Assistant discovery fields | `test_json_codec`, `test_refactor_smoke`, `test/check_ha_manufacturer.ps1` |
+| CSV compatibility | Documented in `README.md` and `docs/engineering/current-state.md`; parser extraction planned |
+| HTML escaping | `test_html_escape` |
+| Web input validation | `test_html_escape`; SD/history route validation extraction planned |
+
+## Native-Test Feasibility
+
+A PlatformIO `native` environment should not be introduced until pure logic is separated from Arduino headers. The best first candidates are already close to pure logic:
+
+- CRC functions and LoRa frame build/parse in `lib/dd3_transport_logic`.
+- Batch reassembly in `lib/dd3_transport_logic`.
+- Payload encoding/decoding and varints in `lib/dd3_legacy_core`.
+- HTML escaping, URL encoding, and device-ID sanitization in `lib/dd3_legacy_core`.
+- HA discovery JSON builder once `ArduinoJson` and `String` dependencies are handled.
+
+Current blockers:
+
+- Public helper headers include `Arduino.h`.
+- Several otherwise pure helpers expose Arduino `String`.
+- ArduinoJson is used directly in host-interesting code.
+- Runtime logic depends on global Arduino services: `millis`, `delay`, `Serial`, `Serial2`, `LoRa`, `WiFi`, `SD`, `Preferences`, and ESP32 FreeRTOS APIs.
+- Sender state-transition logic is still coupled to module-static globals and hardware-facing helpers.
+
+Staged extraction plan:
+
+1. Remove unnecessary `Arduino.h` includes from pure helper headers by replacing Arduino integer types with standard C/C++ headers.
+2. Add native tests for `lora_frame_logic` and `batch_reassembly_logic`.
+3. Split payload codec into a header/source pair that only needs standard C++ types, then add native golden-vector tests.
+4. Add small pure helpers for ACK parse/validation, sparse-mask timestamp reconstruction, retry timeout calculation, and CSV line parsing.
+5. Keep Arduino wrappers in firmware modules and test extracted helpers natively plus embedded compile-only suites.