Add firmware verification harness

2026-07-13 11:10:19 +02:00
parent def09160d0
commit 99c07ecda6
16 changed files with 1004 additions and 32 deletions
@@ -0,0 +1,37 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  verify:
+    name: Non-hardware verification
+    runs-on: [trusted, docker, linux]
+    container:
+      image: python:3.11-bookworm
+    timeout-minutes: 90
+    steps:
+      - name: Install runner prerequisites
+        run: |
+          apt-get update
+          apt-get install -y --no-install-recommends git ca-certificates
+          rm -rf /var/lib/apt/lists/*
+
+      - name: Checkout
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
+        with:
+          persist-credentials: false
+
+      - name: Install PlatformIO
+        run: python -m pip install --no-cache-dir platformio==6.1.18
+
+      - name: Verify
+        env:
+          PYTHONIOENCODING: utf-8
+          PYTHONUTF8: "1"
+          PIO_FORCE_COLOR: "0"
+        run: python tools/verify.py
@@ -5,3 +5,9 @@
 .vscode/ipch

 __pycache__/
+
+hil_tests/*.local.json
+hil_tests/*.local.yaml
+hil_tests/*.local.yml
+hil_tests/config.local.*
+hil_tests/secrets.local.*
@@ -0,0 +1,12 @@
+# Codex Guidance
+
+- PlatformIO is the canonical build and test tool. Keep Arduino as the framework and keep the pinned pioarduino Espressif platform unless a dependency task explicitly changes it.
+- Supported environments are `production`, `debug`, and `test`. Use `python -m platformio run -e production`, `python -m platformio run -e debug`, and `python -m platformio run -e test`.
+- The canonical non-hardware verification command is `python tools/verify.py`. It runs config checks, docs checks, dependency checks, builds, compile-only Unity tests, static analysis, and size checks where supported.
+- `src/main.cpp` stays a thin coordinator. Sender runtime belongs in `SenderStateMachine`; receiver runtime belongs in `ReceiverPipeline`.
+- Preserve sender and receiver behavior unless explicitly tasked otherwise. Preserve LoRa frame layout, ACK payloads, payload schema v4, MQTT topics and JSON keys, CSV layout, Preferences keys, role detection, timing, sleep, retry, and watchdog behavior.
+- Add focused regression tests for bug fixes. Run the smallest relevant test first, then run `python tools/verify.py` before handoff when practical.
+- Report every command executed and its result. Report checks that could not be executed and why; never claim hardware validation without hardware.
+- Do not flash hardware, require USB devices, edit generated `.pio` files, add credentials or private keys, suppress warnings/tests to pass, update dependencies incidentally, or do unrelated cleanup.
+- Documentation changes must keep `README.md`, `Requirements.md`, `docs/TESTS.md`, and `docs/engineering/` aligned with current protocol, persistence, and verification contracts.
+- Definition of done: behavior intentionally unchanged, relevant tests added or updated, all supported environments build, compile-only tests pass, docs are updated, and remaining risks are documented.
@@ -47,7 +47,7 @@ Sender state machine invariants must remain behavior-equivalent:
  - frame format: `[msg_kind][short_id_be][payload][crc16_ccitt]`.
  - reject invalid CRC/msg-kind/length.
 - Payload codec:
-  - schema `3` with `present_mask` (30-bit sparse second map).
+  - schema `4` with `present_mask` (30-bit sparse second map).
  - support `n==0` sync-request packets.
 - Time bootstrap guardrail:
  - sender must not run normal sampling/transmit until valid ACK time received.
@@ -221,7 +221,7 @@ Sender state machine invariants must remain behavior-equivalent:
 ## `lib/dd3_legacy_core/src/payload_codec.cpp`

 - `bool encode_batch(const BatchInput&, uint8_t*, size_t, size_t*)`
-  - schema v3 encoder with metadata, sparse present mask, delta coding.
+  - schema v4 encoder with metadata, sparse present mask, delta coding.
 - `bool decode_batch(const uint8_t*, size_t, BatchInput*)`
  - strict schema/magic/flags decode + bounds checks.
 - Varint primitives:
@@ -475,7 +475,7 @@ Current core orchestration requirements:
 ## 6. Rust Porting Constraints and Recommendations

 - Preserve wire compatibility first:
-  - LoRa frame byte layout, CRC16, ACK format, payload schema v3.
+  - LoRa frame byte layout, CRC16, ACK format, payload schema v4.
  - sender optimization changes must not alter payload field meanings.
 - Preserve persistent storage keys:
  - Preferences keys (`ssid`, `pass`, `mqhost`, `mqport`, `mquser`, `mqpass`, `ntp1`, `ntp2`, `webuser`, `webpass`, `valid`).
@@ -1,47 +1,98 @@
-# Legacy Unity Tests
+# Firmware Tests

-This change intentionally keeps the existing PlatformIO legacy Unity harness unchanged.
-No `platformio.ini`, CI, or test-runner configuration was modified.
+The canonical non-hardware command is:

-## Compile-Only (Legacy Gate)
-
-Use compile-only checks in environments that do not have a connected board:
-
-```powershell
-pio test -e lilygo-t3-v1-6-1-test --without-uploading --without-testing
-pio test -e lilygo-t3-v1-6-1-868-test --without-uploading --without-testing
+```bash
+python tools/verify.py
 ```

-Suite-specific compile checks:
+Current PlatformIO environments from `platformio.ini`:

-```powershell
-pio test -e lilygo-t3-v1-6-1-test --without-uploading --without-testing -f test_html_escape
-pio test -e lilygo-t3-v1-6-1-test --without-uploading --without-testing -f test_payload_codec
-pio test -e lilygo-t3-v1-6-1-test --without-uploading --without-testing -f test_lora_transport
-pio test -e lilygo-t3-v1-6-1-test --without-uploading --without-testing -f test_json_codec
-pio test -e lilygo-t3-v1-6-1-test --without-uploading --without-testing -f test_refactor_smoke
+- `production`: deployment build, serial debug off, light sleep on.
+- `debug`: serial diagnostics on, real meter and real LoRa.
+- `test`: synthetic meter samples and payload codec self-test.
+
+Payload schema: v4.
+
+## Compile-Only Tests
+
+These commands do not require USB devices or physical boards.
+
+```bash
+python -m platformio test -e test --without-uploading --without-testing
+python -m platformio test -e test --without-uploading --without-testing -f test_html_escape
+python -m platformio test -e test --without-uploading --without-testing -f test_payload_codec
+python -m platformio test -e test --without-uploading --without-testing -f test_lora_transport
+python -m platformio test -e test --without-uploading --without-testing -f test_json_codec
+python -m platformio test -e test --without-uploading --without-testing -f test_refactor_smoke
+python -m platformio test -e test --without-uploading --without-testing -f test_meter_fault_count
+python -m platformio test -e test --without-uploading --without-testing -f test_security_fuzz
 ```

-## Full On-Device Unity Run
+## On-Device Unity Tests

-When hardware is connected, run full legacy Unity tests:
+These commands require one connected LilyGO T3 v1.6.1 board. They do not require a sender/receiver pair unless the test itself is changed to exercise radio interaction.

-```powershell
-pio test -e lilygo-t3-v1-6-1-test
-pio test -e lilygo-t3-v1-6-1-868-test
+```bash
+python -m platformio test -e test
+python -m platformio test -e test -f test_html_escape
+python -m platformio test -e test -f test_payload_codec
+python -m platformio test -e test -f test_lora_transport
+python -m platformio test -e test -f test_json_codec
+python -m platformio test -e test -f test_refactor_smoke
+python -m platformio test -e test -f test_meter_fault_count
+python -m platformio test -e test -f test_security_fuzz
 ```

+Do not run upload, flash, monitor, or HIL commands from non-hardware CI.
+
 ## Suite Coverage

- `test_html_escape`: `html_escape`, `url_encode_component`, and `sanitize_device_id` edge/adversarial coverage.
- `test_payload_codec`: payload schema v3 roundtrip/reject paths and golden vectors.
- `test_lora_transport`: CRC16, frame encode/decode integrity, and chunk reassembly behavior.
- `test_json_codec`: state JSON key stability and Home Assistant discovery payload manufacturer/key stability.
- `test_refactor_smoke`: baseline include/type smoke and manufacturer constant guard, using stable public headers from `include/` (no `../../src` includes).
+| Suite | Coverage | Hardware notes |
+|---|---|---|
+| `test_payload_codec` | Payload schema v4 roundtrip/reject paths, sparse masks, sync packets, golden vectors. | Compile-only works; on-device needs one board. |
+| `test_lora_transport` | CRC16, LoRa frame encode/decode integrity, malformed frame rejection, chunk reassembly. | Compile-only works; on-device needs one board. |
+| `test_json_codec` | MQTT state JSON key stability and Home Assistant discovery field stability. | Compile-only works; no Wi-Fi/MQTT broker required. |
+| `test_html_escape` | HTML escaping, URL encoding, and web device-ID input validation. | Compile-only works; no web server required. |
+| `test_refactor_smoke` | Public sender/receiver headers and HA manufacturer constant guard. | Compile-only works. |
+| `test_meter_fault_count` | Regression for stale-meter fault counting during catch-up ticks. | Compile-only works; no smart meter required. |
+| `test_security_fuzz` | Negative/fuzz seeds for payload decode, varints, LoRa frames, and reassembly. | Compile-only works. |

-## Manufacturer Drift Guard
+## Hardware Feature Requirements

-Run the static guard script to enforce Home Assistant manufacturer wiring:
+| Check | Boards | Sender/receiver pair | Smart meter | Wi-Fi | MQTT | SD |
+|---|---:|---:|---:|---:|---:|---:|
+| Compile-only Unity tests | 0 | No | No | No | No | No |
+| On-device Unity tests | 1 | No | No | No | No | No |
+| Test-mode LoRa smoke | 2 | Yes | No | Optional | Optional | Optional |
+| Production sender sampling | 1 sender | No | Yes or UART simulator | No | No | No |
+| Production receiver AP/STA smoke | 1 receiver | No | No | Optional | Optional | Optional |
+| End-to-end production transfer | 2 | Yes | Yes or UART simulator | Yes | Yes | Optional |
+| SD logging and history | 1 receiver plus traffic | Usually | No | Optional | Optional | Yes |
+
+## Traceability
+
+| Contract | Current tests |
+|---|---|
+| LoRa frame format | `test_lora_transport`, `test_security_fuzz` |
+| CRC validation | `test_lora_transport`, `test_security_fuzz` |
+| Chunk reassembly | `test_lora_transport`, `test_security_fuzz` |
+| Payload schema | `test_payload_codec`, `test_security_fuzz` |
+| Sender-ID validation | `test_refactor_smoke` compile coverage; HIL planned for runtime rejection path |
+| ACK matching | HIL planned; runtime contract documented in `README.md` |
+| Time bootstrap | `test_payload_codec` sync vector; HIL planned for runtime bootstrap |
+| Retry bounds | `test_meter_fault_count` adjacent regression; helper extraction planned |
+| Queue bounds | `test_refactor_smoke` compile coverage; helper extraction planned |
+| Duplicate handling | HIL planned; runtime contract documented in `README.md` |
+| MQTT JSON keys | `test_json_codec` |
+| Home Assistant discovery fields | `test_json_codec`, `test_refactor_smoke`, `test/check_ha_manufacturer.ps1` |
+| CSV compatibility | HIL or extracted parser tests planned; contract documented in `README.md` |
+| HTML escaping | `test_html_escape` |
+| Web input validation | `test_html_escape`; route-level tests planned |
+
+## Additional Static Guard
+
+Run the Home Assistant manufacturer drift guard:

 ```powershell
 powershell -ExecutionPolicy Bypass -File test/check_ha_manufacturer.ps1
@@ -0,0 +1,117 @@
+# Current State
+
+Assessment date: 2026-07-13
+Branch assessed: `lora-refactor`
+Commit assessed: `def0916 refactor lora payload timing`
+
+## Architecture Verification
+
+| Assumption | Status | Evidence |
+|---|---:|---|
+| `src/main.cpp` is the thin top-level coordinator. | Verified | Initializes shared services, creates role configs, delegates `loop()` to role objects. |
+| Sender runtime is owned by `SenderStateMachine`. | Verified | `src/sender_state_machine.h/.cpp` own sender state, queueing, sync, ACK, retry, and sleep loop. |
+| Receiver runtime is owned by `ReceiverPipeline`. | Verified | `src/receiver_pipeline.h/.cpp` own receive, reassembly, ACK, duplicate, MQTT, SD, and web update flow. |
+| One firmware image supports both roles. | Verified | `detect_role()` selects role at boot; `loop()` dispatches sender or receiver in one image. |
+| PlatformIO is the canonical build system. | Verified | `platformio.ini` defines all firmware environments. |
+| Arduino remains the framework. | Verified | `[env] framework = arduino`. |
+| pioarduino Espressif platform remains pinned. | Verified | Platform URL is `https://github.com/pioarduino/platform-espressif32/releases/download/51.03.07/platform-espressif32.zip`. |
+| Existing environments include `production`, `debug`, and `test`. | Verified | `platformio.ini` has exactly those project environments. |
+| Payload schema v4 is current. | Verified | `lib/dd3_legacy_core/src/payload_codec.cpp` uses `kSchema = 4`; payload tests use schema v4 vectors. |
+| Existing golden vectors remain relevant. | Verified by inspection | `test/test_payload_codec` contains schema v4 golden vectors. |
+| Power behavior was changed by this task. | Not changed | This harness task did not edit firmware source. |
+
+## Current Build Configuration
+
+- PlatformIO Core observed locally: `6.1.18`.
+- Expected PlatformIO Core for CI and reproducible verification: `6.1.18`.
+- PlatformIO emits a local warning that multiple/obsolete cores may be installed: `Obsolete PIO Core v6.1.18 is used (previous was 6.1.19)`.
+- Board: `ttgo-lora32-v1`.
+- Framework: Arduino.
+- Environments:
+  - `production`: serial debug off, light sleep on.
+  - `debug`: serial diagnostics on, real meter and real LoRa.
+- `test`: `ENABLE_TEST_MODE` and `PAYLOAD_CODEC_TEST`.
+
+## Verification Snapshot
+
+- `pio run -e production`: passed; RAM `95588` bytes, flash `1182653` bytes.
+- `pio run -e debug`: passed; RAM `95628` bytes, flash `1189501` bytes.
+- `pio run -e test`: passed; RAM `95060` bytes, flash `1182177` bytes.
+- `pio test -e test --without-uploading --without-testing`: passed compile-only for all seven embedded Unity suites.
+- `python tools/check_size.py`: passed with current artifact baselines in `docs/engineering/size-baseline.json`.
+- `python tools/verify.py`: failed only at static analysis; `pio check -e production --fail-on-defect=high` exits 1 because cppcheck fails on the Xtensa C++ toolchain package header.
+- `pio check -e production --fail-on-defect=high --skip-packages`: also failed because cppcheck reports high defects in third-party ArduinoJson headers under `.pio/libdeps`.
+- The strict static-analysis failure is intentionally preserved until analyzer scope and third-party dependency policy are reviewed.
+
+## Resolved Dependency Baseline
+
+Observed with `pio pkg list` after forcing UTF-8 console output:
+
+| Dependency | Configured requirement | Resolved version |
+|---|---|---:|
+| pioarduino `platform-espressif32` | release URL `51.03.07` | `51.3.6` |
+| Arduino ESP32 framework | platform-provided | `3.0.7` |
+| ESP32 Arduino libs | platform-provided | `5.1.0+sha.632e0c2a9f` |
+| `sandeepmistry/LoRa` | `^0.8.0` | `0.8.0` |
+| `bblanchon/ArduinoJson` | `^6.21.5` | `6.21.6` |
+| `adafruit/Adafruit SSD1306` | `^2.5.9` | `2.5.17` |
+| `adafruit/Adafruit GFX Library` | `^1.11.9` | `1.12.6` |
+| `Adafruit BusIO` | transitive | `1.17.4` |
+| `knolleary/PubSubClient` | `^2.8` | `2.8.0` |
+| `throwtheswitch/Unity` | `^2.6.1` | `2.6.1` |
+
+Proposed exact direct pins for a separate dependency-only commit:
+
+```ini
+sandeepmistry/LoRa@0.8.0
+bblanchon/ArduinoJson@6.21.6
+adafruit/Adafruit SSD1306@2.5.17
+adafruit/Adafruit GFX Library@1.12.6
+knolleary/PubSubClient@2.8.0
+throwtheswitch/Unity@2.6.1
+```
+
+## Hardware and Persistence Contracts
+
+- Role pin: `GPIO14`; HIGH is sender, LOW is receiver.
+- OLED control pin: `GPIO13`.
+- LoRa defaults in `include/config.h`: 868 MHz, SF12, BW 125 kHz, coding rate 4/5, sync word `0x34`.
+- Configured sender short ID: `0x6540`; configured receiver short ID: `0x7EB4`.
+- Preferences namespace: `dd3cfg`.
+- Preferences keys: `ssid`, `pass`, `mqhost`, `mqport`, `mquser`, `mqpass`, `ntp1`, `ntp2`, `webuser`, `webpass`, `valid`.
+- CSV current header: `ts_utc,ts_hms_local,p_w,p1_w,p2_w,p3_w,e_kwh,bat_v,bat_pct,rssi,snr,err_m,err_d,err_tx,err_last`.
+
+## Protocol Contracts
+
+- LoRa frame: `[msg_kind:1][device_short_id_be:2][payload...][crc16_ccitt_be:2]`.
+- `LoraMsgKind`: `BatchUp=0`, `AckDown=1`.
+- `BatchUp` chunk header: `[batch_id_le:2][chunk_index:1][chunk_count:1][total_len_le:2][chunk_payload...]`.
+- Payload codec: schema v4, magic `0xDDB3`, 30-bit `present_mask`, `n == 0` sync-request support.
+- `AckDown` payload length: 7 bytes, `[flags:1][batch_id_be:2][epoch_utc_be:4]`.
+- ACK time is accepted only when `flags bit0` is set and epoch is at least `MIN_ACCEPTED_EPOCH_UTC`.
+- MQTT state topic: `smartmeter/<device_id>/state`.
+- MQTT fault topic: `smartmeter/<device_id>/faults`.
+- Test MQTT topic: `smartmeter/<device_id>/test`.
+- Home Assistant discovery topic: `homeassistant/sensor/<device_id>/<key>/config`.
+
+## Repository Shape
+
+- `lib/dd3_legacy_core`: shared legacy logic for data model, HTML utilities, JSON codec, and payload codec.
+- `lib/dd3_transport_logic`: pure-ish transport helpers for LoRa frame, HA discovery JSON, and batch reassembly.
+- `test/`: seven embedded Unity suites, compile-only runnable without a board.
+- `.vscode/`: generated PlatformIO files plus extension recommendations. `.vscode/extensions.json` had pre-existing local modifications before this task.
+- Existing Gitea configuration: none found before this task.
+
+## Security Assessment
+
+Confirmed current behavior:
+
+- AP password default is `changeme123`.
+- Web UI defaults are `admin/admin`.
+- `WEB_AUTH_REQUIRE_AP` and `WEB_AUTH_REQUIRE_STA` are both `true`.
+- Wi-Fi, MQTT, NTP, and web-auth credentials are stored through ESP32 Preferences in namespace `dd3cfg`.
+- The source does not implement LoRa cryptographic authentication; it uses CRC, configured sender/receiver IDs, batch IDs, and ACK rate limiting.
+- The source contains no OTA implementation or firmware signing path.
+- No project configuration for secure boot or flash encryption was found.
+
+Recommendations are in `docs/engineering/risk-register.md`; authentication, encryption, secure boot, flash encryption, OTA signing, and eFuse changes need a separate approved task.
@@ -0,0 +1,64 @@
+# HIL Strategy
+
+This document is design-only. Do not flash hardware or run destructive hardware operations from the non-hardware verifier.
+
+## Goals
+
+- Exercise one sender board and one receiver board as a pair.
+- Capture serial logs from both boards.
+- Configure serial ports, baud rates, reset controls, and power controls outside source control.
+- Simulate a DD3 smart meter over UART.
+- Use a test MQTT broker and controlled Wi-Fi conditions.
+- Optionally inject SD faults.
+
+## Proposed Stack
+
+- Python `pytest` for orchestration.
+- `pyserial` for sender, receiver, and smart-meter simulator UARTs.
+- A local MQTT broker such as Mosquitto in a test container.
+- Runner-side power/reset control through a safe abstraction, for example USB relay, programmable hub, or lab PSU API.
+- Local configuration in ignored files such as `hil_tests/config.local.yaml`.
+
+## Hardware Topologies
+
+| Topology | Purpose |
+|---|---|
+| One sender board | Sender boot, time bootstrap request, meter parsing, backlog behavior, power telemetry. |
+| One receiver board | AP/STA web behavior, MQTT publish behavior, SD behavior, malformed packet handling where injectable. |
+| Sender + receiver pair | Normal LoRa transfer, ACK behavior, duplicate handling, long soak. |
+| Receiver + smart-meter simulator | Receiver-only paths are limited; most meter simulation is sender-side. |
+
+## Test Scenarios
+
+| Scenario | Required equipment | Assertions |
+|---|---|---|
+| Time bootstrap | Sender + receiver | Sender stays sync-only before valid ACK and starts normal flow after valid ACK time. |
+| Normal batch transfer | Pair + meter simulator + MQTT broker | Receiver ACKs, publishes MQTT JSON, updates last batch, and logs CSV if SD enabled. |
+| ACK loss | Pair with RF shield/control or receiver ACK suppression build fixture | Sender retries within `BATCH_MAX_RETRIES` and preserves stop-and-wait behavior. |
+| Duplicate batches | Pair with induced ACK loss | Receiver ACKs duplicates but suppresses duplicate publish/log. |
+| Malformed and truncated LoRa packets | Receiver plus packet injector fixture | Receiver rejects without ACK, SD, MQTT, or web updates. |
+| Sender backlog | Sender + receiver + meter simulator | Backlog drains with single inflight batch and no queue overflow beyond policy. |
+| Receiver restart | Pair with reset control | Sender recovers after receiver reboot and resumes ACKed transfers. |
+| Meter failure | Sender + UART simulator | Fault counters increase according to stale/failure semantics. |
+| Meter timestamp jumps | Sender + UART simulator | Jump is detected, counted once per event, and timestamp anchor recovers. |
+| MQTT failure | Receiver + broker control | Receiver continues LoRa ACK path and reconnects MQTT without corrupting state. |
+| Wi-Fi failure and AP fallback | Receiver + Wi-Fi control | AP fallback starts, saved config retry returns to STA when Wi-Fi recovers. |
+| Missing or full SD | Receiver + SD fault fixture | Firmware continues without crashing; history endpoints report errors. |
+| Repeated resets | Pair + reset control | No persistent config corruption; role detection remains stable. |
+| Watchdog recovery | Pair + fault injection | Watchdog resets are visible and system returns to expected role behavior. |
+| Heap and stack regression | Pair with telemetry parsing | Minimum heap/stack stay above agreed thresholds during web/history/LoRa load. |
+| Long-duration soak | Pair + broker + meter simulator + SD | No hangs, reset storms, queue runaway, or publish/log drift over target duration. |
+
+## Configuration Rules
+
+- Keep local serial ports, Wi-Fi SSIDs, MQTT credentials, power-control endpoints, and board IDs out of source control.
+- Use ignored local config files under `hil_tests/`.
+- Commit only sanitized examples that contain fake ports, fake hosts, and no usable secrets.
+- HIL runners must be labeled separately from non-hardware CI runners and must not run untrusted pull-request code.
+
+## Safety Gates
+
+- Non-hardware CI must never flash boards.
+- HIL jobs must require explicit labels and a trusted branch or manual approval.
+- Power-cycle operations must validate configured device paths before acting.
+- Firmware flashing, eFuse changes, secure boot, flash encryption, and OTA signing are outside this harness task.
@@ -0,0 +1,60 @@
+# Risk Register
+
+Assessment date: 2026-07-13
+
+## Verified Defects
+
+| ID | Finding | Evidence | Impact | Next action |
+|---|---|---|---|---|
+| VD-001 | First-boot web credential behavior is documented inconsistently. | `include/config.h` says first-boot AP forces password change; `src/web_server.cpp` serves `/wifi` behind current credentials and does not enforce a first-boot password change. | Operators may deploy with default `admin/admin` longer than intended. | Design an explicit credential-onboarding flow in a separate security task. |
+| VD-002 | `Requirements.md` referenced payload schema v3 while code, README, and tests use schema v4. | `payload_codec.cpp` uses `kSchema = 4`; `test_payload_codec` validates schema v4 vectors. | Contract documentation could mislead future protocol work. | Updated schema references to v4 in this harness task. |
+| VD-003 | `docs/TESTS.md` referenced old `lilygo-t3...` environments. | Current `platformio.ini` has `production`, `debug`, and `test`. | Developers could run non-existent verification commands. | Replaced with current compile-only and on-device commands. |
+| VD-004 | PlatformIO static analysis currently fails outside repository-owned firmware logic. | Default `pio check -e production --fail-on-defect=high` fails when cppcheck reaches `toolchain-xtensa-esp32/.../bits/c++config.h:520`; `--skip-packages` avoids that header but still reports high defects under `.pio/libdeps/production/ArduinoJson/.../Variant/ConverterImpl.hpp`. | The canonical verification command is strict and correctly returns non-zero until analyzer scope/policy is explicitly decided. | Resolve in a separate tooling task; do not silently add skips, suppressions, or dependency changes just to obtain a passing check. |
+
+## Probable Risks
+
+| ID | Finding | Evidence | Impact | Next action |
+|---|---|---|---|---|
+| PR-001 | Direct `lib_deps` are version-ranged, not exact. | `platformio.ini` uses `^` ranges. | New installs may resolve different library versions. | Apply exact pins in a separate dependency-only commit after review. |
+| PR-002 | PlatformIO Core version is not locked by the repository. | Local `pio --version` is `6.1.18`; CI installs the expected version but local shells may differ. | Local and CI behavior can drift. | Keep expected core version documented and update only in a dedicated tooling task. |
+| PR-003 | Static analysis coverage depends on PlatformIO's available check tools. | `pio check` is available, but tool availability can vary by runner cache/network. | CI may fail for tool setup rather than firmware defects. | Keep output visible; add pinned analyzer packages only if needed later. |
+| PR-004 | `.vscode/extensions.json` had pre-existing local modifications. | `git status --short` reported it modified before this task. | Review may mix user-local editor changes with harness work if committed together. | Keep it out of this task unless the user explicitly asks. |
+
+## Missing Tests
+
+| ID | Gap | Existing partial coverage | Next action |
+|---|---|---|---|
+| MT-001 | No native host test environment yet. | Transport logic is close to pure logic but still includes `Arduino.h`. | Follow staged extraction plan in `docs/engineering/test-matrix.md`. |
+| MT-002 | ACK sender-side validation is not directly unit-tested as production code. | Some ACK-related behavior is documented and indirectly covered by compile checks. | Extract ACK parse/validation into a small pure helper, then add native or embedded tests. |
+| MT-003 | Retry timeout calculation is not directly unit-tested. | Sender stats and constants exist; behavior is runtime-coupled. | Extract timeout calculation before adding tests. |
+| MT-004 | SD history parser is not covered by a dedicated test suite. | Web input and CSV contracts are documented; compile checks cover buildability. | Extract current/legacy CSV line parsing to a pure helper. |
+| MT-005 | Web POST validation is not covered beyond helper sanitizers. | `test_html_escape` covers escaping, URL encoding, and device ID sanitization. | Add focused tests when web parsing is extracted. |
+
+## Missing Documentation
+
+| ID | Gap | Next action |
+|---|---|---|
+| MD-001 | No measured hardware power baseline is recorded in this branch. | Record current, light-sleep-off, and light-sleep-on measurements in a separate HIL/power task. |
+| MD-002 | No Gitea runner inventory is recorded. | Document runner labels and container prerequisites in CI operations notes once the runner exists. |
+| MD-003 | No sanitized HIL local config example exists yet. | Add `hil_tests/config.example.yaml` when pytest harness scaffolding starts. |
+
+## Security Decisions
+
+| ID | Current decision | Status | Recommendation |
+|---|---|---|---|
+| SD-001 | AP password default is `changeme123`. | Confirmed. | Replace with per-device provisioning in a separate approved task. |
+| SD-002 | Web defaults are `admin/admin`. | Confirmed. | Add forced first-boot credential change in a separate approved task. |
+| SD-003 | Credentials are stored in Preferences. | Confirmed. | Assess NVS encryption, flash encryption, and credential rotation separately. |
+| SD-004 | LoRa packets are not cryptographically authenticated. | Confirmed. | Design message authentication and replay handling separately; preserve current wire contract until approved. |
+| SD-005 | Replay protection is limited to batch IDs, duplicate tracking, sender IDs, and ACK rate limiting. | Confirmed. | Add explicit threat model before changing protocol. |
+| SD-006 | OTA, firmware signing, secure boot, and flash encryption are not configured in this project. | Confirmed by source/config search; hardware fuse state unknown. | Decide production boot-chain policy in a separate security task. |
+
+## Hardware-Dependent Questions
+
+| ID | Question | Why it matters | Proposed verification |
+|---|---|---|---|
+| HQ-001 | Does GPIO14 role selection remain stable with SD SCK reuse on receiver boot? | `main.cpp` releases the role pin before SD uses GPIO14. | HIL boot-role and SD mount tests. |
+| HQ-002 | Does chunked light sleep preserve 1 Hz meter UART capture on real meters? | Power behavior depends on UART timing and FreeRTOS scheduling. | Sender HIL with smart-meter UART simulator and real meter. |
+| HQ-003 | Are ACK windows sufficient at worst-case RSSI/SNR and duplicate retries? | LoRa airtime and receiver delay depend on RF environment. | Paired-board RF soak tests with ACK loss injection. |
+| HQ-004 | Is SD logging reliable across missing, full, and corrupt cards? | Runtime currently treats SD as optional but history depends on it. | HIL SD fault injection. |
+| HQ-005 | Are heap and stack margins acceptable during web history queries and long LoRa batches? | ESP32 memory pressure is hardware/runtime dependent. | Long-duration HIL with heap/stack telemetry. |
@@ -0,0 +1,22 @@
+{
+  "generated_on": "2026-07-13",
+  "platformio_core": "6.1.18",
+  "note": "Baseline captured after successful pio run -e production/debug/test. max_firmware_bin_bytes allows 32768 bytes of reviewable growth.",
+  "environments": {
+    "production": {
+      "firmware_bin_bytes": 1189248,
+      "firmware_elf_bytes": 14672908,
+      "max_firmware_bin_bytes": 1222016
+    },
+    "debug": {
+      "firmware_bin_bytes": 1196096,
+      "firmware_elf_bytes": 14698704,
+      "max_firmware_bin_bytes": 1228864
+    },
+    "test": {
+      "firmware_bin_bytes": 294464,
+      "firmware_elf_bytes": 4644536,
+      "max_firmware_bin_bytes": 327232
+    }
+  }
+}
@@ -0,0 +1,88 @@
+# Test Matrix
+
+Use `python tools/verify.py` as the canonical non-hardware verification command.
+
+## Non-Hardware Verification
+
+| Check | Command | Hardware | Required in CI |
+|---|---|---|---|
+| Repository, dependency, and PlatformIO policy | `python tools/check_dependencies.py` | None | Yes |
+| Text hygiene and Python syntax | `python tools/check_format.py` | None | Yes |
+| Production build | `python -m platformio run -e production` | None | Yes |
+| Debug build | `python -m platformio run -e debug` | None | Yes |
+| Test firmware build | `python -m platformio run -e test` | None | Yes |
+| Compile-only Unity suites | `python -m platformio test -e test --without-uploading --without-testing` | None | Yes |
+| Static analysis | `python -m platformio check -e production --fail-on-defect=high` | None | Yes |
+| Documentation consistency | `python tools/check_docs.py` | None | Yes |
+| Firmware size report | `python tools/check_size.py` | None after builds | Yes |
+
+## Existing Embedded Unity Suites
+
+| Suite | Primary coverage | Compile-only command | On-device command |
+|---|---|---|---|
+| `test_payload_codec` | Payload schema v4, sparse mask, varints, golden vectors | `python -m platformio test -e test --without-uploading --without-testing -f test_payload_codec` | `python -m platformio test -e test -f test_payload_codec` |
+| `test_lora_transport` | CRC16, LoRa frame build/parse, chunk reassembly | `python -m platformio test -e test --without-uploading --without-testing -f test_lora_transport` | `python -m platformio test -e test -f test_lora_transport` |
+| `test_json_codec` | MQTT JSON keys, optional keys, HA discovery fields | `python -m platformio test -e test --without-uploading --without-testing -f test_json_codec` | `python -m platformio test -e test -f test_json_codec` |
+| `test_html_escape` | HTML escaping, URL encoding, web device-ID validation | `python -m platformio test -e test --without-uploading --without-testing -f test_html_escape` | `python -m platformio test -e test -f test_html_escape` |
+| `test_refactor_smoke` | Public refactor headers and HA manufacturer guard | `python -m platformio test -e test --without-uploading --without-testing -f test_refactor_smoke` | `python -m platformio test -e test -f test_refactor_smoke` |
+| `test_meter_fault_count` | Stale-meter fault counter regression | `python -m platformio test -e test --without-uploading --without-testing -f test_meter_fault_count` | `python -m platformio test -e test -f test_meter_fault_count` |
+| `test_security_fuzz` | Malformed payload/frame/reassembly rejection | `python -m platformio test -e test --without-uploading --without-testing -f test_security_fuzz` | `python -m platformio test -e test -f test_security_fuzz` |
+
+## Hardware Requirements
+
+| Scenario | Boards | Smart meter | Wi-Fi | MQTT | SD |
+|---|---:|---:|---:|---:|---:|
+| Compile-only Unity tests | 0 | No | No | No | No |
+| On-device Unity test suite | 1 | No | No | No | No |
+| Test-mode sender or receiver smoke | 1 | No | Optional | Optional | Optional |
+| Normal sender boot and sampling | 1 sender | Yes or simulator | No | No | No |
+| Normal receiver boot and web/AP fallback | 1 receiver | No | Optional | Optional | Optional |
+| End-to-end batch transfer | 1 sender + 1 receiver | Yes or simulator | Yes | Yes | Optional |
+| SD logging and history | 1 receiver plus sender traffic | Optional | Optional | Optional | Yes |
+| Power measurement | 1 sender plus receiver ACK source | Yes or simulator | No | No | No |
+
+## Contract Traceability
+
+| Contract | Tests or checks |
+|---|---|
+| LoRa frame format | `test_lora_transport`, `test_security_fuzz`, `tools/check_docs.py` |
+| CRC validation | `test_lora_transport`, `test_security_fuzz` |
+| Chunk reassembly | `test_lora_transport`, `test_security_fuzz` |
+| Payload schema | `test_payload_codec`, `test_security_fuzz`, `tools/check_docs.py` |
+| Sender-ID validation | Compile coverage in `test_refactor_smoke`; runtime behavior documented in `README.md`; HIL planned |
+| ACK matching | Runtime behavior documented in `README.md`; HIL planned |
+| Time bootstrap | Payload sync vector in `test_payload_codec`; runtime behavior documented in `README.md`; HIL planned |
+| Retry bounds | Constants and state machine compile coverage; `test_meter_fault_count` covers one retry-adjacent regression; extraction planned |
+| Queue bounds | State machine compile coverage; extraction planned |
+| Duplicate handling | Runtime behavior documented in `README.md`; HIL planned |
+| MQTT JSON keys | `test_json_codec` |
+| Home Assistant discovery fields | `test_json_codec`, `test_refactor_smoke`, `test/check_ha_manufacturer.ps1` |
+| CSV compatibility | Documented in `README.md` and `docs/engineering/current-state.md`; parser extraction planned |
+| HTML escaping | `test_html_escape` |
+| Web input validation | `test_html_escape`; SD/history route validation extraction planned |
+
+## Native-Test Feasibility
+
+A PlatformIO `native` environment should not be introduced until pure logic is separated from Arduino headers. The best first candidates are already close to pure logic:
+
+- CRC functions and LoRa frame build/parse in `lib/dd3_transport_logic`.
+- Batch reassembly in `lib/dd3_transport_logic`.
+- Payload encoding/decoding and varints in `lib/dd3_legacy_core`.
+- HTML escaping, URL encoding, and device-ID sanitization in `lib/dd3_legacy_core`.
+- HA discovery JSON builder once `ArduinoJson` and `String` dependencies are handled.
+
+Current blockers:
+
+- Public helper headers include `Arduino.h`.
+- Several otherwise pure helpers expose Arduino `String`.
+- ArduinoJson is used directly in host-interesting code.
+- Runtime logic depends on global Arduino services: `millis`, `delay`, `Serial`, `Serial2`, `LoRa`, `WiFi`, `SD`, `Preferences`, and ESP32 FreeRTOS APIs.
+- Sender state-transition logic is still coupled to module-static globals and hardware-facing helpers.
+
+Staged extraction plan:
+
+1. Remove unnecessary `Arduino.h` includes from pure helper headers by replacing Arduino integer types with standard C/C++ headers.
+2. Add native tests for `lora_frame_logic` and `batch_reassembly_logic`.
+3. Split payload codec into a header/source pair that only needs standard C++ types, then add native golden-vector tests.
+4. Add small pure helpers for ACK parse/validation, sparse-mask timestamp reconstruction, retry timeout calculation, and CSV line parsing.
+5. Keep Arduino wrappers in firmware modules and test extracted helpers natively plus embedded compile-only suites.
@@ -0,0 +1,21 @@
+# HIL Tests
+
+This directory is reserved for a future pytest-based hardware-in-the-loop suite.
+
+Planned capabilities:
+
+- sender and receiver serial capture;
+- configurable serial ports and baud rates;
+- controlled reset and power-cycle adapters;
+- UART smart-meter simulation;
+- test MQTT broker integration;
+- controlled Wi-Fi outage and AP fallback checks;
+- optional SD fault injection.
+
+Keep hardware configuration out of source control. Use ignored files such as:
+
+- `hil_tests/config.local.yaml`
+- `hil_tests/ports.local.json`
+- `hil_tests/secrets.local.yaml`
+
+Do not run HIL against untrusted pull-request code on a hardware runner.
@@ -0,0 +1,137 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import configparser
+import os
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+if hasattr(sys.stdout, "reconfigure"):
+    sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+if hasattr(sys.stderr, "reconfigure"):
+    sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+EXPECTED_PLATFORMIO_CORE = "6.1.18"
+EXPECTED_PLATFORM = "https://github.com/pioarduino/platform-espressif32/releases/download/51.03.07/platform-espressif32.zip"
+EXPECTED_ENVS = {"production", "debug", "test"}
+EXPECTED_BOARD = "ttgo-lora32-v1"
+EXPECTED_FRAMEWORK = "arduino"
+PROPOSED_EXACT_PINS = {
+    "sandeepmistry/LoRa": "0.8.0",
+    "bblanchon/ArduinoJson": "6.21.6",
+    "adafruit/Adafruit SSD1306": "2.5.17",
+    "adafruit/Adafruit GFX Library": "1.12.6",
+    "knolleary/PubSubClient": "2.8.0",
+    "throwtheswitch/Unity": "2.6.1",
+}
+
+
+def command_text(command: list[str]) -> str:
+    return " ".join(f'"{part}"' if any(ch.isspace() for ch in part) else part for part in command)
+
+
+def run(command: list[str]) -> subprocess.CompletedProcess[str]:
+    env = os.environ.copy()
+    env["PYTHONIOENCODING"] = "utf-8"
+    env["PYTHONUTF8"] = "1"
+    env["PIO_FORCE_COLOR"] = "0"
+    print(f"$ {command_text(command)}", flush=True)
+    result = subprocess.run(
+        command,
+        cwd=ROOT,
+        env=env,
+        text=True,
+        encoding="utf-8",
+        errors="replace",
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT,
+    )
+    print(result.stdout, end="")
+    return result
+
+
+def split_multiline(value: str) -> list[str]:
+    return [line.strip() for line in value.splitlines() if line.strip()]
+
+
+def parse_platformio() -> configparser.RawConfigParser:
+    parser = configparser.RawConfigParser()
+    parser.optionxform = str
+    with (ROOT / "platformio.ini").open(encoding="utf-8") as handle:
+        parser.read_file(handle)
+    return parser
+
+
+def main() -> int:
+    errors: list[str] = []
+    warnings: list[str] = []
+
+    version_result = run([sys.executable, "-m", "platformio", "--version"])
+    if version_result.returncode != 0:
+        errors.append("PlatformIO version command failed.")
+    else:
+        match = re.search(r"version\s+([0-9.]+)", version_result.stdout)
+        observed = match.group(1) if match else "unknown"
+        if observed != EXPECTED_PLATFORMIO_CORE:
+            errors.append(f"Expected PlatformIO Core {EXPECTED_PLATFORMIO_CORE}, observed {observed}.")
+
+    config_result = run([sys.executable, "-m", "platformio", "project", "config", "--json-output"])
+    if config_result.returncode != 0:
+        errors.append("PlatformIO project config command failed.")
+
+    pkg_result = run([sys.executable, "-m", "platformio", "pkg", "list"])
+    if pkg_result.returncode != 0:
+        errors.append("PlatformIO package resolution failed.")
+
+    parser = parse_platformio()
+    env_sections = {section.split(":", 1)[1] for section in parser.sections() if section.startswith("env:")}
+    missing = EXPECTED_ENVS - env_sections
+    if missing:
+        errors.append(f"Missing required PlatformIO environments: {', '.join(sorted(missing))}")
+
+    base_platform = parser.get("env", "platform", fallback="")
+    if base_platform != EXPECTED_PLATFORM:
+        errors.append("Base PlatformIO platform is not the expected pinned pioarduino release URL.")
+
+    base_board = parser.get("env", "board", fallback="")
+    base_framework = parser.get("env", "framework", fallback="")
+    if base_board != EXPECTED_BOARD:
+        errors.append(f"Expected board {EXPECTED_BOARD}, observed {base_board}.")
+    if base_framework != EXPECTED_FRAMEWORK:
+        errors.append(f"Expected framework {EXPECTED_FRAMEWORK}, observed {base_framework}.")
+
+    lib_deps = split_multiline(parser.get("env", "lib_deps", fallback=""))
+    for dep in lib_deps:
+        if "@" not in dep:
+            errors.append(f"Dependency lacks an explicit version requirement: {dep}")
+        elif "@^" in dep:
+            warnings.append(f"Ranged dependency allowed for now, exact pin proposed separately: {dep}")
+
+    for env_name in EXPECTED_ENVS:
+        section = f"env:{env_name}"
+        if not parser.has_section(section):
+            continue
+        if parser.get(section, "platform", fallback=base_platform) != EXPECTED_PLATFORM:
+            errors.append(f"{section} platform differs from the pinned base platform.")
+        if parser.get(section, "board", fallback=base_board) != EXPECTED_BOARD:
+            errors.append(f"{section} board differs from {EXPECTED_BOARD}.")
+        if parser.get(section, "framework", fallback=base_framework) != EXPECTED_FRAMEWORK:
+            errors.append(f"{section} framework differs from Arduino.")
+
+    print("Proposed exact direct dependency pins for a separate commit:")
+    for package, version in PROPOSED_EXACT_PINS.items():
+        print(f"- {package}@{version}")
+
+    for warning in warnings:
+        print(f"WARNING: {warning}")
+    if errors:
+        for error in errors:
+            print(f"ERROR: {error}")
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
@@ -0,0 +1,131 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import configparser
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+if hasattr(sys.stdout, "reconfigure"):
+    sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+if hasattr(sys.stderr, "reconfigure"):
+    sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+REQUIRED_PATHS = [
+    "AGENTS.md",
+    ".gitea/workflows/ci.yml",
+    "README.md",
+    "Requirements.md",
+    "docs/TESTS.md",
+    "docs/POWER_OPTIMIZATION.md",
+    "docs/engineering/current-state.md",
+    "docs/engineering/risk-register.md",
+    "docs/engineering/test-matrix.md",
+    "docs/engineering/hil-strategy.md",
+    "hil_tests/README.md",
+]
+TEST_SUITES = [
+    "test_html_escape",
+    "test_json_codec",
+    "test_lora_transport",
+    "test_meter_fault_count",
+    "test_payload_codec",
+    "test_refactor_smoke",
+    "test_security_fuzz",
+]
+TRACEABILITY_CONTRACTS = [
+    "LoRa frame format",
+    "CRC validation",
+    "Chunk reassembly",
+    "Payload schema",
+    "Sender-ID validation",
+    "ACK matching",
+    "Time bootstrap",
+    "Retry bounds",
+    "Queue bounds",
+    "Duplicate handling",
+    "MQTT JSON keys",
+    "Home Assistant discovery fields",
+    "CSV compatibility",
+    "HTML escaping",
+    "Web input validation",
+]
+
+
+def read_text(path: str) -> str:
+    return (ROOT / path).read_text(encoding="utf-8")
+
+
+def platformio_envs() -> set[str]:
+    parser = configparser.RawConfigParser()
+    parser.optionxform = str
+    with (ROOT / "platformio.ini").open(encoding="utf-8") as handle:
+        parser.read_file(handle)
+    return {section.split(":", 1)[1] for section in parser.sections() if section.startswith("env:")}
+
+
+def main() -> int:
+    errors: list[str] = []
+
+    for rel in REQUIRED_PATHS:
+        if not (ROOT / rel).exists():
+            errors.append(f"Missing required documentation or harness file: {rel}")
+
+    if errors:
+        for error in errors:
+            print(error)
+        return 1
+
+    envs = platformio_envs()
+    tests_doc = read_text("docs/TESTS.md")
+    readme = read_text("README.md")
+    requirements = read_text("Requirements.md")
+    current_state = read_text("docs/engineering/current-state.md")
+    risk_register = read_text("docs/engineering/risk-register.md")
+    test_matrix = read_text("docs/engineering/test-matrix.md")
+    agents = read_text("AGENTS.md")
+
+    for env in sorted(envs):
+        if f"`{env}`" not in tests_doc:
+            errors.append(f"docs/TESTS.md does not document environment {env}.")
+    for stale in ["lilygo-t3-v1-6-1", "schema v3", "schema `3`"]:
+        if stale in tests_doc:
+            errors.append(f"docs/TESTS.md still contains stale text: {stale}")
+    for suite in TEST_SUITES:
+        if suite not in tests_doc:
+            errors.append(f"docs/TESTS.md does not document suite {suite}.")
+    for contract in TRACEABILITY_CONTRACTS:
+        if contract not in tests_doc or contract not in test_matrix:
+            errors.append(f"Traceability contract missing from docs: {contract}")
+
+    schema_v4_markers = ["schema v4", "schema `4`", "schema `v4`"]
+    for path, text in [("README.md", readme), ("Requirements.md", requirements), ("current-state.md", current_state)]:
+        if not any(marker in text for marker in schema_v4_markers):
+            errors.append(f"{path} does not document payload schema v4.")
+        if "schema v3" in text or "schema `3`" in text:
+            errors.append(f"{path} still documents schema v3.")
+
+    for category in [
+        "Verified Defects",
+        "Probable Risks",
+        "Missing Tests",
+        "Missing Documentation",
+        "Security Decisions",
+        "Hardware-Dependent Questions",
+    ]:
+        if category not in risk_register:
+            errors.append(f"risk-register.md missing category: {category}")
+
+    if "python tools/verify.py" not in agents:
+        errors.append("AGENTS.md does not name the canonical verification command.")
+
+    if errors:
+        for error in errors:
+            print(f"ERROR: {error}")
+        return 1
+
+    print("Documentation consistency checks passed.")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import py_compile
+import subprocess
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+if hasattr(sys.stdout, "reconfigure"):
+    sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+if hasattr(sys.stderr, "reconfigure"):
+    sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+TEXT_SUFFIXES = {
+    ".c",
+    ".cc",
+    ".cpp",
+    ".h",
+    ".hpp",
+    ".ini",
+    ".json",
+    ".md",
+    ".ps1",
+    ".py",
+    ".txt",
+    ".yaml",
+    ".yml",
+}
+SKIP_PATHS = {
+    ".vscode/c_cpp_properties.json",
+    ".vscode/launch.json",
+}
+
+
+def tracked_and_untracked_files() -> list[Path]:
+    result = subprocess.run(
+        ["git", "ls-files", "--cached", "--others", "--exclude-standard"],
+        cwd=ROOT,
+        text=True,
+        encoding="utf-8",
+        stdout=subprocess.PIPE,
+        stderr=subprocess.PIPE,
+    )
+    if result.returncode != 0:
+        print(result.stderr, end="")
+        raise SystemExit(result.returncode)
+    return [ROOT / line.strip() for line in result.stdout.splitlines() if line.strip()]
+
+
+def is_text_candidate(path: Path) -> bool:
+    rel = path.relative_to(ROOT).as_posix()
+    if rel in SKIP_PATHS:
+        return False
+    return path.suffix.lower() in TEXT_SUFFIXES
+
+
+def main() -> int:
+    errors: list[str] = []
+    for path in tracked_and_untracked_files():
+        if not path.is_file() or not is_text_candidate(path):
+            continue
+        rel = path.relative_to(ROOT).as_posix()
+        try:
+            text = path.read_text(encoding="utf-8")
+        except UnicodeDecodeError as exc:
+            errors.append(f"{rel}: not valid UTF-8: {exc}")
+            continue
+        for line_no, line in enumerate(text.splitlines(), 1):
+            if line.startswith(("<<<<<<<", "=======", ">>>>>>>")):
+                errors.append(f"{rel}:{line_no}: merge conflict marker")
+
+    for script in sorted((ROOT / "tools").glob("*.py")):
+        try:
+            py_compile.compile(str(script), doraise=True)
+        except py_compile.PyCompileError as exc:
+            errors.append(str(exc))
+
+    if not (ROOT / ".clang-format").exists():
+        print("No .clang-format found; source formatting is not reformatted by this check.")
+    print("Checked text encodings, conflict markers, and Python syntax.")
+
+    if errors:
+        for error in errors:
+            print(error)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
@@ -0,0 +1,62 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import json
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+if hasattr(sys.stdout, "reconfigure"):
+    sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+if hasattr(sys.stderr, "reconfigure"):
+    sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+BASELINE_PATH = ROOT / "docs" / "engineering" / "size-baseline.json"
+ENVS = ["production", "debug", "test"]
+
+
+def file_size(path: Path) -> int:
+    return path.stat().st_size if path.exists() else -1
+
+
+def main() -> int:
+    if not BASELINE_PATH.exists():
+        print(f"Missing size baseline: {BASELINE_PATH.relative_to(ROOT)}")
+        return 1
+
+    baseline = json.loads(BASELINE_PATH.read_text(encoding="utf-8"))
+    baseline_envs = baseline.get("environments", {})
+    errors: list[str] = []
+
+    print("Firmware size report:")
+    for env in ENVS:
+        build_dir = ROOT / ".pio" / "build" / env
+        bin_path = build_dir / "firmware.bin"
+        elf_path = build_dir / "firmware.elf"
+        bin_size = file_size(bin_path)
+        elf_size = file_size(elf_path)
+        if bin_size < 0 or elf_size < 0:
+            errors.append(f"{env}: missing firmware artifacts; run python -m platformio run -e {env}")
+            continue
+        entry = baseline_envs.get(env)
+        if not entry:
+            errors.append(f"{env}: missing baseline entry")
+            continue
+        baseline_bin = int(entry["firmware_bin_bytes"])
+        max_bin = int(entry.get("max_firmware_bin_bytes", baseline_bin))
+        delta = bin_size - baseline_bin
+        print(
+            f"- {env}: firmware.bin={bin_size} bytes "
+            f"(baseline {baseline_bin}, delta {delta:+}), firmware.elf={elf_size} bytes"
+        )
+        if bin_size > max_bin:
+            errors.append(f"{env}: firmware.bin {bin_size} exceeds limit {max_bin}")
+
+    if errors:
+        for error in errors:
+            print(f"ERROR: {error}")
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+if hasattr(sys.stdout, "reconfigure"):
+    sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+if hasattr(sys.stderr, "reconfigure"):
+    sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+
+
+def command_text(command: list[str]) -> str:
+    parts: list[str] = []
+    for part in command:
+        if any(ch.isspace() for ch in part):
+            parts.append(f'"{part}"')
+        else:
+            parts.append(part)
+    return " ".join(parts)
+
+
+def run_step(name: str, command: list[str]) -> bool:
+    print(f"\n== {name} ==", flush=True)
+    print(f"$ {command_text(command)}", flush=True)
+    env = os.environ.copy()
+    env["PYTHONIOENCODING"] = "utf-8"
+    env["PYTHONUTF8"] = "1"
+    env["PIO_FORCE_COLOR"] = "0"
+    result = subprocess.run(command, cwd=ROOT, env=env)
+    if result.returncode != 0:
+        print(f"FAILED: {name} exited with {result.returncode}", flush=True)
+        return False
+    print(f"PASSED: {name}", flush=True)
+    return True
+
+
+def main() -> int:
+    python = sys.executable
+    steps = [
+        ("repository formatting check", [python, "tools/check_format.py"]),
+        ("dependency and PlatformIO policy check", [python, "tools/check_dependencies.py"]),
+        ("production build", [python, "-m", "platformio", "run", "-e", "production"]),
+        ("debug build", [python, "-m", "platformio", "run", "-e", "debug"]),
+        ("test firmware build", [python, "-m", "platformio", "run", "-e", "test"]),
+        (
+            "compile-only embedded Unity tests",
+            [python, "-m", "platformio", "test", "-e", "test", "--without-uploading", "--without-testing"],
+        ),
+        ("static analysis", [python, "-m", "platformio", "check", "-e", "production", "--fail-on-defect=high"]),
+        ("documentation consistency check", [python, "tools/check_docs.py"]),
+        ("firmware size check", [python, "tools/check_size.py"]),
+    ]
+
+    failures: list[str] = []
+    for name, command in steps:
+        if not run_step(name, command):
+            failures.append(name)
+
+    if failures:
+        print("\nVerification failed:")
+        for failure in failures:
+            print(f"- {failure}")
+        return 1
+
+    print("\nVerification passed.")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())