100 lines
4.9 KiB
Markdown
100 lines
4.9 KiB
Markdown
# Firmware Tests
|
|
|
|
The canonical non-hardware command is:
|
|
|
|
```bash
|
|
python tools/verify.py
|
|
```
|
|
|
|
Current PlatformIO environments from `platformio.ini`:
|
|
|
|
- `production`: deployment build, serial debug off, light sleep on.
|
|
- `debug`: serial diagnostics on, real meter and real LoRa.
|
|
- `test`: synthetic meter samples and payload codec self-test.
|
|
|
|
Payload schema: v4.
|
|
|
|
## Compile-Only Tests
|
|
|
|
These commands do not require USB devices or physical boards.
|
|
|
|
```bash
|
|
python -m platformio test -e test --without-uploading --without-testing
|
|
python -m platformio test -e test --without-uploading --without-testing -f test_html_escape
|
|
python -m platformio test -e test --without-uploading --without-testing -f test_payload_codec
|
|
python -m platformio test -e test --without-uploading --without-testing -f test_lora_transport
|
|
python -m platformio test -e test --without-uploading --without-testing -f test_json_codec
|
|
python -m platformio test -e test --without-uploading --without-testing -f test_refactor_smoke
|
|
python -m platformio test -e test --without-uploading --without-testing -f test_meter_fault_count
|
|
python -m platformio test -e test --without-uploading --without-testing -f test_security_fuzz
|
|
```
|
|
|
|
## On-Device Unity Tests
|
|
|
|
These commands require one connected LilyGO T3 v1.6.1 board. They do not require a sender/receiver pair unless the test itself is changed to exercise radio interaction.
|
|
|
|
```bash
|
|
python -m platformio test -e test
|
|
python -m platformio test -e test -f test_html_escape
|
|
python -m platformio test -e test -f test_payload_codec
|
|
python -m platformio test -e test -f test_lora_transport
|
|
python -m platformio test -e test -f test_json_codec
|
|
python -m platformio test -e test -f test_refactor_smoke
|
|
python -m platformio test -e test -f test_meter_fault_count
|
|
python -m platformio test -e test -f test_security_fuzz
|
|
```
|
|
|
|
Do not run upload, flash, monitor, or HIL commands from non-hardware CI.
|
|
|
|
## Suite Coverage
|
|
|
|
| Suite | Coverage | Hardware notes |
|
|
|---|---|---|
|
|
| `test_payload_codec` | Payload schema v4 roundtrip/reject paths, sparse masks, sync packets, golden vectors. | Compile-only works; on-device needs one board. |
|
|
| `test_lora_transport` | CRC16, LoRa frame encode/decode integrity, malformed frame rejection, chunk reassembly. | Compile-only works; on-device needs one board. |
|
|
| `test_json_codec` | MQTT state JSON key stability and Home Assistant discovery field stability. | Compile-only works; no Wi-Fi/MQTT broker required. |
|
|
| `test_html_escape` | HTML escaping, URL encoding, and web device-ID input validation. | Compile-only works; no web server required. |
|
|
| `test_refactor_smoke` | Public sender/receiver headers and HA manufacturer constant guard. | Compile-only works. |
|
|
| `test_meter_fault_count` | Regression for stale-meter fault counting during catch-up ticks. | Compile-only works; no smart meter required. |
|
|
| `test_security_fuzz` | Negative/fuzz seeds for payload decode, varints, LoRa frames, and reassembly. | Compile-only works. |
|
|
|
|
## Hardware Feature Requirements
|
|
|
|
| Check | Boards | Sender/receiver pair | Smart meter | Wi-Fi | MQTT | SD |
|
|
|---|---:|---:|---:|---:|---:|---:|
|
|
| Compile-only Unity tests | 0 | No | No | No | No | No |
|
|
| On-device Unity tests | 1 | No | No | No | No | No |
|
|
| Test-mode LoRa smoke | 2 | Yes | No | Optional | Optional | Optional |
|
|
| Production sender sampling | 1 sender | No | Yes or UART simulator | No | No | No |
|
|
| Production receiver AP/STA smoke | 1 receiver | No | No | Optional | Optional | Optional |
|
|
| End-to-end production transfer | 2 | Yes | Yes or UART simulator | Yes | Yes | Optional |
|
|
| SD logging and history | 1 receiver plus traffic | Usually | No | Optional | Optional | Yes |
|
|
|
|
## Traceability
|
|
|
|
| Contract | Current tests |
|
|
|---|---|
|
|
| LoRa frame format | `test_lora_transport`, `test_security_fuzz` |
|
|
| CRC validation | `test_lora_transport`, `test_security_fuzz` |
|
|
| Chunk reassembly | `test_lora_transport`, `test_security_fuzz` |
|
|
| Payload schema | `test_payload_codec`, `test_security_fuzz` |
|
|
| Sender-ID validation | `test_refactor_smoke` compile coverage; HIL planned for runtime rejection path |
|
|
| ACK matching | HIL planned; runtime contract documented in `README.md` |
|
|
| Time bootstrap | `test_payload_codec` sync vector; HIL planned for runtime bootstrap |
|
|
| Retry bounds | `test_meter_fault_count` adjacent regression; helper extraction planned |
|
|
| Queue bounds | `test_refactor_smoke` compile coverage; helper extraction planned |
|
|
| Duplicate handling | HIL planned; runtime contract documented in `README.md` |
|
|
| MQTT JSON keys | `test_json_codec` |
|
|
| Home Assistant discovery fields | `test_json_codec`, `test_refactor_smoke`, `test/check_ha_manufacturer.ps1` |
|
|
| CSV compatibility | HIL or extracted parser tests planned; contract documented in `README.md` |
|
|
| HTML escaping | `test_html_escape` |
|
|
| Web input validation | `test_html_escape`; route-level tests planned |
|
|
|
|
## Additional Static Guard
|
|
|
|
Run the Home Assistant manufacturer drift guard:
|
|
|
|
```powershell
|
|
powershell -ExecutionPolicy Bypass -File test/check_ha_manufacturer.ps1
|
|
```
|