DD3-LoRa-Bridge-MultiSender/docs/engineering/test-matrix.md

Test Matrix

Use python tools/verify.py as the canonical non-hardware verification command.

Non-Hardware Verification

Check	Command	Hardware	Required in CI
Repository, dependency, and PlatformIO policy	python tools/check_dependencies.py	None	Yes
Text hygiene and Python syntax	python tools/check_format.py	None	Yes
Production build	python -m platformio run -e production	None	Yes
Debug build	python -m platformio run -e debug	None	Yes
Test firmware build	python -m platformio run -e test	None	Yes
Compile-only Unity suites	python -m platformio test -e test --without-uploading --without-testing	None	Yes
Static analysis	python -m platformio check -e production --fail-on-defect=high	None	Yes
Documentation consistency	python tools/check_docs.py	None	Yes
Firmware size report	python tools/check_size.py	None after builds	Yes

Existing Embedded Unity Suites

Suite	Primary coverage	Compile-only command	On-device command
test_payload_codec	Payload schema v4, sparse mask, varints, golden vectors	python -m platformio test -e test --without-uploading --without-testing -f test_payload_codec	python -m platformio test -e test -f test_payload_codec
test_lora_transport	CRC16, LoRa frame build/parse, chunk reassembly	python -m platformio test -e test --without-uploading --without-testing -f test_lora_transport	python -m platformio test -e test -f test_lora_transport
test_json_codec	MQTT JSON keys, optional keys, HA discovery fields	python -m platformio test -e test --without-uploading --without-testing -f test_json_codec	python -m platformio test -e test -f test_json_codec
test_html_escape	HTML escaping, URL encoding, web device-ID validation	python -m platformio test -e test --without-uploading --without-testing -f test_html_escape	python -m platformio test -e test -f test_html_escape
test_refactor_smoke	Public refactor headers and HA manufacturer guard	python -m platformio test -e test --without-uploading --without-testing -f test_refactor_smoke	python -m platformio test -e test -f test_refactor_smoke
test_meter_fault_count	Stale-meter fault counter regression	python -m platformio test -e test --without-uploading --without-testing -f test_meter_fault_count	python -m platformio test -e test -f test_meter_fault_count
test_security_fuzz	Malformed payload/frame/reassembly rejection	python -m platformio test -e test --without-uploading --without-testing -f test_security_fuzz	python -m platformio test -e test -f test_security_fuzz

Hardware Requirements

Scenario	Boards	Smart meter	Wi-Fi	MQTT	SD
Compile-only Unity tests	0	No	No	No	No
On-device Unity test suite	1	No	No	No	No
Test-mode sender or receiver smoke	1	No	Optional	Optional	Optional
Normal sender boot and sampling	1 sender	Yes or simulator	No	No	No
Normal receiver boot and web/AP fallback	1 receiver	No	Optional	Optional	Optional
End-to-end batch transfer	1 sender + 1 receiver	Yes or simulator	Yes	Yes	Optional
SD logging and history	1 receiver plus sender traffic	Optional	Optional	Optional	Yes
Power measurement	1 sender plus receiver ACK source	Yes or simulator	No	No	No

Contract Traceability

Contract	Tests or checks
LoRa frame format	test_lora_transport, test_security_fuzz, tools/check_docs.py
CRC validation	test_lora_transport, test_security_fuzz
Chunk reassembly	test_lora_transport, test_security_fuzz
Payload schema	test_payload_codec, test_security_fuzz, tools/check_docs.py
Sender-ID validation	Compile coverage in test_refactor_smoke; runtime behavior documented in README.md; HIL planned
ACK matching	Runtime behavior documented in README.md; HIL planned
Time bootstrap	Payload sync vector in test_payload_codec; runtime behavior documented in README.md; HIL planned
Retry bounds	Constants and state machine compile coverage; test_meter_fault_count covers one retry-adjacent regression; extraction planned
Queue bounds	State machine compile coverage; extraction planned
Duplicate handling	Runtime behavior documented in README.md; HIL planned
MQTT JSON keys	test_json_codec
Home Assistant discovery fields	test_json_codec, test_refactor_smoke, test/check_ha_manufacturer.ps1
CSV compatibility	Documented in README.md and docs/engineering/current-state.md; parser extraction planned
HTML escaping	test_html_escape
Web input validation	test_html_escape; SD/history route validation extraction planned

Native-Test Feasibility

A PlatformIO native environment should not be introduced until pure logic is separated from Arduino headers. The best first candidates are already close to pure logic:

• CRC functions and LoRa frame build/parse in lib/dd3_transport_logic.
• Batch reassembly in lib/dd3_transport_logic.
• Payload encoding/decoding and varints in lib/dd3_legacy_core.
• HTML escaping, URL encoding, and device-ID sanitization in lib/dd3_legacy_core.
• HA discovery JSON builder once ArduinoJson and String dependencies are handled.

Current blockers:

• Public helper headers include Arduino.h.
• Several otherwise pure helpers expose Arduino String.
• ArduinoJson is used directly in host-interesting code.
• Runtime logic depends on global Arduino services: millis, delay, Serial, Serial2, LoRa, WiFi, SD, Preferences, and ESP32 FreeRTOS APIs.
• Sender state-transition logic is still coupled to module-static globals and hardware-facing helpers.

Staged extraction plan:

1. Remove unnecessary Arduino.h includes from pure helper headers by replacing Arduino integer types with standard C/C++ headers.
2. Add native tests for lora_frame_logic and batch_reassembly_logic.
3. Split payload codec into a header/source pair that only needs standard C++ types, then add native golden-vector tests.
4. Add small pure helpers for ACK parse/validation, sparse-mask timestamp reconstruction, retry timeout calculation, and CSV line parsing.
5. Keep Arduino wrappers in firmware modules and test extracted helpers natively plus embedded compile-only suites.