## Bug Fixes - Fix integer overflow potential in history bin allocation (web_server.cpp) Using uint64_t for intermediate multiplication prevents overflow with different constants - Prevent data loss during WiFi failures (main.cpp) Device now automatically attempts WiFi reconnection every 30 seconds when in AP mode Exits AP mode and resumes MQTT transmission as soon as WiFi becomes available Data collection and SD logging continue regardless of connectivity ## New Features - Add standalone MQTT data republisher for lost data recovery - Command-line tool (republish_mqtt.py) with interactive and scripting modes - GUI tool (republish_mqtt_gui.py) for user-friendly recovery - Rate-limited publishing (5 msg/sec default, configurable 1-100) - Manual time range selection or auto-detect missing data via InfluxDB - Cross-platform support (Windows, macOS, Linux) - Converts SD card CSV exports back to MQTT format ## Documentation - Add comprehensive code review (CODE_REVIEW.md) - 16 detailed security and quality assessments - Identifies critical HTTPS/auth gaps, medium-priority overflow issues - Confirms absence of buffer overflows and unsafe string functions - Grade: B+ with areas for improvement - Add republisher documentation (REPUBLISH_README.md, REPUBLISH_GUI_README.md) - Installation and usage instructions - Example commands and scenarios - Troubleshooting guide - Performance characteristics ## Dependencies - Add requirements_republish.txt - paho-mqtt>=1.6.1 - influxdb-client>=1.18.0 ## Impact - Eliminates data loss scenario where unreliable WiFi leaves device stuck in AP mode - Provides recovery mechanism for any historical data missed during outages - Improves code safety with explicit overflow-resistant arithmetic - Increases operational visibility with comprehensive code review
DD3-LoRa-Bridge-MultiSender
Firmware for LilyGO T3 v1.6.1 (ESP32 + SX1276 + SSD1306) that runs in two roles:
Sender(GPIO14HIGH): reads one IEC 62056-21 meter, builds 30-slot sparse batches, sends via LoRa.Receiver(GPIO14LOW): receives/ACKs batches, publishes MQTT, serves web UI, logs to SD.
Architecture Summary
- Single codebase, role selected at boot by
detect_role()(src/config.cpp). - LoRa transport is wrapped with firmware-level CRC16-CCITT (
src/lora_transport.cpp). - Sender meter ingest is decoupled from LoRa waits via FreeRTOS meter reader task + queue on ESP32 (
src/main.cpp). - Batch payload codec is schema
v3with a 30-bitpresent_maskover[t_last-29, t_last](src/payload_codec.cpp). - Sender only starts normal metering/transmit flow after valid time bootstrap from receiver ACK.
- Receiver runs STA mode if stored config is valid and connects, otherwise AP fallback.
LoRa Protocol
On-air frame:
[msg_kind:1][device_short_id:2][payload...][crc16:2]
msg_kind:
0:BatchUp1:AckDown
BatchUp
Transport layer chunks payload into:
[batch_id_le:2][chunk_index:1][chunk_count:1][total_len_le:2][chunk_payload...]
Receiver reassembles all chunks before decode.
Payload codec (schema=3, magic 0xDDB3) carries:
- metadata: sender ID, batch ID,
t_last,present_mask, battery mV, error counters - arrays per present sample:
energy_wh[],p1_w[],p2_w[],p3_w[]
n == 0 with present_mask == 0 is valid and used for sync request packets.
AckDown (7 bytes payload)
[flags:1][batch_id_be:2][epoch_utc_be:4]
flags bit0:time_valid- ACK is repeated (
ACK_REPEAT_COUNT=3,ACK_REPEAT_DELAY_MS=200) - Sender sets local time only if
time_valid=1andepoch >= MIN_ACCEPTED_EPOCH_UTC(2026-02-01 00:00:00 UTC)
Time Bootstrap and Timezone
Sender boot starts in sync-only mode:
g_time_acquired=false- sends sync requests every
SYNC_REQUEST_INTERVAL_MS(15s) - does not run normal 1 Hz sample/batch flow yet
After valid ACK time:
time_set_utc()is calledg_time_acquired=true- normal 1 Hz sampling + periodic batch transmission starts
Timezone:
TIMEZONE_TZfrominclude/config.his applied intime_manager.- Web/OLED local-time rendering uses this timezone.
- Default:
CET-1CEST,M3.5.0/2,M10.5.0/3.
Sender Meter Path
Implemented by src/meter_driver.cpp and sender loop in src/main.cpp:
- UART:
Serial2,GPIO34,9600 7E1 - ESP32 RX buffer enlarged to
8192 - Frame detection
/ ... !, timeoutMETER_FRAME_TIMEOUT_MS=3000 - Parsed OBIS fields:
0-0:96.8.0*255meter Sekundenindex (hex u32)1-0:1.8.0total energy (auto scales Wh -> kWh when unit is Wh)1-0:16.7.0total active power1-0:36.7.0,56.7.0,76.7.0phase powers
Timestamp derivation:
- anchor offset:
epoch_offset = epoch_now - meter_seconds - sample epoch:
ts_utc = meter_seconds + epoch_offset - jump checks: rollback, wall-time delta mismatch, anchor drift
Sender builds sparse 30-slot windows and sends every METER_SEND_INTERVAL_MS (30s).
Receiver Behavior
For decoded BatchUp:
- Reassemble and decode.
- Send
AckDownimmediately. - Track duplicates per configured sender (
EXPECTED_SENDER_IDS). - If duplicate: update duplicate counters/time, skip data write/publish.
- If
n==0: sync request path only. - Else reconstruct each sample timestamp from
t_last + present_mask, then:- append to SD CSV
- publish MQTT state
- update web status and last batch table
MQTT
State topic:
smartmeter/<device_id>/state
Fault topic (retained):
smartmeter/<device_id>/faults
State JSON (src/json_codec.cpp) includes:
id,ts,e_kwhp_w,p1_w,p2_w,p3_wbat_v,bat_pct- optional link:
rssi,snr err_last,rx_reject,rx_reject_text- non-zero fault counters when available
Home Assistant discovery:
- enabled by
ENABLE_HA_DISCOVERY=true - publishes to
homeassistant/sensor/<device_id>/<key>/config
Web UI, Wi-Fi, SD
- Wi-Fi/MQTT/NTP/web-auth config is stored in Preferences.
- AP fallback SSID prefix:
DD3-Bridge-. - Default web credentials:
admin/admin. - AP auth requirement is controlled by
WEB_AUTH_REQUIRE_AP(defaultfalse). - STA auth requirement is controlled by
WEB_AUTH_REQUIRE_STA(defaulttrue).
Web timestamp display:
- human-facing timestamps show
epoch (HH:MM:SS TZ)in local configured timezone.
SD CSV logging (src/sd_logger.cpp):
- header:
ts_utc,ts_hms_utc,p_w,p1_w,p2_w,p3_w,e_kwh,bat_v,bat_pct,rssi,snr,err_m,err_d,err_tx,err_last ts_hms_utcis UTCHH:MM:SS
History parser (src/web_server.cpp):
- expects the current CSV layout above
- legacy CSV layouts are not parsed (no backward compatibility)
OLED duplicate display:
- receiver sender-pages show duplicate rate as
pct (absolute)and last duplicate asHH:MM.
Build Environments
From platformio.ini:
lilygo-t3-v1-6-1lilygo-t3-v1-6-1-testlilygo-t3-v1-6-1-868lilygo-t3-v1-6-1-868-testlilygo-t3-v1-6-1-payload-testlilygo-t3-v1-6-1-868-payload-testlilygo-t3-v1-6-1-prodlilygo-t3-v1-6-1-868-prod
Example:
python -m platformio run -e lilygo-t3-v1-6-1
Test Mode
ENABLE_TEST_MODE replaces normal loops with test_sender_loop / test_receiver_loop (src/test_mode.cpp):
- Sender emits periodic JSON test payloads over LoRa.
- Receiver decodes test payloads, updates display test codes, publishes MQTT to:
smartmeter/<device_id>/test