DD3-LoRa-Bridge-MultiSender/README.md

DD3 LoRa Bridge (Multi-Sender)

Unified firmware for LilyGO T3 v1.6.1 (ESP32 + SX1276 + SSD1306) that runs as Sender or Receiver based on a GPIO jumper. Senders read DD3 smart meter values and transmit compact binary batches over LoRa. The receiver validates packets, publishes to MQTT, provides a web UI, and shows per-sender status on the OLED.

Hardware

Board: LilyGO T3 LoRa32 v1.6.1 (ESP32 + SX1276 + SSD1306 128x64 + LiPo) Variants:

• SX1276 433 MHz module (default build)
• SX1276 868 MHz module (use 868 build environments)

Pin Mapping

• LoRa (SX1276)
  • SCK: GPIO5
  • MISO: GPIO19
  • MOSI: GPIO27
  • NSS/CS: GPIO18
  • RST: GPIO23
  • DIO0: GPIO26
• OLED (SSD1306)
  • SDA: GPIO21
  • SCL: GPIO22
  • RST: not used (SSD1306 init uses -1 reset pin)
  • I2C address: 0x3C
• microSD (on-board)
  • CS: GPIO13
  • MOSI: GPIO15
  • SCK: GPIO14
  • MISO: GPIO2
• I2C RTC (DS3231)
  • SDA: GPIO21
  • SCL: GPIO22
  • I2C address: 0x68
• Battery ADC: GPIO35 (via on-board divider)
• Role select: GPIO14 (INPUT_PULLDOWN, sampled at boot, shared with SD SCK)
  • HIGH = Sender
  • LOW/floating = Receiver
• OLED control: GPIO13 (INPUT_PULLDOWN, sender only, shared with SD CS)
  • HIGH = force OLED on
  • LOW = allow auto-off after timeout
  • Not used on receiver (OLED always on)
• Smart meter UART RX: GPIO34 (input-only, always connected)

Notes on GPIOs

• GPIO34/35/36/39 are input-only and have no internal pullups/pulldowns.
• Strap pins (GPIO0/2/4/5/12/15) can affect boot; avoid for role or control jumpers.
• GPIO14 is shared between role select and SD SCK. Do not attach the role jumper in Receiver mode if the SD card is connected/used, and never force GPIO14 high when using SD.
• GPIO13 is shared between OLED control and SD CS. Avoid driving OLED control when SD is active.
• Receiver firmware releases GPIO14 to INPUT (no pulldown) after boot before SD SPI init.

Firmware Roles

Sender (battery-powered)

• Reads smart meter via optical IR (UART 9600 7E1).
• Extracts OBIS values:
  • Energy total: 1-0:1.8.0*255
  • Total power: 1-0:16.7.0*255
  • Phase power: 36.7 / 56.7 / 76.7
• Meter input is parsed via a non-blocking RX state machine; the last valid frame is reused for 1 Hz sampling.
• Reads battery voltage and estimates SoC.
• Builds compact binary batch payload, wraps in LoRa packet, transmits.
• Light sleeps between meter reads; batches are sent every 30s.
• Listens for LoRa time sync packets to set UTC clock.
• Uses DS3231 RTC after boot if no time sync has arrived yet.
• OLED shows status + meter data pages.

Sender flow (pseudo-code):

void sender_loop() {
  meter_read_every_second(); // OBIS -> MeterData samples
  read_battery(data);        // VBAT + SoC

  if (time_to_send_batch()) {
    payload = encode_batch(samples, batch_id); // compact binary batch
    lora_send(packet(MeterBatch, payload));
  }

  display_set_last_meter(data);
  display_set_last_read(ok);
  display_set_last_tx(ok);
  display_tick();

  lora_receive_time_sync(); // optional
  light_sleep_until_next_event();
}

Key sender functions:

bool meter_read(MeterData &data);        // parse OBIS fields
void read_battery(MeterData &data);      // ADC -> volts + percent
bool meterDataToJson(const MeterData&, String&);
bool compressBuffer(const uint8_t*, size_t, uint8_t*, size_t, size_t&); // MeterData only
bool lora_send(const LoraPacket &pkt);   // add header + CRC16 and transmit

Receiver (USB-powered)

• WiFi STA connect using stored config; if not available/fails, starts AP.
• NTP sync (UTC) and local display in Europe/Berlin.
• Receives LoRa packets, verifies CRC16, decompresses MeterData JSON, decodes binary batches.
• Publishes meter JSON to MQTT.
• Sends ACKs for MeterBatch packets and de-duplicates by batch_id.
• Web UI:
  • AP mode: status + WiFi/MQTT config.
  • STA mode: status + per-sender pages.
• OLED cycles through receiver status and per-sender pages (receiver OLED never sleeps).

Receiver loop (pseudo-code):

void receiver_loop() {
  if (lora_receive(pkt)) {
    if (pkt.type == MeterData) {
      json = decompressBuffer(pkt.payload);
      if (jsonToMeterData(json, data)) {
        update_sender_status(data);
        mqtt_publish_state(data);
      }
    } else if (pkt.type == MeterBatch) {
      batch = reassemble_and_decode_batch(pkt);
      for (sample in batch) {
        update_sender_status(sample);
        mqtt_publish_state(sample);
      }
    }
  }

  if (time_to_send_timesync()) {
    time_send_timesync(self_short_id); // 60s for first 10 min, then hourly if RTC is present
  }

  mqtt_loop();
  web_server_loop();
  display_set_receiver_status(...);
  display_tick();
}

Key receiver functions:

bool lora_receive(LoraPacket &pkt, uint32_t timeout_ms);
bool jsonToMeterData(const String &json, MeterData &data);
bool decode_batch(const uint8_t *buf, size_t len, BatchInput *out);
bool mqtt_publish_state(const MeterData &data);
void web_server_loop();                   // AP or STA UI
void time_send_timesync(uint16_t self_id);

Test Mode (compile-time)

Enabled by -DENABLE_TEST_MODE (see platformio.ini test environment).

• Sender: sends 4-digit test code every ~30s in JSON.
• Receiver: shows last test code per sender and publishes to /test topic.
• Normal behavior is excluded from test builds.

Test sender (pseudo-code):

void test_sender_loop() {
  code = random_4_digits();
  json = {id, role:"sender", test_code: code, ts};
  lora_send(packet(TestCode, compress(json)));
  display_set_test_code(code);
}

Test receiver (pseudo-code):

void test_receiver_loop() {
  if (pkt.type == TestCode) {
    json = decompress(pkt.payload);
    update_sender_test_code(json);
    mqtt_publish_test(id, json);
  }
}

LoRa Protocol

Packet layout:

[0]    protocol_version (1)
[1]    role (0=sender, 1=receiver)
[2..3] device_id_short (uint16)
[4]    payload_type (0=meter, 1=test, 2=time_sync, 3=meter_batch, 4=ack)
[5..N-3] payload bytes (compressed JSON for MeterData, binary for MeterBatch/Test/TimeSync)
[N-2..N-1] CRC16 (bytes 0..N-3)

LoRa radio settings:

• Frequency: 433 MHz or 868 MHz (set by build env via LORA_FREQUENCY_HZ)
• SF12, BW 125 kHz, CR 4/5, CRC on, Sync Word 0x34

Data Format

MeterData JSON (sender + MQTT):

{
  "id": "F19C",
  "ts": 1737200000,
  "e_kwh": 1234.57,
  "p_w": 950.00,
  "p1_w": 500.00,
  "p2_w": 450.00,
  "p3_w": 0.00,
  "bat_v": 3.92,
  "bat_pct": 78
}

Binary MeterBatch Payload (LoRa)

Fixed header (little-endian):

• magic u16 = 0xDDB3
• schema u8 = 1
• flags u8 = 0x01 (bit0 = signed phases)
• sender_id u16 (1..NUM_SENDERS, maps to EXPECTED_SENDER_IDS)
• batch_id u16
• t_last u32 (unix seconds of last sample)
• dt_s u8 (seconds, >0)
• n u8 (sample count, <=30)
• battery_mV u16
• err_m u8 (meter read failures, sender-side counter)
• err_d u8 (decode failures, sender-side counter)
• err_tx u8 (LoRa TX failures, sender-side counter)
• err_last u8 (last error code: 0=None, 1=MeterRead, 2=Decode, 3=LoraTx)

Body:

• E0 u32 (absolute energy in Wh)
• dE[1..n-1] ULEB128 (delta vs previous, >=0)
• P1_0 s16 (absolute W)
• dP1[1..n-1] signed varint (ZigZag + ULEB128)
• P2_0 s16
• dP2[1..n-1] signed varint
• P3_0 s16
• dP3[1..n-1] signed varint

Notes:

• Receiver reconstructs timestamps from t_last and dt_s.
• Total power is computed on receiver as p1 + p2 + p3.
• Sender error counters are carried in the batch header and applied to all samples.

Device IDs

• Derived from WiFi STA MAC.
• short_id = (MAC[4] << 8) | MAC[5]
• device_id = dd3-%04X
• JSON id uses only the last 4 hex digits (e.g., F19C) to save airtime.

Receiver expects known senders in include/config.h via:

constexpr uint8_t NUM_SENDERS = 1;
inline constexpr uint16_t EXPECTED_SENDER_IDS[NUM_SENDERS] = { 0xF19C };

OLED Behavior

• Sender: OLED stays ON for 10 seconds on each wake, then powers down for sleep.
• Receiver: OLED is always on (no auto-off).
• Pages rotate every 4s.

Power & Battery

• Sender disables WiFi/BLE, reads VBAT via ADC, uses linear SoC map:
  • 3.0 V = 0%
  • 4.2 V = 100%
• Uses deep sleep between cycles (SENDER_WAKE_INTERVAL_SEC).
• Sender CPU is throttled to 80 MHz and LoRa RX is only enabled in short windows (ACK wait or time-sync).

Web UI

• AP SSID: DD3-Bridge-<short_id> (prefix configurable)
• AP password: changeme123 (configurable)
• Endpoints:
  • /: status overview
  • /wifi: WiFi/MQTT/NTP config (AP and STA)
  • /sender/<device_id>: per-sender details
• Sender IDs on / are clickable (open sender page in a new tab).
• In STA mode, the UI is also available via the board's IP/hostname on your WiFi network.
• Main page shows SD card file listing (downloadable).
• Sender page includes a history chart (power) with configurable range/resolution/mode.

Security

• Basic Auth is supported for the web UI. In STA mode it is enabled by default; AP mode is optional.
• Config flags in include/config.h:
  • WEB_AUTH_REQUIRE_STA (default true)
  • WEB_AUTH_REQUIRE_AP (default false)
  • WEB_AUTH_DEFAULT_USER / WEB_AUTH_DEFAULT_PASS
• Web credentials are stored in NVS. /wifi, /sd/download, /history/*, /, /sender/*, and /manual require auth when enabled.
• Password inputs are not prefilled. Leaving a password blank keeps the stored value; use the "clear password" checkbox to erase it.
• User-controlled strings are HTML-escaped before embedding in pages.

MQTT

• Topic: smartmeter/<deviceId>/state
• QoS 0
• Test mode: smartmeter/<deviceId>/test
• Client ID: dd3-bridge-<device_id> (stable, derived from MAC)

NTP

• NTP servers are configurable in the web UI (/wifi).
• Defaults: pool.ntp.org and time.nist.gov.

RTC (DS3231)

• Optional DS3231 on the I2C bus. Connect SDA to GPIO21 and SCL to GPIO22 (same bus as the OLED).
• Enable/disable with ENABLE_DS3231 in include/config.h.
• Receiver time sync packets set the RTC.
• On boot, if no LoRa time sync has arrived yet, the sender uses the RTC time as the initial ts_utc.
• When no RTC is present or enabled, the receiver keeps sending time sync every 60 seconds.

Build Environments

• lilygo-t3-v1-6-1: production build (debug on)
• lilygo-t3-v1-6-1-test: test build with ENABLE_TEST_MODE
• lilygo-t3-v1-6-1-868: production build for 868 MHz modules (debug on)
• lilygo-t3-v1-6-1-868-test: test build for 868 MHz modules
• lilygo-t3-v1-6-1-payload-test: build with PAYLOAD_CODEC_TEST
• lilygo-t3-v1-6-1-868-payload-test: 868 MHz build with PAYLOAD_CODEC_TEST
• lilygo-t3-v1-6-1-prod: production build with serial debug off
• lilygo-t3-v1-6-1-868-prod: 868 MHz production build with serial debug off

Config Knobs

Key timing settings in include/config.h:

• METER_SAMPLE_INTERVAL_MS
• METER_SEND_INTERVAL_MS
• BATCH_ACK_TIMEOUT_MS
• BATCH_MAX_RETRIES
• BATCH_QUEUE_DEPTH
• BATCH_RETRY_POLICY (keep or drop on retry exhaustion)
• SERIAL_DEBUG_MODE_FLAG (build flag) / SERIAL_DEBUG_DUMP_JSON
• LORA_SEND_BYPASS (debug only)
• ENABLE_SD_LOGGING / PIN_SD_CS
• SENDER_TIMESYNC_WINDOW_MS
• SENDER_TIMESYNC_CHECK_SEC_FAST / SENDER_TIMESYNC_CHECK_SEC_SLOW
• SD_HISTORY_MAX_DAYS / SD_HISTORY_MIN_RES_MIN
• SD_HISTORY_MAX_BINS / SD_HISTORY_TIME_BUDGET_MS
• WEB_AUTH_REQUIRE_STA / WEB_AUTH_REQUIRE_AP / WEB_AUTH_DEFAULT_USER / WEB_AUTH_DEFAULT_PASS

Limits & Known Constraints

• Compression: MeterData uses lightweight RLE (good for JSON but not optimal).
• OBIS parsing: supports IEC 62056-21 ASCII (Mode D); may need tuning for some meters.
• Payload size: single JSON frames < 256 bytes (ArduinoJson static doc); binary batch frames are chunked and reassembled (typically 1 chunk).
• Battery ADC: uses simple linear calibration constant in power_manager.cpp.
• OLED: no hardware reset line is used (matches working reference).
• Batch ACKs: sender waits for ACK after a batch and retries up to BATCH_MAX_RETRIES with BATCH_ACK_TIMEOUT_MS between attempts.

SD Logging (Receiver)

Optional CSV logging to microSD (FAT32) when ENABLE_SD_LOGGING = true.

• Path: /dd3/<device_id>/YYYY-MM-DD.csv
• Columns: ts_utc,p_w,p1_w,p2_w,p3_w,e_kwh,bat_v,bat_pct,rssi,snr,err_m,err_d,err_tx,err_last
• err_last is written as text (meter, decode, loratx) only on the last sample of a batch that reports an error.
• Files are downloadable from the main UI page.
• Downloads only allow absolute paths under /dd3/, reject .., backslashes, and repeated slashes, and enforce a max path length.
• History chart on sender page stream-parses CSVs and bins data in the background.
• SD uses the on-board microSD SPI pins (CS=13, MOSI=15, SCK=14, MISO=2).

Files & Modules

• include/config.h, src/config.cpp: pins, radio settings, sender IDs
• include/data_model.h, src/data_model.cpp: MeterData + ID init
• include/json_codec.h, src/json_codec.cpp: JSON encode/decode
• include/compressor.h, src/compressor.cpp: RLE compression
• include/lora_transport.h, src/lora_transport.cpp: LoRa packet + CRC
• src/payload_codec.h, src/payload_codec.cpp: binary batch encoder/decoder
• include/meter_driver.h, src/meter_driver.cpp: IEC 62056-21 ASCII parse
• include/power_manager.h, src/power_manager.cpp: ADC + sleep
• include/time_manager.h, src/time_manager.cpp: NTP + time sync
• include/wifi_manager.h, src/wifi_manager.cpp: NVS config + WiFi
• include/mqtt_client.h, src/mqtt_client.cpp: MQTT publish
• include/web_server.h, src/web_server.cpp: AP/STA web pages
• include/display_ui.h, src/display_ui.cpp: OLED pages + control
• include/test_mode.h, src/test_mode.cpp: test sender/receiver
• src/main.cpp: role detection and main loop

Quick Start

1. Set role jumper on GPIO14:
   • LOW: sender
   • HIGH: receiver
2. OLED control on GPIO13:
   • HIGH: always on
   • LOW: auto-off after 10 minutes
3. Build and upload:

pio run -e lilygo-t3-v1-6-1 -t upload --upload-port COMx

Test mode:

pio run -e lilygo-t3-v1-6-1-test -t upload --upload-port COMx

868 MHz builds:

pio run -e lilygo-t3-v1-6-1-868 -t upload --upload-port COMx

868 MHz test mode:

pio run -e lilygo-t3-v1-6-1-868-test -t upload --upload-port COMx