C3MA/DD3-LoRa-Bridge-MultiSender
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README for binary batch payload and SF11

Browse Source
This commit is contained in:
acidburns
2026-02-01 22:42:26 +01:00
parent e24798eb55
commit e5c4e04ff9
2 changed files with 38 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files

79
README.md
View File

@@ -1,6 +1,6 @@
# DD3 LoRa Bridge (Multi-Sender)
Unified firmware for LilyGO T3 v1.6.1 (ESP32 + SX1276 + SSD1306) that runs as **Sender** or **Receiver** based on a GPIO jumper. Senders read DD3 smart meter values, compress JSON, and transmit over LoRa. The receiver validates packets, publishes to MQTT, provides a web UI, and shows per-sender status on the OLED.
Unified firmware for LilyGO T3 v1.6.1 (ESP32 + SX1276 + SSD1306) that runs as **Sender** or **Receiver** based on a GPIO jumper. Senders read DD3 smart meter values and transmit compact binary batches over LoRa. The receiver validates packets, publishes to MQTT, provides a web UI, and shows per-sender status on the OLED.
## Hardware
Board: **LilyGO T3 LoRa32 v1.6.1** (ESP32 + SX1276 + SSD1306 128x64 + LiPo)
@@ -46,7 +46,7 @@ Variants:
- Total power: 1-0:16.7.0*255
- Phase power: 36.7 / 56.7 / 76.7
- Reads battery voltage and estimates SoC.
- Builds JSON payload, compresses, wraps in LoRa packet, transmits.
- Builds compact binary batch payload, wraps in LoRa packet, transmits.
- Light sleeps between meter reads; batches are sent every 30s.
- Listens for LoRa time sync packets to set UTC clock.
- Uses DS3231 RTC after boot if no time sync has arrived yet.
@@ -59,9 +59,8 @@ void sender_loop() {
read_battery(data); // VBAT + SoC
if (time_to_send_batch()) {
json = meterBatchToJson(samples, batch_id); // bat_v per batch, t_first/t_last included
compressed = compressBuffer(json);
lora_send(packet(MeterBatch, compressed));
payload = encode_batch(samples, batch_id); // compact binary batch
lora_send(packet(MeterBatch, payload));
}
display_set_last_meter(data);
@@ -79,14 +78,14 @@ void sender_loop() {
bool meter_read(MeterData &data); // parse OBIS fields
void read_battery(MeterData &data); // ADC -> volts + percent
bool meterDataToJson(const MeterData&, String&);
bool compressBuffer(const uint8_t*, size_t, uint8_t*, size_t, size_t&);
bool compressBuffer(const uint8_t*, size_t, uint8_t*, size_t, size_t&); // MeterData only
bool lora_send(const LoraPacket &pkt); // add header + CRC16 and transmit
```
### Receiver (USB-powered)
- WiFi STA connect using stored config; if not available/fails, starts AP.
- NTP sync (UTC) and local display in Europe/Berlin.
- Receives LoRa packets, verifies CRC16, decompresses, parses JSON.
- Receives LoRa packets, verifies CRC16, decompresses MeterData JSON, decodes binary batches.
- Publishes meter JSON to MQTT.
- Sends ACKs for MeterBatch packets and de-duplicates by batch_id.
- Web UI:
@@ -105,8 +104,8 @@ void receiver_loop() {
mqtt_publish_state(data);
}
} else if (pkt.type == MeterBatch) {
json = reassemble_and_decompress_batch(pkt);
for (sample in jsonToMeterBatch(json)) { // uses t_first/t_last for jittered timestamps
batch = reassemble_and_decode_batch(pkt);
for (sample in batch) {
update_sender_status(sample);
mqtt_publish_state(sample);
}
@@ -128,7 +127,7 @@ void receiver_loop() {
```cpp
bool lora_receive(LoraPacket &pkt, uint32_t timeout_ms);
bool jsonToMeterData(const String &json, MeterData &data);
bool jsonToMeterBatch(const String &json, MeterData *samples, size_t max, size_t &count);
bool decode_batch(const uint8_t *buf, size_t len, BatchInput *out);
bool mqtt_publish_state(const MeterData &data);
void web_server_loop(); // AP or STA UI
void time_send_timesync(uint16_t self_id);
@@ -170,16 +169,16 @@ Packet layout:
[1] role (0=sender, 1=receiver)
[2..3] device_id_short (uint16)
[4] payload_type (0=meter, 1=test, 2=time_sync, 3=meter_batch, 4=ack)
[5..N-3] compressed payload
[5..N-3] payload bytes (compressed JSON for MeterData, binary for MeterBatch/Test/TimeSync)
[N-2..N-1] CRC16 (bytes 0..N-3)
```
LoRa radio settings:
- Frequency: **433 MHz** or **868 MHz** (set by build env via `LORA_FREQUENCY_HZ`)
- SF10, BW 125 kHz, CR 4/5, CRC on, Sync Word 0x34
- SF11, BW 125 kHz, CR 4/5, CRC on, Sync Word 0x34
## Data Format
JSON payload (sender + MQTT):
MeterData JSON (sender + MQTT):
```json
{
@@ -195,36 +194,31 @@ JSON payload (sender + MQTT):
}
```
MeterBatch JSON (compressed over LoRa) uses per-field arrays with integer units for easier ingestion:
### Binary MeterBatch Payload (LoRa)
Fixed header (little-endian):
- `magic` u16 = 0xDDB3
- `schema` u8 = 1
- `flags` u8 = 0x01 (bit0 = signed phases)
- `sender_id` u16 (1..NUM_SENDERS, maps to `EXPECTED_SENDER_IDS`)
- `batch_id` u16
- `t_last` u32 (unix seconds of last sample)
- `dt_s` u8 (seconds, >0)
- `n` u8 (sample count, <=30)
- `battery_mV` u16
```json
{
"schema": 1,
"sender": "s01",
"batch_id": 1842,
"t0": 1738288000,
"t_first": 1738288000,
"t_last": 1738288030,
"dt_s": 1,
"n": 3,
"e_wh": [123456700, 123456701, 123456701],
"p_w": [930, 940, 950],
"p1_w": [480, 490, 500],
"p2_w": [450, 450, 450],
"p3_w": [0, 0, 0],
"bat_v": 3.92,
"meta": {
"rssi": -92,
"snr": 7.5,
"rx_ts": 1738288031
}
}
```
Body:
- `E0` u32 (absolute energy in Wh)
- `dE[1..n-1]` ULEB128 (delta vs previous, >=0)
- `P1_0` s16 (absolute W)
- `dP1[1..n-1]` signed varint (ZigZag + ULEB128)
- `P2_0` s16
- `dP2[1..n-1]` signed varint
- `P3_0` s16
- `dP3[1..n-1]` signed varint
Notes:
- `sender` maps to `EXPECTED_SENDER_IDS` order (`s01` = first sender).
- `meta` is injected by the receiver after batch reassembly.
- `bat_v` is a single batch-level value (percent is calculated locally).
- Receiver reconstructs timestamps from `t_last` and `dt_s`.
- Total power is computed on receiver as `p1 + p2 + p3`.
## Device IDs
- Derived from WiFi STA MAC.
@@ -291,9 +285,9 @@ Key timing settings in `include/config.h`:
- `LORA_SEND_BYPASS` (debug only)
## Limits & Known Constraints
- **Compression**: uses lightweight RLE (good for JSON but not optimal).
- **Compression**: MeterData uses lightweight RLE (good for JSON but not optimal).
- **OBIS parsing**: supports IEC 62056-21 ASCII (Mode D); may need tuning for some meters.
- **Payload size**: single JSON frames < 256 bytes (ArduinoJson static doc); batch frames are chunked and reassembled.
- **Payload size**: single JSON frames < 256 bytes (ArduinoJson static doc); binary batch frames are chunked and reassembled (typically 1 chunk).
- **Battery ADC**: uses simple linear calibration constant in `power_manager.cpp`.
- **OLED**: no hardware reset line is used (matches working reference).
- **Batch ACKs**: sender waits for ACK after a batch and retries up to `BATCH_MAX_RETRIES` with `BATCH_ACK_TIMEOUT_MS` between attempts.
@@ -304,6 +298,7 @@ Key timing settings in `include/config.h`:
- `include/json_codec.h`, `src/json_codec.cpp`: JSON encode/decode
- `include/compressor.h`, `src/compressor.cpp`: RLE compression
- `include/lora_transport.h`, `src/lora_transport.cpp`: LoRa packet + CRC
- `src/payload_codec.h`, `src/payload_codec.cpp`: binary batch encoder/decoder
- `include/meter_driver.h`, `src/meter_driver.cpp`: IEC 62056-21 ASCII parse
- `include/power_manager.h`, `src/power_manager.cpp`: ADC + sleep
- `include/time_manager.h`, `src/time_manager.cpp`: NTP + time sync