diff --git a/README.md b/README.md index e6f59ee..7cd0664 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # DD3 LoRa Bridge (Multi-Sender) -Unified firmware for LilyGO T3 v1.6.1 (ESP32 + SX1276 + SSD1306) that runs as **Sender** or **Receiver** based on a GPIO jumper. Senders read DD3 smart meter values, compress JSON, and transmit over LoRa. The receiver validates packets, publishes to MQTT, provides a web UI, and shows per-sender status on the OLED. +Unified firmware for LilyGO T3 v1.6.1 (ESP32 + SX1276 + SSD1306) that runs as **Sender** or **Receiver** based on a GPIO jumper. Senders read DD3 smart meter values and transmit compact binary batches over LoRa. The receiver validates packets, publishes to MQTT, provides a web UI, and shows per-sender status on the OLED. ## Hardware Board: **LilyGO T3 LoRa32 v1.6.1** (ESP32 + SX1276 + SSD1306 128x64 + LiPo) @@ -46,7 +46,7 @@ Variants: - Total power: 1-0:16.7.0*255 - Phase power: 36.7 / 56.7 / 76.7 - Reads battery voltage and estimates SoC. -- Builds JSON payload, compresses, wraps in LoRa packet, transmits. +- Builds compact binary batch payload, wraps in LoRa packet, transmits. - Light sleeps between meter reads; batches are sent every 30s. - Listens for LoRa time sync packets to set UTC clock. - Uses DS3231 RTC after boot if no time sync has arrived yet. @@ -59,9 +59,8 @@ void sender_loop() { read_battery(data); // VBAT + SoC if (time_to_send_batch()) { - json = meterBatchToJson(samples, batch_id); // bat_v per batch, t_first/t_last included - compressed = compressBuffer(json); - lora_send(packet(MeterBatch, compressed)); + payload = encode_batch(samples, batch_id); // compact binary batch + lora_send(packet(MeterBatch, payload)); } display_set_last_meter(data); @@ -79,14 +78,14 @@ void sender_loop() { bool meter_read(MeterData &data); // parse OBIS fields void read_battery(MeterData &data); // ADC -> volts + percent bool meterDataToJson(const MeterData&, String&); -bool compressBuffer(const uint8_t*, size_t, uint8_t*, size_t, size_t&); +bool compressBuffer(const uint8_t*, size_t, uint8_t*, size_t, size_t&); // MeterData only bool lora_send(const LoraPacket &pkt); // add header + CRC16 and transmit ``` ### Receiver (USB-powered) - WiFi STA connect using stored config; if not available/fails, starts AP. - NTP sync (UTC) and local display in Europe/Berlin. -- Receives LoRa packets, verifies CRC16, decompresses, parses JSON. +- Receives LoRa packets, verifies CRC16, decompresses MeterData JSON, decodes binary batches. - Publishes meter JSON to MQTT. - Sends ACKs for MeterBatch packets and de-duplicates by batch_id. - Web UI: @@ -105,8 +104,8 @@ void receiver_loop() { mqtt_publish_state(data); } } else if (pkt.type == MeterBatch) { - json = reassemble_and_decompress_batch(pkt); - for (sample in jsonToMeterBatch(json)) { // uses t_first/t_last for jittered timestamps + batch = reassemble_and_decode_batch(pkt); + for (sample in batch) { update_sender_status(sample); mqtt_publish_state(sample); } @@ -128,7 +127,7 @@ void receiver_loop() { ```cpp bool lora_receive(LoraPacket &pkt, uint32_t timeout_ms); bool jsonToMeterData(const String &json, MeterData &data); -bool jsonToMeterBatch(const String &json, MeterData *samples, size_t max, size_t &count); +bool decode_batch(const uint8_t *buf, size_t len, BatchInput *out); bool mqtt_publish_state(const MeterData &data); void web_server_loop(); // AP or STA UI void time_send_timesync(uint16_t self_id); @@ -170,16 +169,16 @@ Packet layout: [1] role (0=sender, 1=receiver) [2..3] device_id_short (uint16) [4] payload_type (0=meter, 1=test, 2=time_sync, 3=meter_batch, 4=ack) -[5..N-3] compressed payload +[5..N-3] payload bytes (compressed JSON for MeterData, binary for MeterBatch/Test/TimeSync) [N-2..N-1] CRC16 (bytes 0..N-3) ``` LoRa radio settings: - Frequency: **433 MHz** or **868 MHz** (set by build env via `LORA_FREQUENCY_HZ`) -- SF10, BW 125 kHz, CR 4/5, CRC on, Sync Word 0x34 +- SF11, BW 125 kHz, CR 4/5, CRC on, Sync Word 0x34 ## Data Format -JSON payload (sender + MQTT): +MeterData JSON (sender + MQTT): ```json { @@ -195,36 +194,31 @@ JSON payload (sender + MQTT): } ``` -MeterBatch JSON (compressed over LoRa) uses per-field arrays with integer units for easier ingestion: +### Binary MeterBatch Payload (LoRa) +Fixed header (little-endian): +- `magic` u16 = 0xDDB3 +- `schema` u8 = 1 +- `flags` u8 = 0x01 (bit0 = signed phases) +- `sender_id` u16 (1..NUM_SENDERS, maps to `EXPECTED_SENDER_IDS`) +- `batch_id` u16 +- `t_last` u32 (unix seconds of last sample) +- `dt_s` u8 (seconds, >0) +- `n` u8 (sample count, <=30) +- `battery_mV` u16 -```json -{ - "schema": 1, - "sender": "s01", - "batch_id": 1842, - "t0": 1738288000, - "t_first": 1738288000, - "t_last": 1738288030, - "dt_s": 1, - "n": 3, - "e_wh": [123456700, 123456701, 123456701], - "p_w": [930, 940, 950], - "p1_w": [480, 490, 500], - "p2_w": [450, 450, 450], - "p3_w": [0, 0, 0], - "bat_v": 3.92, - "meta": { - "rssi": -92, - "snr": 7.5, - "rx_ts": 1738288031 - } -} -``` +Body: +- `E0` u32 (absolute energy in Wh) +- `dE[1..n-1]` ULEB128 (delta vs previous, >=0) +- `P1_0` s16 (absolute W) +- `dP1[1..n-1]` signed varint (ZigZag + ULEB128) +- `P2_0` s16 +- `dP2[1..n-1]` signed varint +- `P3_0` s16 +- `dP3[1..n-1]` signed varint Notes: -- `sender` maps to `EXPECTED_SENDER_IDS` order (`s01` = first sender). -- `meta` is injected by the receiver after batch reassembly. -- `bat_v` is a single batch-level value (percent is calculated locally). +- Receiver reconstructs timestamps from `t_last` and `dt_s`. +- Total power is computed on receiver as `p1 + p2 + p3`. ## Device IDs - Derived from WiFi STA MAC. @@ -291,9 +285,9 @@ Key timing settings in `include/config.h`: - `LORA_SEND_BYPASS` (debug only) ## Limits & Known Constraints -- **Compression**: uses lightweight RLE (good for JSON but not optimal). +- **Compression**: MeterData uses lightweight RLE (good for JSON but not optimal). - **OBIS parsing**: supports IEC 62056-21 ASCII (Mode D); may need tuning for some meters. -- **Payload size**: single JSON frames < 256 bytes (ArduinoJson static doc); batch frames are chunked and reassembled. +- **Payload size**: single JSON frames < 256 bytes (ArduinoJson static doc); binary batch frames are chunked and reassembled (typically 1 chunk). - **Battery ADC**: uses simple linear calibration constant in `power_manager.cpp`. - **OLED**: no hardware reset line is used (matches working reference). - **Batch ACKs**: sender waits for ACK after a batch and retries up to `BATCH_MAX_RETRIES` with `BATCH_ACK_TIMEOUT_MS` between attempts. @@ -304,6 +298,7 @@ Key timing settings in `include/config.h`: - `include/json_codec.h`, `src/json_codec.cpp`: JSON encode/decode - `include/compressor.h`, `src/compressor.cpp`: RLE compression - `include/lora_transport.h`, `src/lora_transport.cpp`: LoRa packet + CRC +- `src/payload_codec.h`, `src/payload_codec.cpp`: binary batch encoder/decoder - `include/meter_driver.h`, `src/meter_driver.cpp`: IEC 62056-21 ASCII parse - `include/power_manager.h`, `src/power_manager.cpp`: ADC + sleep - `include/time_manager.h`, `src/time_manager.cpp`: NTP + time sync diff --git a/include/config.h b/include/config.h index c9e8fc9..59cfc60 100644 --- a/include/config.h +++ b/include/config.h @@ -49,7 +49,7 @@ constexpr uint8_t PIN_METER_RX = 34; #define LORA_FREQUENCY_HZ 433E6 #endif constexpr long LORA_FREQUENCY = LORA_FREQUENCY_HZ; -constexpr uint8_t LORA_SPREADING_FACTOR = 10; +constexpr uint8_t LORA_SPREADING_FACTOR = 11; constexpr long LORA_BANDWIDTH = 125E3; constexpr uint8_t LORA_CODING_RATE = 5; constexpr uint8_t LORA_SYNC_WORD = 0x34;