Remove legacy state JSON size cutoff and raise doc capacity

README.md

@@ -1,88 +1,135 @@
 # DD3-LoRa-Bridge-MultiSender
 
-Unified firmware for LilyGO T3 v1.6.1 (ESP32 + SX1276 + SSD1306) running as either:
-- `Sender`: reads meter values and sends binary batches over LoRa.
-- `Receiver`: accepts batches, ACKs with optional time, publishes to MQTT/web.
+Firmware for LilyGO T3 v1.6.1 (`ESP32 + SX1276 + SSD1306`) that runs as either:
+- `Sender` (PIN `GPIO14` HIGH): reads one IEC 62056-21 meter, batches samples, sends over LoRa.
+- `Receiver` (PIN `GPIO14` LOW): receives/ACKs batches, publishes MQTT, serves web UI, logs to SD.
 
-## Protocol (minimal)
+## Current Architecture
 
-Frame format:
+- Single codebase, role selected at boot via `detect_role()` (`include/config.h`, `src/config.cpp`).
+- LoRa link uses explicit CRC16 frame protection in firmware (`src/lora_transport.cpp`).
+- Sender batches up to `30` samples and retries on missing ACK (`BATCH_MAX_RETRIES=2`, retry policy `Keep`).
+- Sender meter parsing is decoupled from LoRa ACK waits using a dedicated FreeRTOS reader task + queue (`src/main.cpp`).
+- Receiver uses STA mode when config is valid, otherwise AP fallback with web config.
+- No debug auto-reboot timer is active in normal firmware loops.
 
-`[msg_kind:1][dev_id_short:2][payload...][crc16:2]`
+## LoRa Frame Protocol (Current)
+
+On-air frame format:
+
+`[msg_kind:1][device_short_id:2][payload...][crc16:2]`
 
 `msg_kind`:
-- `0`: `BATCH_UP` (Sender -> Receiver)
-- `1`: `ACK_DOWN` (Receiver -> Sender)
+- `0` = `BatchUp`
+- `1` = `AckDown`
 
-Removed from protocol:
-- protocol version field
-- payload type field
-- MeterData JSON/compressed LoRa path
-- standalone TimeSync packets
+### `BatchUp`
 
-CRC16 validation is still required on every frame.
+Transport is chunked (`batch_id`, `chunk_index`, `chunk_count`, `total_len`) and reassembled before payload decode.
 
-## Payloads
+Payload codec (`src/payload_codec.cpp`) currently uses:
+- `kMagic=0xDDB3`
+- `kSchema=2`
+- metadata: sender, batch, timestamp, interval, battery, fault counters
+- data arrays: `energy_wh[]`, `p1_w[]`, `p2_w[]`, `p3_w[]`
 
-### 1) `BATCH_UP`
-- Uses existing binary batch/chunk transport.
-- `sample_count == 0` is valid and means `SYNC_REQUEST`.
+`n == 0` is valid and used for sync request packets.
 
-### 2) `ACK_DOWN` (7 bytes)
-- `flags` (`u8`): bit0 = `time_valid`
-- `batch_id` (`u16`, big-endian)
-- `epoch_utc` (`u32`, big-endian)
+### `AckDown` (7 bytes)
 
-Receiver sets:
-- `time_valid=1` only when receiver time is authoritative and sane.
-- Otherwise `time_valid=0` and `epoch_utc=0`.
+`[flags:1][batch_id_be:2][epoch_utc_be:4]`
 
-## Time bootstrap safety
+- `flags bit0`: `time_valid`
+- Receiver repeats ACK (`ACK_REPEAT_COUNT=3`, `ACK_REPEAT_DELAY_MS=200`).
+- Sender accepts time only if `time_valid=1` and `epoch >= MIN_ACCEPTED_EPOCH_UTC` (`2026-02-01 00:00:00 UTC`).
 
-Sender starts with:
+## Time Bootstrap Guardrail
+
+On sender boot:
 - `g_time_acquired=false`
-- no real sampling/batching
-- periodic `SYNC_REQUEST` every `SYNC_REQUEST_INTERVAL_MS` (default `15000ms`)
+- no normal sampling/transmit yet
+- sync request every `SYNC_REQUEST_INTERVAL_MS` (15s)
 
-Sender only accepts time from `ACK_DOWN` if:
-- `time_valid == 1`
-- `epoch_utc >= 2026-02-01 00:00:00 UTC` (`MIN_ACCEPTED_EPOCH_UTC = 1769904000`)
-
-Only then:
+Only after valid ACK time is received:
 - system time is set
-- `g_time_acquired=true`
-- normal 1 Hz sampling + batch transmit starts
+- normal 1 Hz sampling and periodic LoRa batch transmit start
 
-This guarantees no pre-`2026-02-01` epoch reaches MQTT or SD/DB paths.
+This blocks pre-threshold timestamps from MQTT/SD paths.
 
-## Receiver behavior
+Timezone handling:
+- Local time rendering uses `TIMEZONE_TZ` from `include/config.h`.
+- Default value is `CET-1CEST,M3.5.0/2,M10.5.0/3` and can be changed at compile time.
 
-On `BATCH_UP`:
-1. Decode batch/chunks.
-2. Send `ACK_DOWN` immediately.
-3. If `sample_count == 0`: treat as `SYNC_REQUEST`, do not publish MQTT/update stats.
-4. Else decode and publish samples as normal.
+## Sender Meter Path
 
-## Sender/Receiver debug logs (`SERIAL_DEBUG_MODE`)
+Implemented in `src/meter_driver.cpp` + sender loop in `src/main.cpp`:
 
-Sender:
-- `sync: request tx batch_id=%u`
-- `ack: rx ok batch_id=%u time_valid=%u epoch=%lu set=%u`
-- `ack: timeout batch_id=%u retry=%u`
+- UART: `Serial2`, RX pin `GPIO34` (`PIN_METER_RX`), `9600 7E1`
+- ESP32 RX buffer is enlarged to `8192` bytes to survive long LoRa blocking sections.
+- Frame detection: starts at `'/'`, ends at `'!'`, timeout protection included (`METER_FRAME_TIMEOUT_MS=20000`).
+- Parsing runs in a dedicated sender task and is handed to the main sender loop via queue.
+- Parsed OBIS values:
+  - `1-0:1.8.0` (total energy)
+  - `1-0:16.7.0` (total power)
+  - `1-0:36.7.0`, `56.7.0`, `76.7.0` (phase powers)
+- `1-0:1.8.0*Wh` is automatically scaled to kWh
 
-Receiver:
-- `ack: tx batch_id=%u time_valid=%u epoch=%lu samples=%u`
+Sender samples every second and transmits batches every 30 seconds.
 
-## Removed hardware dependency
+## Receiver Behavior
 
-DS3231 RTC support was removed:
-- no RTC files
-- no RTC init/load/set logic
-- no `ENABLE_DS3231` flow
+For valid `BatchUp` decode:
+1. Reassemble chunks and decode payload.
+2. Send `AckDown` immediately.
+3. Drop duplicate batches per sender (`batch_id` tracking).
+4. If `n==0`: treat as sync request only.
+5. Else convert samples, log to SD, update web UI, publish MQTT.
 
-## Build
+## MQTT Topics and Payloads
+
+State topic:
+- `smartmeter/<device_id>/state`
+
+Fault topic (retained):
+- `smartmeter/<device_id>/faults`
+
+State JSON fields (`src/json_codec.cpp`):
+- `id`, `ts`, `e_kwh`
+- `p_w`, `p1_w`, `p2_w`, `p3_w`
+- `bat_v`, `bat_pct`
+- optional link fields: `rssi`, `snr`
+- fault/reject fields: `err_last`, `rx_reject`, `rx_reject_text` (+ non-zero counters)
+
+Home Assistant discovery is enabled (`ENABLE_HA_DISCOVERY=true`) and publishes config topics under:
+- `homeassistant/sensor/<device_id>/<key>/config`
+
+## Web UI, Wi-Fi, Storage
+
+- Wi-Fi/MQTT/NTP/web-auth config persists in Preferences (`wifi_manager`).
+- AP fallback SSID prefix: `DD3-Bridge-`.
+- Default web credentials: `admin/admin`.
+- SD logging enabled (`ENABLE_SD_LOGGING=true`).
+
+## Build Environments
+
+From `platformio.ini`:
+
+- `lilygo-t3-v1-6-1`
+- `lilygo-t3-v1-6-1-test`
+- `lilygo-t3-v1-6-1-868`
+- `lilygo-t3-v1-6-1-868-test`
+- `lilygo-t3-v1-6-1-payload-test`
+- `lilygo-t3-v1-6-1-868-payload-test`
+- `lilygo-t3-v1-6-1-prod`
+- `lilygo-t3-v1-6-1-868-prod`
+
+Example:
 
 ```bash
-pio run -e lilygo-t3-v1-6-1
-pio run -e lilygo-t3-v1-6-1-test
+~/.platformio/penv/bin/pio run -e lilygo-t3-v1-6-1
 ```
+
+## Test Mode
+
+`ENABLE_TEST_MODE` replaces normal sender/receiver loops with dedicated test loops (`src/test_mode.cpp`).
+It sends/receives JSON test frames and publishes to `smartmeter/<device_id>/test`.

include/config.h

@@ -79,6 +79,7 @@ constexpr uint16_t SD_HISTORY_MAX_DAYS = 30;
 constexpr uint16_t SD_HISTORY_MIN_RES_MIN = 1;
 constexpr uint16_t SD_HISTORY_MAX_BINS = 4000;
 constexpr uint16_t SD_HISTORY_TIME_BUDGET_MS = 10;
+constexpr const char *TIMEZONE_TZ = "CET-1CEST,M3.5.0/2,M10.5.0/3";
 constexpr const char *AP_SSID_PREFIX = "DD3-Bridge-";
 constexpr const char *AP_PASSWORD = "changeme123";
 constexpr bool WEB_AUTH_REQUIRE_STA = true;

src/json_codec.cpp

@@ -3,6 +3,8 @@
 #include <limits.h>
 #include <math.h>
 
+static constexpr size_t STATE_JSON_DOC_CAPACITY = 512;
+
 static float round2(float value) {
   if (isnan(value)) {
     return value;
@@ -58,7 +60,7 @@ static void set_int_or_null(JsonDocument &doc, const char *key, float value) {
 }
 
 bool meterDataToJson(const MeterData &data, String &out_json) {
-  StaticJsonDocument<256> doc;
+  StaticJsonDocument<STATE_JSON_DOC_CAPACITY> doc;
   doc["id"] = short_id_from_device_id(data.device_id);
   doc["ts"] = data.ts_utc;
   char buf[16];
@@ -90,5 +92,5 @@ bool meterDataToJson(const MeterData &data, String &out_json) {
 
   out_json = "";
   size_t len = serializeJson(doc, out_json);
-  return len > 0 && len < 256;
+  return len > 0;
 }

src/main.cpp

@@ -17,6 +17,9 @@
 #ifdef ARDUINO_ARCH_ESP32
 #include <esp_task_wdt.h>
 #include <esp_system.h>
+#include <freertos/FreeRTOS.h>
+#include <freertos/queue.h>
+#include <freertos/task.h>
 #endif
 
 static DeviceRole g_role = DeviceRole::Sender;
@@ -26,7 +29,6 @@ static char g_device_id[16] = "";
 static SenderStatus g_sender_statuses[NUM_SENDERS];
 static bool g_ap_mode = false;
 static WifiMqttConfig g_cfg;
-static uint32_t g_boot_ms = 0;
 static FaultCounters g_sender_faults = {};
 static FaultCounters g_receiver_faults = {};
 static FaultCounters g_receiver_faults_published = {};
@@ -55,6 +57,10 @@ struct BatchBuffer {
   uint16_t batch_id;
   bool batch_id_valid;
   uint8_t count;
+  uint16_t attempt_count;
+  uint16_t valid_count;
+  uint16_t invalid_count;
+  FaultType last_error;
   MeterData samples[METER_BATCH_MAX_SAMPLES];
 };
 
@@ -96,8 +102,33 @@ static uint32_t g_last_meter_rx_ms = 0;
 static uint32_t g_meter_stale_seconds = 0;
 static bool g_time_acquired = false;
 static uint32_t g_last_sync_request_ms = 0;
+static uint32_t g_build_attempts = 0;
+static uint32_t g_build_valid = 0;
+static uint32_t g_build_invalid = 0;
+static constexpr uint32_t METER_SAMPLE_MAX_AGE_MS = 15000;
+struct MeterSampleEvent {
+  MeterData data;
+  uint32_t rx_ms;
+};
+#ifdef ARDUINO_ARCH_ESP32
+static QueueHandle_t g_meter_sample_queue = nullptr;
+static TaskHandle_t g_meter_reader_task = nullptr;
+static bool g_meter_reader_task_running = false;
+static constexpr UBaseType_t METER_SAMPLE_QUEUE_LEN = 8;
+static constexpr uint32_t METER_READER_TASK_STACK_WORDS = 4096;
+static constexpr UBaseType_t METER_READER_TASK_PRIORITY = 2;
+static constexpr BaseType_t METER_READER_TASK_CORE = 0;
+#endif
+
+enum class TxBuildError : uint8_t {
+  None = 0,
+  Encode = 1
+};
+
+static TxBuildError g_last_tx_build_error = TxBuildError::None;
 
 static void watchdog_kick();
+static void finish_inflight_batch();
 
 static void serial_debug_printf(const char *fmt, ...) {
   if (!SERIAL_DEBUG_MODE) {
@@ -111,6 +142,117 @@ static void serial_debug_printf(const char *fmt, ...) {
   Serial.println(buf);
 }
 
+static void set_last_meter_sample(const MeterData &parsed, uint32_t rx_ms) {
+  g_last_meter_data = parsed;
+  g_last_meter_valid = true;
+  g_last_meter_rx_ms = rx_ms;
+  g_meter_stale_seconds = 0;
+}
+
+static bool parse_meter_frame_sample(const char *frame, size_t frame_len, MeterData &parsed) {
+  parsed = {};
+  parsed.energy_total_kwh = NAN;
+  parsed.total_power_w = NAN;
+  parsed.phase_power_w[0] = NAN;
+  parsed.phase_power_w[1] = NAN;
+  parsed.phase_power_w[2] = NAN;
+  parsed.valid = false;
+  return meter_parse_frame(frame, frame_len, parsed);
+}
+
+#ifdef ARDUINO_ARCH_ESP32
+static void meter_queue_push_latest(const MeterSampleEvent &event) {
+  if (!g_meter_sample_queue) {
+    return;
+  }
+  if (xQueueSend(g_meter_sample_queue, &event, 0) == pdTRUE) {
+    return;
+  }
+  MeterSampleEvent dropped = {};
+  xQueueReceive(g_meter_sample_queue, &dropped, 0);
+  if (xQueueSend(g_meter_sample_queue, &event, 0) != pdTRUE && SERIAL_DEBUG_MODE) {
+    serial_debug_printf("meter: queue push failed");
+  }
+}
+
+static void meter_reader_task_entry(void *arg) {
+  (void)arg;
+  for (;;) {
+    const char *frame = nullptr;
+    size_t frame_len = 0;
+    if (!meter_poll_frame(frame, frame_len)) {
+      vTaskDelay(pdMS_TO_TICKS(5));
+      continue;
+    }
+
+    MeterData parsed = {};
+    if (parse_meter_frame_sample(frame, frame_len, parsed)) {
+      MeterSampleEvent event = {};
+      event.data = parsed;
+      event.rx_ms = millis();
+      meter_queue_push_latest(event);
+    }
+  }
+}
+
+static bool meter_reader_start() {
+  if (g_meter_reader_task_running) {
+    return true;
+  }
+  if (!g_meter_sample_queue) {
+    g_meter_sample_queue = xQueueCreate(METER_SAMPLE_QUEUE_LEN, sizeof(MeterSampleEvent));
+    if (!g_meter_sample_queue) {
+      if (SERIAL_DEBUG_MODE) {
+        serial_debug_printf("meter: queue alloc failed");
+      }
+      return false;
+    }
+  }
+
+  BaseType_t rc = xTaskCreatePinnedToCore(
+      meter_reader_task_entry,
+      "meter_reader",
+      METER_READER_TASK_STACK_WORDS,
+      nullptr,
+      METER_READER_TASK_PRIORITY,
+      &g_meter_reader_task,
+      METER_READER_TASK_CORE);
+  if (rc != pdPASS) {
+    if (SERIAL_DEBUG_MODE) {
+      serial_debug_printf("meter: task start failed rc=%ld", static_cast<long>(rc));
+    }
+    return false;
+  }
+  g_meter_reader_task_running = true;
+  serial_debug_printf("meter: reader task core=%ld queue=%u",
+                      static_cast<long>(METER_READER_TASK_CORE),
+                      static_cast<unsigned>(METER_SAMPLE_QUEUE_LEN));
+  return true;
+}
+#endif
+
+static void meter_reader_pump(uint32_t now_ms) {
+#ifdef ARDUINO_ARCH_ESP32
+  if (g_meter_reader_task_running && g_meter_sample_queue) {
+    MeterSampleEvent event = {};
+    while (xQueueReceive(g_meter_sample_queue, &event, 0) == pdTRUE) {
+      set_last_meter_sample(event.data, event.rx_ms);
+    }
+    return;
+  }
+#endif
+
+  const char *frame = nullptr;
+  size_t frame_len = 0;
+  if (!meter_poll_frame(frame, frame_len)) {
+    return;
+  }
+  MeterData parsed = {};
+  if (parse_meter_frame_sample(frame, frame_len, parsed)) {
+    set_last_meter_sample(parsed, now_ms);
+  }
+}
+
 static uint16_t g_last_batch_id_rx[NUM_SENDERS] = {};
 
 struct BatchRxState {
@@ -207,6 +349,10 @@ static void batch_queue_enqueue(const MeterData *samples, uint8_t count) {
   slot.batch_id = 0;
   slot.batch_id_valid = false;
   slot.count = count;
+  slot.attempt_count = static_cast<uint16_t>(g_build_attempts);
+  slot.valid_count = static_cast<uint16_t>(g_build_valid);
+  slot.invalid_count = static_cast<uint16_t>(g_build_invalid);
+  slot.last_error = g_sender_last_error;
   for (uint8_t i = 0; i < count; ++i) {
     slot.samples[i] = samples[i];
   }
@@ -214,6 +360,28 @@ static void batch_queue_enqueue(const MeterData *samples, uint8_t count) {
   g_batch_count++;
 }
 
+static void reset_build_counters() {
+  g_build_attempts = 0;
+  g_build_valid = 0;
+  g_build_invalid = 0;
+}
+
+static bool append_meter_sample(const MeterData &data, bool meter_ok) {
+  if (!meter_ok) {
+    g_build_invalid++;
+    return false;
+  }
+  g_last_sample_ts_utc = data.ts_utc;
+  g_build_samples[g_build_count++] = data;
+  g_build_valid++;
+  if (g_build_count >= METER_BATCH_MAX_SAMPLES) {
+    batch_queue_enqueue(g_build_samples, g_build_count);
+    g_build_count = 0;
+    reset_build_counters();
+  }
+  return true;
+}
+
 static uint32_t last_sample_ts() {
   if (g_last_sample_ts_utc == 0) {
     uint32_t now_utc = time_get_utc();
@@ -356,6 +524,23 @@ static bool float_to_i16_w(float value, int16_t &out) {
   return true;
 }
 
+static int16_t float_to_i16_w_clamped(float value, bool &clamped) {
+  clamped = false;
+  if (isnan(value)) {
+    return 0;
+  }
+  long rounded = lroundf(value);
+  if (rounded < INT16_MIN) {
+    clamped = true;
+    return INT16_MIN;
+  }
+  if (rounded > INT16_MAX) {
+    clamped = true;
+    return INT16_MAX;
+  }
+  return static_cast<int16_t>(rounded);
+}
+
 static uint16_t battery_mv_from_voltage(float value) {
   if (isnan(value) || value <= 0.0f) {
     return 0;
@@ -482,6 +667,15 @@ static bool prepare_inflight_from_queue() {
     batch->batch_id = g_batch_id;
     batch->batch_id_valid = true;
   }
+  if (SERIAL_DEBUG_MODE) {
+    serial_debug_printf("batch: id=%u desired=%u attempts=%u valid=%u invalid=%u err_last=%u",
+                        batch->batch_id,
+                        static_cast<unsigned>(METER_BATCH_MAX_SAMPLES),
+                        static_cast<unsigned>(batch->attempt_count),
+                        static_cast<unsigned>(batch->valid_count),
+                        static_cast<unsigned>(batch->invalid_count),
+                        static_cast<unsigned>(batch->last_error));
+  }
   g_inflight_count = batch->count;
   g_inflight_batch_id = batch->batch_id;
   for (uint8_t i = 0; i < g_inflight_count; ++i) {
@@ -492,6 +686,7 @@ static bool prepare_inflight_from_queue() {
 }
 
 static bool send_inflight_batch(uint32_t ts_for_display) {
+  g_last_tx_build_error = TxBuildError::None;
   if (!g_inflight_active) {
     return false;
   }
@@ -510,19 +705,44 @@ static bool send_inflight_batch(uint32_t ts_for_display) {
   input.err_tx = g_sender_faults.lora_tx_fail > 255 ? 255 : static_cast<uint8_t>(g_sender_faults.lora_tx_fail);
   input.err_last = static_cast<uint8_t>(g_sender_last_error);
   input.err_rx_reject = static_cast<uint8_t>(g_sender_rx_reject_reason);
+  uint8_t energy_regressions = 0;
+  uint8_t phase_clamps = 0;
   for (uint8_t i = 0; i < input.n; ++i) {
     input.energy_wh[i] = kwh_to_wh_from_float(g_inflight_samples[i].energy_total_kwh);
-    if (!float_to_i16_w(g_inflight_samples[i].phase_power_w[0], input.p1_w[i]) ||
-        !float_to_i16_w(g_inflight_samples[i].phase_power_w[1], input.p2_w[i]) ||
-        !float_to_i16_w(g_inflight_samples[i].phase_power_w[2], input.p3_w[i])) {
-      return false;
+    if (i > 0 && input.energy_wh[i] < input.energy_wh[i - 1]) {
+      input.energy_wh[i] = input.energy_wh[i - 1];
+      if (energy_regressions < 255) {
+        energy_regressions++;
+      }
     }
+    bool c1 = false;
+    bool c2 = false;
+    bool c3 = false;
+    input.p1_w[i] = float_to_i16_w_clamped(g_inflight_samples[i].phase_power_w[0], c1);
+    input.p2_w[i] = float_to_i16_w_clamped(g_inflight_samples[i].phase_power_w[1], c2);
+    input.p3_w[i] = float_to_i16_w_clamped(g_inflight_samples[i].phase_power_w[2], c3);
+    if (c1 && phase_clamps < 255) {
+      phase_clamps++;
+    }
+    if (c2 && phase_clamps < 255) {
+      phase_clamps++;
+    }
+    if (c3 && phase_clamps < 255) {
+      phase_clamps++;
+    }
+  }
+  if (SERIAL_DEBUG_MODE && (energy_regressions > 0 || phase_clamps > 0)) {
+    serial_debug_printf("tx: sanitize batch_id=%u energy_regress=%u phase_clamps=%u",
+                        g_inflight_batch_id,
+                        static_cast<unsigned>(energy_regressions),
+                        static_cast<unsigned>(phase_clamps));
   }
 
   static uint8_t encoded[BATCH_MAX_COMPRESSED];
   size_t encoded_len = 0;
   uint32_t encode_start = millis();
   if (!encode_batch(input, encoded, sizeof(encoded), &encoded_len)) {
+    g_last_tx_build_error = TxBuildError::Encode;
     return false;
   }
   uint32_t encode_ms = millis() - encode_start;
@@ -568,6 +788,13 @@ static bool send_meter_batch(uint32_t ts_for_display) {
     g_last_sent_batch_id = g_inflight_batch_id;
     g_batch_ack_pending = true;
   } else {
+    if (g_last_tx_build_error == TxBuildError::Encode) {
+      serial_debug_printf("tx: encode failed batch_id=%u dropped", g_inflight_batch_id);
+      note_fault(g_sender_faults, g_sender_last_error, g_sender_last_error_utc, g_sender_last_error_ms, FaultType::Decode);
+      display_set_last_error(g_sender_last_error, g_sender_last_error_utc, g_sender_last_error_ms);
+      finish_inflight_batch();
+      return false;
+    }
     g_inflight_active = false;
     g_inflight_count = 0;
     g_inflight_batch_id = 0;
@@ -587,6 +814,15 @@ static bool send_sync_request() {
   g_inflight_sync_request = true;
   g_inflight_count = 0;
   g_inflight_batch_id = g_batch_id;
+  if (SERIAL_DEBUG_MODE && g_build_attempts > 0) {
+    serial_debug_printf("batch: id=%u desired=%u attempts=%u valid=%u invalid=%u err_last=%u sync=1",
+                        g_inflight_batch_id,
+                        static_cast<unsigned>(METER_BATCH_MAX_SAMPLES),
+                        static_cast<unsigned>(g_build_attempts),
+                        static_cast<unsigned>(g_build_valid),
+                        static_cast<unsigned>(g_build_invalid),
+                        static_cast<unsigned>(g_sender_last_error));
+  }
   bool ok = send_inflight_batch(time_get_utc());
   if (ok) {
     g_last_sent_batch_id = g_inflight_batch_id;
@@ -698,7 +934,6 @@ void setup() {
 #endif
 
   watchdog_init();
-  g_boot_ms = millis();
   g_role = detect_role();
   init_device_ids(g_short_id, g_device_id, sizeof(g_device_id));
   display_set_role(g_role);
@@ -718,6 +953,11 @@ void setup() {
     power_sender_init();
     power_configure_unused_pins_sender();
     meter_init();
+#ifdef ARDUINO_ARCH_ESP32
+    if (!meter_reader_start()) {
+      serial_debug_printf("meter: using inline polling fallback");
+    }
+#endif
     g_last_sample_ms = millis() - METER_SAMPLE_INTERVAL_MS;
     g_last_send_ms = millis();
     g_last_sync_request_ms = millis() - SYNC_REQUEST_INTERVAL_MS;
@@ -775,42 +1015,33 @@ static void sender_loop() {
                         g_batch_retry_count);
   }
 
-  if (g_time_acquired) {
-    const char *frame = nullptr;
-    size_t frame_len = 0;
-    if (meter_poll_frame(frame, frame_len)) {
-      MeterData parsed = {};
-      parsed.energy_total_kwh = NAN;
-      parsed.total_power_w = NAN;
-      parsed.phase_power_w[0] = NAN;
-      parsed.phase_power_w[1] = NAN;
-      parsed.phase_power_w[2] = NAN;
-      parsed.valid = false;
-      if (meter_parse_frame(frame, frame_len, parsed)) {
-        g_last_meter_data = parsed;
-        g_last_meter_valid = true;
-        g_last_meter_rx_ms = now_ms;
-        g_meter_stale_seconds = 0;
-      }
-    }
+  meter_reader_pump(now_ms);
 
-    if (now_ms - g_last_sample_ms >= METER_SAMPLE_INTERVAL_MS) {
-      g_last_sample_ms = now_ms;
+  if (g_time_acquired) {
+    while (now_ms - g_last_sample_ms >= METER_SAMPLE_INTERVAL_MS) {
+      g_last_sample_ms += METER_SAMPLE_INTERVAL_MS;
       MeterData data = {};
       data.short_id = g_short_id;
       strncpy(data.device_id, g_device_id, sizeof(data.device_id));
+      data.energy_total_kwh = NAN;
+      data.total_power_w = NAN;
+      data.phase_power_w[0] = NAN;
+      data.phase_power_w[1] = NAN;
+      data.phase_power_w[2] = NAN;
 
-      bool meter_ok = g_last_meter_valid;
+      g_build_attempts++;
+      uint32_t meter_age_ms = g_last_meter_valid ? (now_ms - g_last_meter_rx_ms) : UINT32_MAX;
+      // Reuse recent good samples to bridge short parser gaps without accepting stale data forever.
+      bool meter_ok = g_last_meter_valid && meter_age_ms <= METER_SAMPLE_MAX_AGE_MS;
       if (meter_ok) {
         data.energy_total_kwh = g_last_meter_data.energy_total_kwh;
         data.total_power_w = g_last_meter_data.total_power_w;
         data.phase_power_w[0] = g_last_meter_data.phase_power_w[0];
         data.phase_power_w[1] = g_last_meter_data.phase_power_w[1];
         data.phase_power_w[2] = g_last_meter_data.phase_power_w[2];
-        uint32_t age_ms = now_ms - g_last_meter_rx_ms;
-        g_meter_stale_seconds = age_ms >= 1000 ? (age_ms / 1000) : 0;
+        g_meter_stale_seconds = meter_age_ms >= 1000 ? (meter_age_ms / 1000) : 0;
       } else {
-        g_meter_stale_seconds++;
+        g_meter_stale_seconds = g_last_meter_valid ? (meter_age_ms / 1000) : (g_meter_stale_seconds + 1);
       }
       if (!meter_ok) {
         note_fault(g_sender_faults, g_sender_last_error, g_sender_last_error_utc, g_sender_last_error_ms, FaultType::MeterRead);
@@ -822,14 +1053,27 @@ static void sender_loop() {
       data.battery_voltage_v = g_last_battery_voltage_v;
       data.battery_percent = g_last_battery_percent;
       data.rx_reject_reason = static_cast<uint8_t>(g_sender_rx_reject_reason);
-      data.ts_utc = time_get_utc();
+      uint32_t sample_ts_utc = time_get_utc();
+      if (sample_ts_utc > 0 && now_ms > g_last_sample_ms) {
+        uint32_t lag_s = (now_ms - g_last_sample_ms) / 1000;
+        if (sample_ts_utc > lag_s) {
+          sample_ts_utc -= lag_s;
+        }
+      }
+      data.ts_utc = sample_ts_utc;
       data.valid = meter_ok;
 
-      g_last_sample_ts_utc = data.ts_utc;
-      g_build_samples[g_build_count++] = data;
-      if (g_build_count >= METER_BATCH_MAX_SAMPLES) {
-        batch_queue_enqueue(g_build_samples, g_build_count);
-        g_build_count = 0;
+      bool appended = append_meter_sample(data, meter_ok);
+      if (SERIAL_DEBUG_MODE) {
+        serial_debug_printf("sample: i=%lu ok=%u appended=%u e_kwh=%.3f p1=%.1f p2=%.1f p3=%.1f ms=%lu",
+                            static_cast<unsigned long>(g_build_attempts),
+                            meter_ok ? 1U : 0U,
+                            appended ? 1U : 0U,
+                            static_cast<double>(data.energy_total_kwh),
+                            static_cast<double>(data.phase_power_w[0]),
+                            static_cast<double>(data.phase_power_w[1]),
+                            static_cast<double>(data.phase_power_w[2]),
+                            static_cast<unsigned long>(now_ms));
       }
       display_set_last_meter(data);
       display_set_last_read(meter_ok, data.ts_utc);
@@ -837,7 +1081,18 @@ static void sender_loop() {
 
     if (!g_batch_ack_pending && now_ms - g_last_send_ms >= METER_SEND_INTERVAL_MS) {
       g_last_send_ms = now_ms;
-      send_meter_batch(last_sample_ts());
+      if (g_build_count > 0) {
+        batch_queue_enqueue(g_build_samples, g_build_count);
+        g_build_count = 0;
+        reset_build_counters();
+      }
+      if (g_batch_count > 0) {
+        send_meter_batch(last_sample_ts());
+      } else if (g_build_attempts > 0) {
+        if (send_sync_request()) {
+          reset_build_counters();
+        }
+      }
     }
   } else {
     if (!g_batch_ack_pending && now_ms - g_last_sync_request_ms >= SYNC_REQUEST_INTERVAL_MS) {

src/meter_driver.cpp

@@ -4,7 +4,9 @@
 #include <stdlib.h>
 #include <string.h>
 
-static constexpr uint32_t METER_FRAME_TIMEOUT_MS = 1500;
+// LoRa TX/RX windows can block the main loop for several seconds at SF12.
+// Keep partial frame state long enough so valid telegrams are not dropped.
+static constexpr uint32_t METER_FRAME_TIMEOUT_MS = 20000;
 static constexpr size_t METER_FRAME_MAX = 512;
 
 enum class MeterRxState : uint8_t {
@@ -24,6 +26,10 @@ static uint32_t g_rx_timeout = 0;
 static uint32_t g_last_log_ms = 0;
 
 void meter_init() {
+#ifdef ARDUINO_ARCH_ESP32
+  // Buffer enough serial data to survive long LoRa blocking sections.
+  Serial2.setRxBufferSize(8192);
+#endif
   Serial2.begin(9600, SERIAL_7E1, PIN_METER_RX, -1);
 }
 

src/time_manager.cpp

@@ -1,4 +1,5 @@
 #include "time_manager.h"
+#include "config.h"
 #include <time.h>
 
 static bool g_time_synced = false;
@@ -12,15 +13,20 @@ static void note_last_sync(uint32_t epoch) {
   g_last_sync_utc = epoch;
 }
 
+static void ensure_timezone_set() {
+  if (g_tz_set) {
+    return;
+  }
+  setenv("TZ", TIMEZONE_TZ, 1);
+  tzset();
+  g_tz_set = true;
+}
+
 void time_receiver_init(const char *ntp_server_1, const char *ntp_server_2) {
   const char *server1 = (ntp_server_1 && ntp_server_1[0] != '\0') ? ntp_server_1 : "pool.ntp.org";
   const char *server2 = (ntp_server_2 && ntp_server_2[0] != '\0') ? ntp_server_2 : "time.nist.gov";
   configTime(0, 0, server1, server2);
-  if (!g_tz_set) {
-    setenv("TZ", "CET-1CEST,M3.5.0/2,M10.5.0/3", 1);
-    tzset();
-    g_tz_set = true;
-  }
+  ensure_timezone_set();
 }
 
 uint32_t time_get_utc() {
@@ -40,11 +46,7 @@ bool time_is_synced() {
 }
 
 void time_set_utc(uint32_t epoch) {
-  if (!g_tz_set) {
-    setenv("TZ", "CET-1CEST,M3.5.0/2,M10.5.0/3", 1);
-    tzset();
-    g_tz_set = true;
-  }
+  ensure_timezone_set();
   struct timeval tv;
   tv.tv_sec = epoch;
   tv.tv_usec = 0;

src/web_server.cpp

@@ -57,6 +57,32 @@ static HistoryJob g_history = {};
 static constexpr size_t SD_LIST_MAX_FILES = 200;
 static constexpr size_t SD_DOWNLOAD_MAX_PATH = 160;
 
+static String format_utc_timestamp(uint32_t ts_utc) {
+  if (ts_utc == 0) {
+    return "n/a";
+  }
+  time_t t = static_cast<time_t>(ts_utc);
+  struct tm tm_utc;
+  gmtime_r(&t, &tm_utc);
+  char buf[32];
+  snprintf(buf, sizeof(buf), "%04d-%02d-%02d %02d:%02d:%02d UTC",
+           tm_utc.tm_year + 1900,
+           tm_utc.tm_mon + 1,
+           tm_utc.tm_mday,
+           tm_utc.tm_hour,
+           tm_utc.tm_min,
+           tm_utc.tm_sec);
+  return String(buf);
+}
+
+static uint32_t timestamp_age_seconds(uint32_t ts_utc) {
+  uint32_t now_utc = time_get_utc();
+  if (ts_utc == 0 || now_utc < ts_utc) {
+    return 0;
+  }
+  return now_utc - ts_utc;
+}
+
 static int32_t round_power_w(float value) {
   if (isnan(value)) {
     return 0;
@@ -374,6 +400,11 @@ static String render_sender_block(const SenderStatus &status) {
   if (!status.has_data) {
     s += "No data";
   } else {
+    s += "Last update: " + format_utc_timestamp(status.last_update_ts_utc);
+    if (time_is_synced()) {
+      s += " (" + String(timestamp_age_seconds(status.last_update_ts_utc)) + "s ago)";
+    }
+    s += "<br>";
     s += "Energy: " + String(status.last_data.energy_total_kwh, 2) + " kWh<br>";
     s += "Power: " + String(round_power_w(status.last_data.total_power_w)) + " W<br>";
     s += "P1/P2/P3: " + String(round_power_w(status.last_data.phase_power_w[0])) + " / " +