# DD3 LoRa Bridge Raspi Debugger

Headless Raspberry Pi Zero W project:
- WiFi client or fallback AP (`serial` / `serialserial`)
- Web portal (`http://192.168.4.1/` in AP mode)
- ESP32 USB serial bridge with live SSE stream and daily log files (`/home/pi/xxx_YYYY-MM-DD_HH-MM-SS.log`)
- 6-hour timeline page with CSV upload + merged timeline downloads
- Stable symlink to active log (`/home/pi/xxx.log`)
- RTC boot restore + NTP sync + RTC write-back
- Autostart via systemd

## Current State

Current implementation status:
- Boot flow restores system time from RTC (`hwclock -s`) before starting services.
- RTC device auto-detection checks: configured device, `/dev/rtc`, `/dev/rtc0`, `/dev/rtc1`.
- If WLAN is disconnected for ~60s, AP fallback starts on `wlan0`:
  - SSID: `serial`
  - Password: `serialserial`
  - AP IP: `192.168.4.1/24`
  - DHCP range: `192.168.4.10` to `192.168.4.200`
- Web portal is available on port `80`:
  - `/` WiFi scan + connect UI + system actions (reboot/shutdown)
  - `/serial` live serial console (SSE via `/events/serial`)
  - `/timeline` last-6-hours split timeline (serial + uploaded CSV data)
  - `/api/status` polling is reduced/throttled (15s) for Pi Zero W performance
- ESP32 serial bridge:
  - Auto-detects `/dev/ttyUSB*`, `/dev/ttyACM*`, `/dev/serial/by-id/*`
  - Reconnects automatically on unplug/replug
  - Daily log rollover at midnight with datetime filename
  - Each line written to file is timestamped with full local ISO datetime (including UTC offset)
  - SSE payload includes `line`, `ts_iso`, `ts_hms`, `source`
  - No log file is created while no serial device is connected
- Timeline CSV upload + merge:
  - Upload endpoint: `POST /api/timeline/uploads` (multipart field `file`)
  - Timeline data endpoint: `GET /api/timeline?hours=6&upload_id=<id>`
  - Download endpoint: `GET /api/timeline/download?kind=serial|merged&hours=6&upload_id=<id>&csrf_token=<token>`
  - Upload persistence: `/home/pi/timeline_uploads/<id>.csv` + sidecar metadata `/home/pi/timeline_uploads/<id>.json`
  - CSV parsing supports auto-detected timestamp columns (`ts_utc`, `timestamp`, `ts`, `unix`, `epoch`, `datetime`, `ts_local`, `ts_hms_local`, `time`)
  - Timestamp parsing supports epoch seconds/milliseconds, ISO datetime strings, and `HH:MM:SS` with date inferred from filename (`YYYY-MM-DD`) or upload date
  - Downloads include CSV formula-injection hardening (`=`, `+`, `-`, `@` prefixed with `'`)
- Upload/download hardening:
  - Same-origin checks required for upload/delete/download timeline endpoints
  - CSRF token required for upload/delete/download timeline endpoints
  - In-memory rate limiting for upload/delete/download endpoints
  - Strict upload ID validation and fixed server-side storage paths
  - Upload caps: `10 MiB` per file, `20` files max, `200 MiB` total, `250000` CSV rows max, `64` columns max, `4096` chars per cell max
- Once internet is available, NTP sync runs and writes corrected time back to RTC (`hwclock -w`).
- After boot is ready, power/activity LED is set to 1 Hz blink (`timer`, 500ms on / 500ms off), if LED sysfs control is available.

Note:
- Reboot/shutdown actions in the web UI currently have no authentication. Restrict access to trusted networks only.

Runtime check commands:
```bash
systemctl status serial-bridge
journalctl -u serial-bridge -f
ip a show wlan0
ls -l /home/pi/xxx.log /home/pi/xxx_*.log
ls -l /home/pi/timeline_uploads
sudo hwclock -r
```

Optional environment variables:
```bash
SERIAL_LOG_DIR=/home/pi
SERIAL_LOG_PREFIX=xxx
TIMELINE_UPLOAD_DIR=/home/pi/timeline_uploads
SERIAL_WEB_SECRET=<long-random-secret>
```

## RTC GPIO Wiring (Raspberry Pi Zero W)

Use I2C1 pins on the 40-pin header:

| RTC module pin | Raspberry Pi pin | BCM GPIO | Notes |
|---|---|---|---|
| `VCC` | `Pin 1` | 3V3 | Use 3.3V |
| `GND` | `Pin 6` | GND | Common ground |
| `SDA` | `Pin 3` | GPIO2 (`SDA1`) | I2C data |
| `SCL` | `Pin 5` | GPIO3 (`SCL1`) | I2C clock |

Minimal pin marker (header top):
```text
Pin 1 (3V3)   Pin 2 (5V)
Pin 3 (SDA1)  Pin 4 (5V)
Pin 5 (SCL1)  Pin 6 (GND)
```

Enable I2C + RTC overlay (example DS3231):
```bash
sudo raspi-config nonint do_i2c 0
echo 'dtoverlay=i2c-rtc,ds3231' | sudo tee -a /boot/firmware/config.txt
sudo reboot
```

Verify:
```bash
ls -l /dev/rtc*
sudo i2cdetect -y 1
sudo hwclock -r
```

## Quick install (on Raspberry Pi)

```bash
chmod +x install.sh
sudo ./install.sh
```

Installer behavior:
- First run: full install (apt packages, venv, pip deps, configs, services).
- Re-run/update: fast path, skips apt and pip when not needed.
- `pip install` runs only when `requirements.txt` hash changed (or venv missing).

Optional reboot after first install or low-level config changes:

```bash
sudo reboot
```