DD3-Lora-Bridge-Raspi-Debugger/README.md

DD3 LoRa Bridge Raspi Debugger

Headless Raspberry Pi Zero W project:

• WiFi client or fallback AP (serial / serialserial)
• Web portal (http://192.168.4.1/ in AP mode)
• ESP32 USB serial bridge with live SSE stream and daily log files (/home/pi/xxx_YYYY-MM-DD_HH-MM-SS.log)
• 6-hour timeline page with CSV upload + merged timeline downloads
• Stable symlink to active log (/home/pi/xxx.log)
• RTC boot restore + NTP sync + RTC write-back
• Autostart via systemd

Current State

Current implementation status:

• Boot flow restores system time from RTC (hwclock -s) before starting services.
• RTC device auto-detection checks: configured device, /dev/rtc, /dev/rtc0, /dev/rtc1.
• If WLAN is disconnected for ~60s, AP fallback starts on wlan0:
  • SSID: serial
  • Password: serialserial
  • AP IP: 192.168.4.1/24
  • DHCP range: 192.168.4.10 to 192.168.4.200
• Web portal is available on port 80:
  • / WiFi scan + connect UI + system actions (reboot/shutdown)
  • /serial live serial console (SSE via /events/serial)
  • /timeline last-6-hours split timeline (serial + uploaded CSV data)
  • /api/status polling is reduced/throttled (15s) for Pi Zero W performance
• ESP32 serial bridge:
  • Auto-detects /dev/ttyUSB*, /dev/ttyACM*, /dev/serial/by-id/*
  • Reconnects automatically on unplug/replug
  • Daily log rollover at midnight with datetime filename
  • Each line written to file is timestamped with full local ISO datetime (including UTC offset)
  • SSE payload includes line, ts_iso, ts_hms, source
  • No log file is created while no serial device is connected
• Timeline CSV upload + merge:
  • Upload endpoint: POST /api/timeline/uploads (multipart field file)
  • Timeline data endpoint: GET /api/timeline?hours=6&upload_id=<id>
  • Download endpoint: GET /api/timeline/download?kind=serial|merged&hours=6&upload_id=<id>&csrf_token=<token>
  • Upload persistence: /home/pi/timeline_uploads/<id>.csv + sidecar metadata /home/pi/timeline_uploads/<id>.json
  • CSV parsing supports auto-detected timestamp columns (ts_utc, timestamp, ts, unix, epoch, datetime, ts_local, ts_hms_local, time)
  • Timestamp parsing supports epoch seconds/milliseconds, ISO datetime strings, and HH:MM:SS with date inferred from filename (YYYY-MM-DD) or upload date
  • Downloads include CSV formula-injection hardening (=, +, -, @ prefixed with ')
• Upload/download hardening:
  • Same-origin checks required for upload/delete/download timeline endpoints
  • CSRF token required for upload/delete/download timeline endpoints
  • In-memory rate limiting for upload/delete/download endpoints
  • Strict upload ID validation and fixed server-side storage paths
  • Upload caps: 10 MiB per file, 20 files max, 200 MiB total, 250000 CSV rows max, 64 columns max, 4096 chars per cell max
• Once internet is available, NTP sync runs and writes corrected time back to RTC (hwclock -w).
• After boot is ready, power/activity LED is set to 1 Hz blink (timer, 500ms on / 500ms off), if LED sysfs control is available.

Note:

• Reboot/shutdown actions in the web UI currently have no authentication. Restrict access to trusted networks only.

Runtime check commands:

systemctl status serial-bridge
journalctl -u serial-bridge -f
ip a show wlan0
ls -l /home/pi/xxx.log /home/pi/xxx_*.log
ls -l /home/pi/timeline_uploads
sudo hwclock -r

Optional environment variables:

SERIAL_LOG_DIR=/home/pi
SERIAL_LOG_PREFIX=xxx
TIMELINE_UPLOAD_DIR=/home/pi/timeline_uploads
SERIAL_WEB_SECRET=<long-random-secret>

RTC GPIO Wiring (Raspberry Pi Zero W)

Use I2C1 pins on the 40-pin header:

RTC module pin	Raspberry Pi pin	BCM GPIO	Notes
VCC	Pin 1	3V3	Use 3.3V
GND	Pin 6	GND	Common ground
SDA	Pin 3	GPIO2 (SDA1)	I2C data
SCL	Pin 5	GPIO3 (SCL1)	I2C clock

Minimal pin marker (header top):

Pin 1 (3V3)   Pin 2 (5V)
Pin 3 (SDA1)  Pin 4 (5V)
Pin 5 (SCL1)  Pin 6 (GND)

Enable I2C + RTC overlay (example DS3231):

sudo raspi-config nonint do_i2c 0
echo 'dtoverlay=i2c-rtc,ds3231' | sudo tee -a /boot/firmware/config.txt
sudo reboot

Verify:

ls -l /dev/rtc*
sudo i2cdetect -y 1
sudo hwclock -r

Quick install (on Raspberry Pi)

chmod +x install.sh
sudo ./install.sh

Installer behavior:

• First run: full install (apt packages, venv, pip deps, configs, services).
• Re-run/update: fast path, skips apt and pip when not needed.
• pip install runs only when requirements.txt hash changed (or venv missing).

Optional reboot after first install or low-level config changes:

sudo reboot