lutz/EMS
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Initial commit: EMS — Energie Management System

Browse Source 
Complete self-consumption optimisation system for 7 kWp PV installation:
- Prometheus collector (grid power, SOC, PV, per-phase, compressor)
- Pure decision engine with SOC gates, hysteresis, priority ordering
- Shelly Gen1/Gen2 actuator (SHA-256 Digest auth, PM power readback)
- Viessmann OAuth2 client for DHW temperature control
- PV forecast integration (forecast.solar)
- Wallbox mutual exclusion (VX3 4.6 kW AC output constraint)
- Car-not-charging detection via Shelly PM
- Compressor idle → early SG-Ready release
- Per-phase grid power for single-phase wallbox decisions
- Manual override detection and web UI with override buttons
- Full unit test coverage for decision engine
- systemd service, Makefile, complete documentation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Lutz FInsterle
2026-04-06 10:02:16 +02:00
commit 99613c52ae
19 changed files with 3924 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

405
README.md Normal file
View File

@@ -0,0 +1,405 @@
# EMS — Energie Management System
Self-consumption optimisation for a home PV installation with battery storage, heat pump, and EV wallboxes. Reads live data from Prometheus, makes switching decisions based on SOC thresholds and export surplus, and controls Shelly actuators via HTTP.
---
## Table of Contents
1. [System Overview](#system-overview)
2. [Hardware Setup](#hardware-setup)
3. [Architecture](#architecture)
4. [Decision Logic](#decision-logic)
5. [Configuration Reference](#configuration-reference)
6. [Deployment](#deployment)
7. [Web Interface](#web-interface)
8. [Prometheus Metrics](#prometheus-metrics)
9. [Branch Strategy](#branch-strategy)
10. [Development](#development)
---
## System Overview
```
PV panels (7 kWp)
│
▼
Trina VX3 4.6 hybrid inverter ──► Battery (8 kWh)
│
▼ AC output (max 4.6 kW)
Grid connection (PCC)
│
├──► House loads (heat pump, household)
├──► SG-Ready relay (Shelly Gen1) ──► Viessmann Vitocal 200-S
├──► Wallbox A 2kW (Shelly Plus 1 PM) ──► EV charging
└──► Wallbox B 4kW (Shelly Plus 1) ──► EV charging
```
The EMS runs on a Raspberry Pi on the local network. Every 2 minutes it:
1. Reads system state from Prometheus (grid power, battery SOC, PV production, phase currents, compressor power)
2. Reads actual relay states from Shelly devices (with live power from PM variants)
3. Fetches a daily PV forecast (forecast.solar)
4. Runs the decision engine → produces a list of switching actions
5. Executes actions via Shelly HTTP API and Viessmann cloud API
---
## Hardware Setup
| Component | Details |
|-----------|---------|
| PV | 7 kWp, Trina VX3 4.6 hybrid inverter |
| Battery | 8 kWh, managed autonomously by VX3 |
| Heat pump | Viessmann Vitocal 200-S AWB-E-AC |
| SG-Ready | Shelly 1 Gen1 @ 192.168.42.90 — activates heating buffer boost (+5°C) |
| Wallbox A | 2 kW single-phase, Shelly Plus 1 PM Gen2 @ 192.168.42.185 |
| Wallbox B | 4 kW 3-phase, Shelly Plus 1 Gen2 @ 192.168.42.51 |
| WW control | Viessmann API (OAuth2) — sets DHW target temperature |
| Data source | Prometheus @ 192.168.0.23:9090 (custom Viessmann exporter) |
| EMS host | Raspberry Pi, runs as systemd service |
### VX3 AC output constraint
The inverter's AC output is limited to **4.6 kW**. Only one wallbox may be active at a time:
- Wallbox A (2 kW) OR Wallbox B (4 kW) — never both simultaneously.
---
## Architecture
```
Prometheus (192.168.0.23:9090)
│ PromQL instant queries (every 2 min)
▼
internal/collector/ — SystemState{GridPowerW, BatterySOC, PVProductionW,
PhaseL1/2/3PowerW, CompressorPowerW, ...}
│
▼
internal/engine/ — Pure decision logic (no I/O, fully unit-tested)
│ Decide(state, now, wwBoostC) → []Action
▼
internal/actuator/ — Executes switching
├── Shelly Gen1: GET /relay/0?turn=on|off
├── Shelly Gen2: POST /rpc/Switch.Set (SHA-256 Digest auth)
│ + reads live apower from /rpc/Switch.GetStatus
└── Viessmann API: OAuth2, sets DHW temperature via IoT API
internal/forecast/ — Daily PV forecast from forecast.solar (cached)
internal/viessmann/ — OAuth2 token lifecycle + DHW temperature API
internal/status/ — Web status page (HTML) + manual override handler
internal/metrics/ — Prometheus exporter on :9099
internal/config/ — YAML config loader
```
### Key design decisions
- **Data reads** come from Prometheus (already scraped every 2 min by the Viessmann exporter). No direct Modbus/RS485.
- **Data writes** (Shelly switching, Viessmann DHW temp) are done directly by the EMS.
- **Engine is pure**: `Decide()` has no side effects — easy to unit test without mocks.
- **Shelly state read-back every cycle**: detects manual switches (Shelly app, physical button) and applies an override lockout.
- **Per-device resilience**: if one Shelly is unreachable, the others still work.
- **Wallbox PM integration**: Shelly Plus 1 PM reports live power — used to detect when a car stops charging and release the wallbox slot.
- **Compressor idle detection**: if SG-Ready is active but the heat pump compressor drops to near-zero power, SG-Ready is released early (before its minimum runtime) since the boost is no longer being used.
---
## Decision Logic
### SOC Gates
Battery SOC determines which consumers are allowed at all:
| SOC range | Allowed consumers |
|-----------|-------------------|
| < 50% | None (all blocked) |
| 50–70% | SG-Ready, WW boost |
| 70–90% | SG-Ready, WW boost, Wallbox A |
| ≥ 90% | All consumers |
### Turn-on sequence (priority order)
Each cycle, if grid power is negative (exporting):
1. **SG-Ready** (heating season Oct–Apr only): export > 500 W for 4 min continuously
2. **WW boost** (12:30–18:00 time window + PV forecast required): export > 500 W for 4 min
3. **Wallbox A** (2 kW, single-phase): export > 1500 W on best phase for 4 min. Blocked if Wallbox B is active.
4. **Wallbox B** (4 kW, 3-phase): total export > 3800 W for 4 min. Blocked if Wallbox A is active.
### Shutdown sequence (reverse priority)
If grid import > 200 W for 6 min continuously:
Wallbox B → Wallbox A → WW → SG-Ready (one consumer per cycle, respecting minimum runtimes)
Minimum runtimes: Wallbox 15 min, SG-Ready 30 min.
### Emergency brake
If SOC drops below its allowed threshold, the consumer is shut off immediately (min-runtime ignored).
### WW temperature setpoint
| PV forecast | DHW target |
|-------------|-----------|
| < 15 kWh | 48°C (base) |
| 15–25 kWh | 51°C (+3°C) |
| ≥ 25 kWh | 53°C (+5°C) |
The heat pump's own hysteresis fires the compressor when DHW drops 5°C below target.
### Early release conditions
- **SG-Ready**: released early if `CompressorPowerW < 50 W` for 3 consecutive cycles (~6 min). The heat pump has finished its boost cycle.
- **Wallbox A/B**: released early if Shelly PM reads `< 50 W` for 3 consecutive cycles. Car is full or unplugged.
### Manual override detection
Every cycle the EMS reads back actual Shelly relay states. If the hardware state differs from the engine's expected state, a manual override is detected. The consumer is locked from EMS control for 1 hour (configurable). Shown on the status page.
---
## Configuration Reference
All parameters are in `configs/ems-config.yaml` (or `/etc/ems/ems-config.yaml` when installed).
```yaml
prometheus:
url: "http://<host>:9090"
metrics:
grid_power_exchange: "pcc_transfer_power_exchange_value" # +import/-export (W)
battery_soc: "ess_stateOfCharge_value" # 0–100%
pv_production: "photovoltaic_production_current_value" # kW (multiplied by 1000)
battery_power: "ess_power_value" # W
compressor_power: "heating_compressors_0_power_value" # W
ambient_temp: "heating_sensors_temperature_outside_value" # °C
phase_l1_power: "pcc_ac_active_power_phaseOne" # W
phase_l2_power: "pcc_ac_active_power_phaseTwo" # W
phase_l3_power: "pcc_ac_active_power_phaseThree" # W
shelly:
sg_ready:
ip: "<ip>"
gen: 1 # Gen1: relay endpoint
wallbox_a:
ip: "<ip>"
gen: 2 # Gen2: rpc endpoint
power_w: 2000 # rated power (informational)
password: "" # Shelly admin password (SHA-256 Digest auth)
wallbox_b:
ip: "<ip>"
gen: 2
power_w: 4000
password: ""
viessmann:
token_file: "/etc/ems/viessmann-token.json"
client_id: "<oauth2 client id>"
installation_id: "<id>"
gateway_serial: "<serial>"
device_id: "0"
soc_thresholds:
block_all: 50 # below this: nothing runs
sg_ready_only: 70 # 50–70%: SG-Ready + WW only
plus_wallbox_a: 90 # 70–90%: + Wallbox A
all_consumers: 90 # ≥90%: all consumers
hysteresis:
export_on_duration: "4m" # export must persist before switching on
import_off_duration: "6m" # import must persist before switching off
min_runtime_wallbox: "15m" # wallbox minimum on-time before shutdown
min_runtime_sg_ready: "30m" # SG-Ready minimum on-time before shutdown
thresholds:
sg_ready_export_w: -500 # export needed for SG-Ready (negative = export)
ww_export_w: -500 # export needed for WW boost
wallbox_a_export_w: -1800 # total export fallback for Wallbox A
wallbox_a_phase_export_w: -1500 # per-phase export needed for single-phase Wallbox A
wallbox_b_export_w: -3800 # total export needed for Wallbox B
import_off_w: 200 # import level that starts shutdown timer
consumers:
compressor_idle_w: 50 # compressor below this = heat pump idle (W)
wallbox_min_charge_w: 50 # wallbox below this = car not charging (W)
idle_cycles: 3 # consecutive idle cycles before early release
strategic:
forecast_high_kwh: 25 # high-forecast day threshold
forecast_mid_kwh: 15 # medium-forecast day threshold
ww_base_c: 48 # normal DHW setpoint (°C)
ww_boost_high_c: 5 # additional °C on high-forecast days
ww_boost_mid_c: 3 # additional °C on medium-forecast days
ww_window_start: "12:30" # WW boost only within this window
ww_window_end: "18:00"
schedule_on: "08:00"
schedule_off: "20:00"
season:
heating_start_month: 10 # October
heating_end_month: 4 # April (wrap-around handled)
forecast:
enabled: true
lat: <latitude>
lon: <longitude>
declination: <panel tilt degrees>
azimuth: <degrees from south: 0=south, 90=west, -90=east>
kwp: <installed peak power kWp>
ems:
poll_interval: "2m"
listen_addr: ":9099"
log_level: "info" # debug | info | warn | error
state_file: "/run/ems/heartbeat"
recovery_timeout: "1h" # ignore Shelly state if EMS was down longer than this
override_timeout: "1h" # how long a manual Shelly change is respected
```
---
## Deployment
### Prerequisites
- Go 1.23+
- Prometheus instance scraping the Viessmann exporter
- Shelly devices on the local network
- Viessmann API credentials (OAuth2 refresh token in `/etc/ems/viessmann-token.json`)
### Install
```bash
# Build and install as systemd service
sudo make install
sudo systemctl enable --now ems
# Check status
sudo systemctl status ems
sudo journalctl -u ems -f
```
### Makefile targets
```bash
make build # Build binary → ./ems
make test # Run unit tests
make dry-run # Run without executing actions (log only)
make run # Run with live switching (foreground)
make install # Build + install binary, config, systemd unit
```
### Config file locations (when installed)
| File | Purpose |
|------|---------|
| `/etc/ems/ems-config.yaml` | Main configuration |
| `/etc/ems/viessmann-token.json` | OAuth2 refresh token |
| `/run/ems/heartbeat` | Written each cycle — used for state recovery on restart |
### Viessmann token file format
```json
{
"access_token": "...",
"refresh_token": "...",
"expires_in": 3600,
"validToTimeDate": "2024-01-01T12:00:00Z"
}
```
The EMS automatically refreshes the token when it expires. The updated token is written back to the file.
---
## Web Interface
Available at `http://<pi-ip>:9099/`
Shows:
- Current grid power, battery SOC, PV production
- Per-phase grid power (L1/L2/L3)
- Each consumer: state, manual override status, Shelly PM live power
- Today's PV forecast
- Manual override buttons for Shelly consumers (SG-Ready, Wallbox A, Wallbox B)
Manual overrides via the web UI switch the hardware directly. The EMS detects the change next cycle via state read-back and applies the override lockout automatically.
---
## Prometheus Metrics
The EMS exports its own metrics on `:9099/metrics`:
| Metric | Description |
|--------|-------------|
| `ems_grid_power_watts` | Current grid power (positive=import, negative=export) |
| `ems_battery_soc_percent` | Battery SOC (0–100%) |
| `ems_pv_production_watts` | Current PV production |
| `ems_grid_import_kwh_total` | Cumulative grid import estimate |
| `ems_grid_export_kwh_total` | Cumulative grid export estimate |
| `ems_consumer_active{consumer}` | 1 if consumer is active, 0 otherwise |
| `ems_switch_cycles_total{consumer,action}` | Total switching events |
| `ems_decision_duration_seconds` | Engine decision latency |
---
## Branch Strategy
This repository uses separate branches per installation:
| Branch | Purpose |
|--------|---------|
| `main` | Core code + generic config template |
| `house/lutz` | Lutz's installation (7 kWp, Vitocal 200-S, 2+4 kW wallboxes) |
| `house/son` | Son's installation (hardware TBD) |
**Workflow:**
- Bug fixes and new features are developed on `main`
- Each house branch is kept up to date by merging from `main`: `git merge main`
- House-specific config changes stay on the house branch and are never merged back to `main`
- The config in `main` contains placeholder values — it is a template, not a working config
---
## Development
### Running tests
```bash
make test
# or
go test ./...
```
The engine package has full unit test coverage. Tests run without any hardware or network — the engine is a pure function and all dependencies are injected via structs.
### Adding a new consumer
1. Add a `ConsumerXxx Consumer = iota` constant in `internal/engine/engine.go`
2. Add its `String()` case
3. Add a `ShellyDevice` (or other actuator) entry to `internal/config/config.go`
4. Add the switch case in `internal/actuator/actuator.go` `executeOne()`
5. Add it to the priority order in `Decide()` and `shutdownLastConsumer()`
6. Add it to `ReadAllStates()` in the actuator
7. Update `configs/ems-config.yaml`
### Project structure
```
.
├── main.go # Entry point, control loop, HTTP server
├── configs/
│ └── ems-config.yaml # Configuration (template on main, real values on house branches)
├── internal/
│ ├── actuator/ # Shelly HTTP control + Viessmann DHW API
│ ├── collector/ # Prometheus PromQL queries → SystemState
│ ├── config/ # YAML config types and loader
│ ├── engine/ # Pure decision logic (fully unit-tested)
│ ├── forecast/ # forecast.solar daily PV forecast (cached)
│ ├── metrics/ # Prometheus exporter
│ ├── status/ # Web status page + override handler
│ └── viessmann/ # OAuth2 token management + IoT API client
└── systemd/
└── ems.service # systemd unit file
```