EMS/README.md

EMS — Energie Management System

Self-consumption optimisation for a home PV installation with battery storage, heat pump, and EV wallboxes. Reads live data from Prometheus, makes switching decisions based on SOC thresholds and export surplus, and controls Shelly actuators via HTTP.

---

Table of Contents

1. System Overview
2. Hardware Setup
3. Architecture
4. Decision Logic
5. Configuration Reference
6. Deployment
7. Web Interface
8. Prometheus Metrics
9. Branch Strategy
10. Development

---

System Overview

PV panels (7 kWp)
    │
    ▼
Trina VX3 4.6 hybrid inverter ──► Battery (8 kWh)
    │
    ▼ AC output (max 4.6 kW)
Grid connection (PCC)
    │
    ├──► House loads (heat pump, household)
    ├──► SG-Ready relay (Shelly Gen1) ──► Viessmann Vitocal 200-S
    ├──► Wallbox A 2kW (Shelly Plus 1 PM) ──► EV charging
    └──► Wallbox B 4kW (Shelly Plus 1)   ──► EV charging

The EMS runs on a Raspberry Pi on the local network. Every 2 minutes it:

1. Reads system state from Prometheus (grid power, battery SOC, PV production, phase currents, compressor power)
2. Reads actual relay states from Shelly devices (with live power from PM variants)
3. Fetches a daily PV forecast (forecast.solar)
4. Runs the decision engine → produces a list of switching actions
5. Executes actions via Shelly HTTP API and Viessmann cloud API

---

Hardware Setup

Component	Details
PV	7 kWp, Trina VX3 4.6 hybrid inverter
Battery	8 kWh, managed autonomously by VX3
Heat pump	Viessmann Vitocal 200-S AWB-E-AC
SG-Ready	Shelly 1 Gen1 @ 192.168.42.90 — activates heating buffer boost (+5°C)
Wallbox A	2 kW single-phase, Shelly Plus 1 PM Gen2 @ 192.168.42.185
Wallbox B	4 kW 3-phase, Shelly Plus 1 Gen2 @ 192.168.42.51
WW control	Viessmann API (OAuth2) — sets DHW target temperature
Data source	Prometheus @ 192.168.0.23:9090 (custom Viessmann exporter)
EMS host	Raspberry Pi, runs as systemd service

VX3 AC output constraint

The inverter's AC output is limited to 4.6 kW. Only one wallbox may be active at a time:

• Wallbox A (2 kW) OR Wallbox B (4 kW) — never both simultaneously.

---

Architecture

Prometheus (192.168.0.23:9090)
    │ PromQL instant queries (every 2 min)
    ▼
internal/collector/   — SystemState{GridPowerW, BatterySOC, PVProductionW,
                                     PhaseL1/2/3PowerW, CompressorPowerW, ...}
    │
    ▼
internal/engine/      — Pure decision logic (no I/O, fully unit-tested)
    │  Decide(state, now, wwBoostC) → []Action
    ▼
internal/actuator/    — Executes switching
    ├── Shelly Gen1: GET /relay/0?turn=on|off
    ├── Shelly Gen2: POST /rpc/Switch.Set (SHA-256 Digest auth)
    │   + reads live apower from /rpc/Switch.GetStatus
    └── Viessmann API: OAuth2, sets DHW temperature via IoT API

internal/forecast/    — Daily PV forecast from forecast.solar (cached)
internal/viessmann/   — OAuth2 token lifecycle + DHW temperature API
internal/status/      — Web status page (HTML) + manual override handler
internal/metrics/     — Prometheus exporter on :9099
internal/config/      — YAML config loader

Key design decisions

• Data reads come from Prometheus (already scraped every 2 min by the Viessmann exporter). No direct Modbus/RS485.
• Data writes (Shelly switching, Viessmann DHW temp) are done directly by the EMS.
• Engine is pure: Decide() has no side effects — easy to unit test without mocks.
• Shelly state read-back every cycle: detects manual switches (Shelly app, physical button) and applies an override lockout.
• Per-device resilience: if one Shelly is unreachable, the others still work.
• Wallbox PM integration: Shelly Plus 1 PM reports live power — used to detect when a car stops charging and release the wallbox slot.
• Compressor idle detection: if SG-Ready is active but the heat pump compressor drops to near-zero power, SG-Ready is released early (before its minimum runtime) since the boost is no longer being used.

---

Decision Logic

SOC Gates

Battery SOC determines which consumers are allowed at all:

SOC range	Allowed consumers
< 50%	None (all blocked)
50–70%	SG-Ready, WW boost
70–90%	SG-Ready, WW boost, Wallbox A
≥ 90%	All consumers

Turn-on sequence (priority order)

Each cycle, if grid power is negative (exporting):

1. SG-Ready (heating season Oct–Apr only): export > 500 W for 4 min continuously
2. WW boost (12:30–18:00 time window + PV forecast required): export > 500 W for 4 min
3. Wallbox A (2 kW, single-phase): export > 1500 W on best phase for 4 min. Blocked if Wallbox B is active.
4. Wallbox B (4 kW, 3-phase): total export > 3800 W for 4 min. Blocked if Wallbox A is active.

Shutdown sequence (reverse priority)

If grid import > 200 W for 6 min continuously:

Wallbox B → Wallbox A → WW → SG-Ready (one consumer per cycle, respecting minimum runtimes)

Minimum runtimes: Wallbox 15 min, SG-Ready 30 min.

Emergency brake

If SOC drops below its allowed threshold, the consumer is shut off immediately (min-runtime ignored).

WW temperature setpoint

PV forecast	DHW target
< 15 kWh	48°C (base)
15–25 kWh	51°C (+3°C)
≥ 25 kWh	53°C (+5°C)

The heat pump's own hysteresis fires the compressor when DHW drops 5°C below target.

PV forecast cadence

The forecast is fetched from forecast.solar up to 3 times per day, within the configured fetch_window_start/fetch_window_end and spaced by fetch_interval. Outside the window the last result is reused. The free tier allows ~12 requests per day; 3 fetches per day leaves ample headroom.

Early release conditions

• SG-Ready: released early if CompressorPowerW < 50 W for 3 consecutive cycles (~6 min). The heat pump has finished its boost cycle.
• Wallbox A/B: released early if Shelly PM reads < 50 W for 3 consecutive cycles. Car is full or unplugged.

Charging advice

The status page shows a "Plug in car" advisory when:

• No wallbox is currently active
• The PV forecast predicts a surplus window (net PV output > min_surplus_w) later today
• The window has not yet started

The advisory shows the recommended plug-in time (window start − 30 min) so the car is ready when surplus charging begins.

The surplus window is computed from the hourly forecast (watts array) minus base_load_w. The first contiguous block of hours where pv_w − base_load_w ≥ min_surplus_w becomes the charging window.

Manual override detection

Every cycle the EMS reads back actual Shelly relay states. If the hardware state differs from the engine's expected state, a manual override is detected. The consumer is locked from EMS control for a configurable duration (default 1 h). Shown on the status page.

Override hard-stop conditions

Active overrides are cancelled immediately (without waiting for the duration to expire) if:

• SOC emergency brake — battery SOC drops below the consumer's allowed SOC threshold
• Import hard-stop — grid import exceeds override_max_import_w (e.g. 800 W) — prevents runaway grid consumption when a car is force-started at night or on a cloudy day

---

Configuration Reference

All parameters are in configs/ems-config.yaml (or /etc/ems/ems-config.yaml when installed).

prometheus:
  url: "http://<host>:9090"
  metrics:
    grid_power_exchange: "pcc_transfer_power_exchange_value"  # +import/-export (W)
    battery_soc: "ess_stateOfCharge_value"                    # 0–100%
    pv_production: "photovoltaic_production_current_value"    # kW (multiplied by 1000)
    battery_power: "ess_power_value"                          # W
    compressor_power: "heating_compressors_0_power_value"     # W
    ambient_temp: "heating_sensors_temperature_outside_value" # °C
    phase_l1_power: "pcc_ac_active_power_phaseOne"            # W
    phase_l2_power: "pcc_ac_active_power_phaseTwo"            # W
    phase_l3_power: "pcc_ac_active_power_phaseThree"          # W

shelly:
  sg_ready:
    ip: "<ip>"
    gen: 1                      # Gen1: relay endpoint
  wallbox_a:
    ip: "<ip>"
    gen: 2                      # Gen2: rpc endpoint
    power_w: 2000               # rated power (informational)
    password: ""                # Shelly admin password (SHA-256 Digest auth)
  wallbox_b:
    ip: "<ip>"
    gen: 2
    power_w: 4000
    password: ""

viessmann:
  token_file: "/etc/ems/viessmann-token.json"
  client_id: "<oauth2 client id>"
  installation_id: "<id>"
  gateway_serial: "<serial>"
  device_id: "0"

soc_thresholds:
  block_all: 50                 # below this: nothing runs
  sg_ready_only: 70             # 50–70%: SG-Ready + WW only
  plus_wallbox_a: 90            # 70–90%: + Wallbox A
  all_consumers: 90             # ≥90%: all consumers

hysteresis:
  export_on_duration: "4m"      # export must persist before switching on
  import_off_duration: "6m"     # import must persist before switching off
  min_runtime_wallbox: "15m"    # wallbox minimum on-time before shutdown
  min_runtime_sg_ready: "30m"   # SG-Ready minimum on-time before shutdown

thresholds:
  sg_ready_export_w: -500       # export needed for SG-Ready (negative = export)
  ww_export_w: -500             # export needed for WW boost
  wallbox_a_export_w: -1800     # total export fallback for Wallbox A
  wallbox_a_phase_export_w: -1500  # per-phase export needed for single-phase Wallbox A
  wallbox_b_export_w: -3800     # total export needed for Wallbox B
  import_off_w: 200             # import level that starts shutdown timer

consumers:
  compressor_idle_w: 50         # compressor below this = heat pump idle (W)
  wallbox_min_charge_w: 50      # wallbox below this = car not charging (W)
  idle_cycles: 3                # consecutive idle cycles before early release

strategic:
  forecast_high_kwh: 25         # high-forecast day threshold
  forecast_mid_kwh: 15          # medium-forecast day threshold
  ww_base_c: 48                 # normal DHW setpoint (°C)
  ww_boost_high_c: 5            # additional °C on high-forecast days
  ww_boost_mid_c: 3             # additional °C on medium-forecast days
  ww_window_start: "12:30"      # WW boost only within this window
  ww_window_end: "18:00"
  schedule_on: "08:00"
  schedule_off: "20:00"

season:
  heating_start_month: 10       # October
  heating_end_month: 4          # April (wrap-around handled)

forecast:
  enabled: true
  lat: <latitude>
  lon: <longitude>
  declination: <panel tilt degrees>
  azimuth: <degrees from south: 0=south, 90=west, -90=east>
  kwp: <installed peak power kWp>
  fetch_interval: "4h"          # re-fetch during the day (free tier: max ~12 req/day)
  fetch_window_start: "07:00"   # only fetch within this window
  fetch_window_end: "19:00"
  base_load_w: 450              # steady-state house consumption for surplus estimate (W)
  min_surplus_w: 1800           # min net surplus to consider wallbox charging viable (W)

ems:
  poll_interval: "2m"
  listen_addr: ":9099"
  log_level: "info"             # debug | info | warn | error
  state_file: "/run/ems/heartbeat"
  recovery_timeout: "1h"        # ignore Shelly state if EMS was down longer than this
  override_timeout: "1h"        # how long a manual Shelly change is respected
  override_max_import_w: 800    # cancel active override if grid import exceeds this (0 = disabled)

---

Deployment

Prerequisites

• Go 1.23+
• Prometheus instance scraping the Viessmann exporter
• Shelly devices on the local network
• Viessmann API credentials (OAuth2 refresh token in /etc/ems/viessmann-token.json)

Install

# Build and install as systemd service
sudo make install
sudo systemctl enable --now ems

# Check status
sudo systemctl status ems
sudo journalctl -u ems -f

Makefile targets

make build      # Build binary → ./ems
make test       # Run unit tests
make dry-run    # Run without executing actions (log only)
make run        # Run with live switching (foreground)
make install    # Build + install binary, config (first time only), systemd unit

Note: make install only copies configs/ems-config.yaml to /etc/ems/ems-config.yaml if that file does not already exist. Subsequent installs will not overwrite your passwords or local settings.

Config file locations (when installed)

File	Purpose
/etc/ems/ems-config.yaml	Main configuration
/etc/ems/viessmann-token.json	OAuth2 refresh token
/run/ems/heartbeat	Written each cycle — used for state recovery on restart

Viessmann token file format

{
  "access_token": "...",
  "refresh_token": "...",
  "expires_in": 3600,
  "validToTimeDate": "2024-01-01T12:00:00Z"
}

The EMS automatically refreshes the token when it expires. The updated token is written back to the file.

---

Web Interface

Available at http://<pi-ip>:9099/

Shows:

• Current grid power, battery SOC, PV production
• Per-phase grid power (L1/L2/L3) — helps identify which phase has the most export
• Heat pump compressor power — shows whether the heat pump is actively running
• Each consumer: state, manual override status, live power (W) from Shelly PM variants
• Today's PV forecast (total kWh, charging surplus window if predicted)
• Charging advice card — "Plug in car by HH:MM" when surplus charging is expected later today
• Manual override controls for each consumer (SG-Ready, Wallbox A, Wallbox B)

Manual override controls

Each consumer has Ein (on) and Aus (off) buttons. A duration selector lets you choose how long the override lasts: 30 min / 1 h / 2 h / 4 h.

Overrides applied via the web UI take effect immediately (the relay switches at once) and the lockout is active before the next EMS cycle. The EMS will not fight the override until the duration expires — unless a hard-stop condition is triggered (SOC emergency brake or import threshold exceeded).

Overrides applied via the Shelly app or physical button are detected on the next cycle (within 2 min) and automatically locked for the configured override_timeout.

---

Prometheus Metrics

The EMS exports its own metrics on :9099/metrics:

Metric	Description
ems_grid_power_watts	Current grid power (positive=import, negative=export)
ems_battery_soc_percent	Battery SOC (0–100%)
ems_pv_production_watts	Current PV production
ems_grid_import_kwh_total	Cumulative grid import estimate
ems_grid_export_kwh_total	Cumulative grid export estimate
ems_consumer_active{consumer}	1 if consumer is active, 0 otherwise
ems_switch_cycles_total{consumer,action}	Total switching events
ems_decision_duration_seconds	Engine decision latency

---

Branch Strategy

This repository uses separate branches per installation:

Branch	Purpose
main	Core code + generic config template
house/lutz	Lutz's installation (7 kWp, Vitocal 200-S, 2+4 kW wallboxes)
house/son	Son's installation (hardware TBD)

Workflow:

• Bug fixes and new features are developed on main
• Each house branch is kept up to date by merging from main: git merge main
• House-specific config changes stay on the house branch and are never merged back to main
• The config in main contains placeholder values — it is a template, not a working config

---

Development

Running tests

make test
# or
go test ./...

The engine package has full unit test coverage. Tests run without any hardware or network — the engine is a pure function and all dependencies are injected via structs.

Adding a new consumer

1. Add a ConsumerXxx Consumer = iota constant in internal/engine/engine.go
2. Add its String() case
3. Add a ShellyDevice (or other actuator) entry to internal/config/config.go
4. Add the switch case in internal/actuator/actuator.go executeOne()
5. Add it to the priority order in Decide() and shutdownLastConsumer()
6. Add it to ReadAllStates() in the actuator
7. Update configs/ems-config.yaml

Project structure

.
├── main.go                     # Entry point, control loop, HTTP server
├── configs/
│   └── ems-config.yaml         # Configuration (template on main, real values on house branches)
├── internal/
│   ├── actuator/               # Shelly HTTP control + Viessmann DHW API
│   ├── collector/              # Prometheus PromQL queries → SystemState
│   ├── config/                 # YAML config types and loader
│   ├── engine/                 # Pure decision logic (fully unit-tested)
│   ├── forecast/               # forecast.solar daily PV forecast (cached)
│   ├── metrics/                # Prometheus exporter
│   ├── status/                 # Web status page + override handler
│   └── viessmann/              # OAuth2 token management + IoT API client
└── systemd/
    └── ems.service             # systemd unit file