Hands On System Design - Distributed Systems Implementation

Hands On System Design - Distributed Systems Implementation

Day 179: Device State Tracking and Management from IoT Logs

Week 26: IoT and Edge Log Processing Series: 254-Day Hands-On System Design with Distributed Log Processing System Implementation

Aug 27, 2026
∙ Paid

What We Build Today

  • A device state machine that reconstructs current device status from raw logs

  • A state store that persists device state across restarts without a heavy database

  • A REST and WebSocket API for querying live device states

  • A browser dashboard showing real-time device health across a fleet

This fits into Week 26’s IoT arc: Day 177 handled disconnected edge collection, Day 178 optimized bandwidth-constrained transport, and today we build the intelligence layer — knowing what state every device is in at any moment.


Why Device State Matters

Imagine managing 50,000 temperature sensors in cold-chain logistics. Each sensor streams logs every 30 seconds. A single log line says {"device":"fridge-42","temp":8.2,"battery":12}. But the question that matters to operations is not “what did fridge-42 just report?” — it is “what is the current state of fridge-42, and has anything changed since last check?”

That distinction — from raw event to current state — is what separates a log sink from an operational system.

Cloudflare’s IoT gateway team faces exactly this challenge managing millions of edge nodes. Their internal state service reconstructs device health from streamed telemetry, never relying on devices to explicitly announce state changes. State is inferred from the log stream itself.


Core Concept: State Machines from Log Streams

A device does not exist in a vacuum. At any moment it is in exactly one state: online, degraded, offline, alert, or unknown. Transitions between these states are driven by log events.

The core insight is that logs are evidence of state, not the state itself. Your system observes evidence and maintains a derived state model. This is called event-sourced state reconstruction.

Log Event  →  State Transition Rules  →  Current State

State transitions work as follows:

  • A telemetry log within the expected interval keeps a device online

  • A telemetry log with a battery value below threshold moves it to degraded

  • No telemetry for more than 2x the expected interval moves it to offline

  • A log with an error field or threshold breach moves it to alert

This mirrors how Apache Flink processes IoT streams at Alibaba — the stream processor holds a rolling state view for every device key, updated on each incoming record.


Architecture

The system has four layers:

Log Ingestion Layer — An HTTP endpoint accepts batches of device logs. Each log carries a device ID, timestamp, and payload. The ingestion layer validates and forwards to the state processor.

State Processor — The central brain. For each incoming log, it applies transition rules, updates the state store, and emits a change event if the state actually changed. It also runs a background timer that checks for devices that have gone silent.

State Store — A lightweight SQLite-backed store with an in-memory cache. Stores the current state, last seen timestamp, last log payload, and a transition history ring buffer (last 20 transitions per device).

Query API + WebSocket Hub — A FastAPI server exposing /devices, /devices/{id}, and a WebSocket endpoint /ws/state-changes that pushes diffs to connected dashboards.


User's avatar

Continue reading this post for free, courtesy of System Design Course.

Or purchase a paid subscription.
© 2026 Systemdr, Inc. · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture