Day 175: Customer Experience Monitoring — Turning Logs into User Stories
Week 25: Business Analytics Use Cases | From the “254-Day Hands-On System Design” Series
Yesterday, we built an A/B test analysis framework that compares experiment cohorts statistically. Today we shift perspective entirely — instead of comparing groups, we watch individual user journeys unfold through log data and surface the friction points that silently drive customers away.
What We’re Building Today
Parse raw application logs into structured user sessions
Compute CX metrics: page load time, error encounter rate, session abandonment, funnel completion
Build a Python 3.11 FastAPI backend that aggregates these metrics in real-time
React dashboard showing live CX health at a glance
Integrate cleanly into the log pipeline built across prior weeks
Why This Problem Is Hard
When Shopify’s checkout page added 300ms of latency during Black Friday 2021, it cost merchants millions before anyone noticed — because no single error fired. The signal was buried in the aggregate pattern of user sessions quietly abandoning mid-funnel. Standard error-rate monitoring saw nothing. Customer experience monitoring would have caught it in minutes.
Logs tell you what happened. CX monitoring tells you what the user experienced.
Core Concepts
Session Stitching — Log lines arrive with user IDs and timestamps but no session boundaries. We detect sessions by grouping events per user with a 30-minute inactivity timeout. This mirrors how Google Analytics and Amplitude define sessions.
Funnel State Machine — Every product has a conversion funnel (e.g., landing → product_view → cart → checkout → purchase). We model this as a state machine where each log event advances or exits a session’s state. Abandoned sessions are those that entered but never exited through purchase.
Percentile Latency, Not Averages — Average page load time hides the 5% of users on slow connections who see 8-second loads. We compute p50, p90, p95, p99 — the same metrics Stripe and Cloudflare publish in their status pages.
Error Encounter Rate — The fraction of sessions that hit at least one 5xx or client-reported JS error. A healthy system keeps this under 0.5%. Amplitude’s engineering team found that users who encounter even one error have 3× the churn rate.
Architecture
The system has three layers:
Ingestion Layer — A LogIngestor reads structured JSON logs from your existing pipeline (the Kafka/Redis stream from Week 5). Each log line carries: user_id, session_id, event_type, page, latency_ms, status_code, timestamp.
Computation Layer — A SessionAggregator maintains in-memory session windows per user. When a session closes (timeout or purchase), it emits a SessionSummary object. A MetricsComputer consumes summaries and updates rolling percentile buckets using a T-Digest algorithm — O(1) memory regardless of data volume.
Serving Layer — FastAPI exposes /metrics/cx returning the current CX health snapshot. React polls every 10 seconds and renders a dashboard with trend sparklines.
