Start Here — How to Use SDCourse
Start Here: Your Map to LogStream
If you just subscribed (or you’ve been here a while and want to figure out where you are), this is your map.
You’re going to build LogStream — a real, production-grade distributed log processing platform — over 254 lessons. Same architecture pattern Cloudflare, Datadog, and Stripe use to handle billions of events. You’ll write every line of code. By the end, you’ll have a working system on GitHub that demonstrates senior-level distributed systems skills.
This post tells you:
The 6-module structure (so you know where you’re going)
How to start today
What’s free vs paid
How to get the most out of this
Read tell end to get your fee GIFT - “Interview Pack”
The 6 Modules
— Detailed Index 1. Python/Javascript OR 2. Java/Spring Boot
Think of LogStream as 6 layers stacked on top of each other. Each module is self-contained — if you only care about one, you can skip there.
Module 1 — Ingestion & Transport (Days 1–40)
You start by getting log events into the system. TCP and UDP servers, batching, compression, network protocol design. By Day 40 you have a log shipper sending events from any source to a central server at 10K events/sec.
Module 2 — Distributed Messaging with Kafka (Days 41–90)
You connect your shipper to Kafka. Producers, consumers, exactly-once semantics, consumer groups, dead letter queues, idempotency. By Day 90 you have a fault-tolerant message bus.
Module 3 — Storage & Persistence (Days 91–130)
Logs need to land somewhere durable. Flat files with rotation, time-series storage, S3 archival, compaction. By Day 130 your system survives node failures.
Module 4 — Real-Time Processing (Days 131–170)
Stream processing on the events flowing through. Parsing, enrichment, alerting, anomaly detection. By Day 170 your system reacts to events as they happen.
Module 5 — Search & Query (Days 171–210)
Elasticsearch integration, query optimization, aggregations, dashboards. By Day 210 you can query a year of logs in milliseconds.
Module 6 — Production Operations (Days 211–254)
Kubernetes deployment, monitoring, auto-scaling, chaos testing, observability. By Day 254 your system runs in production with PagerDuty integration.
How to Start Today
If you’re brand new:
Grab the free Distributed Systems Interview Pack — 30 questions and answer outlines I’ve used to prep for staff interviews. It’s free, no spam.
Read Day 1: Setting Up Your Distributed Systems Environment — install the tools, clone the starter repo.(use this link for JAVA)
Decide on your cadence. Daily? 3x/week? Weekends? Pick one and stick to it.
Subscribe to paid when you hit Day 4 — that’s where the meaty content begins. Or grab Lifetime now if you know you’ll commit.
If you’ve been here a while:
Browse the archive by module
Jump to the module that solves your current problem at work
Join the Discord (paid subs only) — there are people working on the same lessons you are
Free vs Paid
Free: Lessons 1–3 of every module, Sunday weekly digest, all Substack Notes. That’s about 18 free lessons + a steady stream of insights via Notes and the weekly.
Pro ($10/month or $99/year): Every lesson, every GitHub repo, Discord access
Lifetime ($599 once): Everything Pro gets, forever, plus:
Monthly Founding Member live Q&A (I show up, answer questions, code if needed)
Completion certificate when you finish all 254 lessons
One year of systemdrd.com — the broader hands-on course portal
Direct DM access (within reason)
How to Get the Most Out of This
Three things separate the people who finish from the people who lurk:
1. Actually run the code.
Every lesson has a working docker-compose up. If you read without running, you’re just consuming content. Run the code. Hit the errors. Debug them. That’s the entire point.
2. Use the GitHub repo as a checkpoint, not a shortcut.
Every lesson links to working code. Try it yourself first, then check the repo. If you copy-paste before trying, you’re cheating yourself. The whole value is in writing every line.
3. Show up in Discord.
Paid subs have access. The single best predictor of who finishes the course is whether they’re active in Discord. Get unstuck faster. Help someone else. Build the network you’ll use for your next job.
What You’ll Have at the End
Open your laptop in 254 days. You’ll have:
A complete distributed log processing platform on GitHub
Production-quality code reviewed by hundreds of engineers in this community
Architecture diagrams you drew yourself, not copied
Working knowledge of Kafka internals, exactly-once semantics, distributed coordination, observability — the things that get asked in staff-level interviews and the things that matter on call at 3am
A completion certificate (Lifetime only) and a community of engineers who built the same thing
This is a portfolio piece. People I’ve trained have used this work in their portfolios for offers from Datadog, Cloudflare, and yes, FAANGs.
A Word on the Pace
You’re going to fall behind. Everyone does. That’s fine.
This is not a race. It’s a reference. The lessons don’t disappear. If you skip a week, you skip a week. Come back when you’re ready.
The people who succeed treat this like a textbook they’re working through, not a Netflix series they’re binging.
Questions?
Hit reply on any email. Comment on any post. Or jump into the first lesson.
Welcome aboard.
— Sumedh
P.S. The free Interview Pack is genuinely worth grabbing even if you don’t subscribe. Get it here.