Kafka is a distributed streaming platform — a tool that lets applications send, store, and process data in real time.

Think of it like a postal system for data. Instead of two apps talking directly to each other, they drop messages into Kafka, and the right services pick them up when needed.

Originally built by LinkedIn to handle their massive activity feeds. Later open-sourced under the Apache Software Foundation. Now used by Uber, Netflix, Spotify and 80%+ of Fortune 100 companies.

The Problem Kafka Solves

Imagine you have 3 services producing data (Producer A, B, C) and 3 services consuming data (Consumer X, Y, Z).

Without Kafka: Every producer has to connect directly to every consumer. That's 9 connections. Now scale that to 50 services — it becomes an unmaintainable nightmare. Every time you add a new service, you have to update all the others.

With Kafka: Every producer just sends data to Kafka. Every consumer just reads from Kafka. That's it. Adding a new service means connecting it to Kafka — nothing else changes.

This makes the system decoupled — services don't need to know about each other at all.

EDA — Event Driven Architecture

Kafka is the backbone of Event Driven Architecture (EDA) - a design pattern where services communicate by firing events instead of calling each other directly.

Real example — you place an order on an e-commerce app:

1. Order Service fires an event: "Order Created"
2. Kafka receives it and puts it on the Orders Topic
3. Payment Service reads from that topic → charges your card → fires "Payment Done"
4. Inventory Service reads that → updates stock → fires "Stock Updated"
5. Shipping Service reads that → dispatches the package