Http on Shane's Personal Blog

The Art of Buying a Toaster

Wed, 13 May 2026 00:00:00 GMT

If you’re new, the story so far — in What a Switch Actually Does I admitted I’d been faking my way through networking for years, dug into what a switch actually does, and finally understood why a listener bound to 127.0.0.1 is invisible to a container connecting via 10.88.0.1.

For everyone else: this post is a slight detour, and on purpose.

The Embarrassment

Somewhere in the middle of writing post 1, I tried to explain what I was learning to a friend who doesn’t write code. We were having dinner. She’d politely asked what I’d been geeking out about all week.

I made it about three sentences in before her eyes did the thing.

You know the thing. The micro-glaze. The “I love you and I am still here, but I am no longer parsing English.” I’d already said “TCP” and “the kernel” and “bridge interface” and I was about to say “MAC address,” and I caught myself and stopped.

Here’s what hit me. I could now explain a switch. I could explain a virtual bridge. I could explain why my Portainer container couldn’t see my SSH tunnel. But if my friend had asked me — what actually happens, from start to finish, when she clicks Buy on a website? — I would have started flailing. I had pieces. I didn’t have a story.

So I sat down to write one. A real story. With characters. With a setting. With stakes. No jargon. The kind of story I could tell her over the rest of dinner.

The decoder ring — what every character “really” is in computer terms — is at the end. Read the story first. If you’ve never thought about how a website actually works, that’s perfect. Especially then.

The Cast

It helps to know the small cast before we begin. They’re going to come and go quickly.

Jack. A man at home, shopping online. He wants a toaster. This is his entire characterisation.

Browser-Bot. Jack’s little assistant. Lives in his laptop. Writes notes, packs them into parcels, hands them to couriers. Faithful, fussy.

The Sea of Couriers. A vast, restless relay of runners between Jack’s home and the shop. Each one knows only the next runner on the route. They never read the notes; they only pass them along.

The Front Gate. A small, stoic doorperson at the entrance of an office building somewhere far from Jack. Its entire job is to accept parcels addressed to this building and refuse all others. It cannot read.

Mama Kernel. The office manager. Knows every door, every desk, every resident. She sits in a back office with three things on her table: a directory of every department in the building, a rulebook for rewriting addresses, and an enormous ledger where she writes down every parcel that came in and what she did to it. Calm, unflappable, has seen everything.

Pre-Clerk and Post-Clerk. Two assistants who work in pairs. The Pre-Clerk meets every incoming parcel at the door and sometimes rewrites the address on the outside. The Post-Clerk does the same for outgoing parcels — and he keeps a copy of every rewrite his colleague made, so he can put things back exactly as they were on the way out.

The Hallway Concierge. Runs the inside corridor of the building. Knows every resident by sight. His only superpower is delivering a parcel from one apartment door to another — never up, never down, just sideways along his hallway.

Frieda the Frontend. A friendly resident on the corridor. Apartment 10. She runs a kind of front desk: she greets every visitor’s parcel, looks at what it’s asking for, and either answers it herself or politely forwards it down the hall to whoever can.

Brutus the Backend. Frieda’s neighbour. Apartment 11. Gruff bookkeeper. Will not talk to outsiders. Will only accept parcels handed to him by Frieda. Does the actual work — opening ledgers, debiting accounts, marking inventory.

The Parcels. Every message in this story is shaped like a Russian nesting doll. A letter inside an envelope inside a packet inside a delivery satchel. Each layer is addressed to a different recipient, for a different leg of the journey. We will watch them get unwrapped and rewrapped many times.

That is everyone. Now to the story.

Act One — The Click

Jack wants a toaster.

He has, in fact, wanted a toaster for several weeks. The old one finally died last Sunday, dramatically, in a small puff of smoke that made the cat leave the room. He has been putting this off because online shopping always feels like more of a commitment than it ought to. But today is the day.

He scrolls. He picks one. He clicks Buy.

In the corner of Jack’s laptop, Browser-Bot springs into action. He’s a tidy little creature, all bow-tie and clipboard. He scribbles a short note — barely a sentence: “this customer would like to buy one toaster” — and begins to wrap it for the road.

First, the note goes inside a small envelope, addressed in Browser-Bot’s neat handwriting. Then that envelope goes inside a larger packet, with a different address on it — the address of the shop, far away. Then the whole packet goes inside an outer satchel, with yet a third address on it — this one for the very first relay courier waiting outside Jack’s front door.

Three layers. Three addresses. Each one meant for a different leg of the journey. Browser-Bot has done this a thousand times today already. He hands the satchel out the door.

Act Two — Across the Sea of Couriers

A relay runner is already waiting. She takes the satchel, glances at the outer address, and runs.

She runs to the next runner, who tears off the outer wrapping — it had served its purpose — looks at the next address inside, wraps that inner packet in her own fresh satchel addressed to the next relay over, and hands it on.

This goes on for a long time.

Every runner along the way does the same thing: tear off the outer wrapping that was only meant for the last leg, look at the address on what’s inside, wrap it fresh for the next leg, send it along. The inner packet — the one with the actual shop’s address on it — never gets opened. It is too sacred. It is meant to travel.

If you could speed up time and watch from above, you’d see a glowing point hopping from runner to runner across a continent, leaving a trail of discarded satchels in its wake. The note inside has not moved relative to its packet. The packet has not moved relative to itself. Only the outermost wrapper keeps getting reborn.

Eventually, after what is in human terms a tiny fraction of a second, the packet reaches the office building it was always destined for. The last relay courier hands it across to the building’s Front Gate, gives a half-salute, and runs off into the night.

Act Three — The Onion Unwrapped

The Front Gate is uninterested in the contents of the satchel. The Front Gate is only interested in one thing: the name written on the outermost layer. It is the building’s name. The Gate accepts the parcel. The Gate’s job is done.

The parcel is conveyed into the back office, where Mama Kernel is waiting.

She picks it up gently. She tears off the outer satchel — that was only ever for the last leg of the road. She lays the inner packet on her table. The address on that is the building’s, all right, but written in a more permanent kind of way. Good. For us.

She is about to unwrap the next layer when the Pre-Clerk appears at her elbow.

“This one’s for Apartment 10,” he says. “Sales inquiry. Frieda’s department.”

Mama Kernel nods. The Pre-Clerk takes a pencil and, very gently, edits the address on the packet — changing the building’s name to Frieda’s exact apartment number. He turns to his ledger and notes carefully: Parcel #8472, originally addressed to the building, redirected to Apartment 10. Remember to reverse this on the way back. He underlines remember three times. He is very serious about his job.

Mama Kernel takes the relabelled packet and consults her directory. Apartment 10 — that’s down the corridor, the Concierge’s territory. She wraps the packet one more time, in a small in-building courier sleeve addressed by name to Frieda herself, and hands it through her window to the Hallway Concierge.

Act Four — Down the Corridor

The Concierge has, by now, memorised the face of every resident on his corridor. Frieda the Frontend, Apartment 10 — of course, of course. He whisks the parcel down the hall and slides it through the slot in her door.

Frieda is, as ever, at her front desk in a sunny mood. She unwraps the parcel — courier sleeve off, the building-layer wrapping off, the original travel packet off — and unfolds the small inner envelope and finally reads the note inside.

“This customer would like to buy one toaster.”

“Right!” Frieda says, brightly, to nobody. “That’s Brutus’s department.”

She produces a clean piece of paper and writes her own note: “a customer would like to buy one toaster — please process.” She wraps it in her own small envelope, addresses the envelope to Apartment 11 next door, wraps that in her own little courier sleeve, and slides the whole thing back out under her door for the Concierge.

This is important. Frieda did not forward the original parcel. She wrote a new parcel, with a new address, asking Brutus for the same thing on her behalf. The original parcel sits on her desk like a receipt. She’ll need it again in a minute.

The Concierge takes Frieda’s new parcel — three short steps down the hall — and slides it under Brutus’s door.

Brutus, in Apartment 11, does not look up from his ledger. He hears the parcel arrive. He sighs. He unwraps it. He reads. He grumbles. “Toaster. Customer. Fine.”

He turns to his great accounting book. He opens it. He debits one toaster from the inventory. He writes a note next to Jack’s name. He stamps the whole thing CONFIRMED in red ink. Then he writes a tiny reply — “order received, will ship, ID #42” — wraps it in a small envelope addressed to Frieda, and slides it back into the hall.

Concierge. Three steps. Frieda’s slot.

Frieda reads Brutus’s reply, smiles, and now turns back to the parcel she had set aside. She picks up her pen and writes a reply of her own, addressed back to the original sender — Jack — saying “order confirmed, your toaster is on its way, thank you for shopping with us.” She wraps it in the same nesting style she received: envelope inside packet inside courier sleeve. She slides it under her door.

Act Five — Home

The reply travels in reverse, exactly the way the request came. Concierge to Mama Kernel. Mama Kernel consults her directory: the sender lives way out beyond the building’s walls, so she’ll need to send it out via the Front Gate.

But before she can wrap it for outbound travel, the Post-Clerk appears at her elbow. He flips open his ledger to the page about parcel #8472.

“Don’t forget,” he says. “When this came in, my colleague rewrote the address from the building’s name to Apartment 10. We need to reverse that now. Frieda’s apartment number should not appear on this parcel as it leaves — to the outside world, the reply came from the building, not from any one apartment.”

He pencils in the correction. He closes the ledger.

Mama Kernel wraps the now-correctly-addressed packet in a fresh outer satchel and hands it to the Front Gate, who passes it to the first relay runner waiting outside. The Sea of Couriers reverses the relay across the continent, satchel by satchel, until at last a single runner sprints up Jack’s driveway and slips the parcel under his door.

Browser-Bot receives it, unwraps it ceremonially, smooths out the reply note, and reads.

“Order confirmed. Your toaster is on its way. Thank you for shopping with us.”

In Jack’s browser, the Buy button quietly turns into a green checkmark and changes its name to ✓ Ordered. Somewhere, a cat looks up from a sunny patch on the rug. Jack smiles.

He thinks: that was so easy.

The Decoder Ring

It was not, in fact, easy.

Here is what each character secretly was. Read it the way you’d read the credits at the end of a play and realise you’ve been watching something more elaborate than you thought.

Jack and his Browser-Bot. A user at a computer, and the web browser running on that computer. The browser is the program that translates human intent — “I clicked Buy” — into the actual sequence of messages a server needs to receive.

The note Browser-Bot wrote, and the nesting wrappers. This is the heart of the whole story, so I’ll spend a moment on it.

Every message that crosses the internet is built like a nesting doll, with each wrapper meant for a different audience. The innermost note is the actual content — in our case, “buy one toaster.” That’s called an HTTP request, the language web browsers and web servers use to talk to each other.

Around that, Browser-Bot wraps a TCP segment — a wrapper that adds reliability features (sequence numbers, retransmission, confirmation receipts) so that if any part of the journey loses the parcel, it can be detected and resent. TCP is what makes the internet reliable; without it, every web page would be a coin flip.

Around that, an IP packet — the layer that carries the actual end-to-end address. The IP packet says this is for the shop’s public address. The IP packet is the one wrapper that, like in the story, mostly never gets opened or rewritten as it crosses the continent. It travels end to end.

Around that, an Ethernet frame — the outermost satchel. The frame only knows about the next single hop. It’s addressed to the next runner, not to the final destination. At every hop along the way, the frame gets torn off and a new one written for the next hop.

That nesting — letter inside TCP inside IP inside Ethernet — is the famous “OSI layered model” most of us were taught in school as seven boxes to memorise. It is, in practice, just this: a parcel inside a parcel inside a parcel, each for a different audience along the route.

The Sea of Couriers. Routers across the public internet. Each router only knows the next hop; together they form a relay across continents. The fact that the inner IP packet never gets rewritten — only the outer Ethernet frame, fresh at every hop — is what makes the internet actually work. The IP packet survives end to end while the wrappers around it are disposable.

The Front Gate. The host’s network card — the NIC — facing the outside world. On a Linux box this interface is usually called eth0. Its job is exactly as described: accept frames addressed to us, drop the rest, hand the accepted ones up to the kernel.

Mama Kernel. The Linux kernel itself. She owns every interface in the building, every routing decision, every address rewrite, every connection in flight. The directory she consults is the kernel routing table; the rulebook beside it is iptables / netfilter; the great ledger of every connection she’s tracking is the conntrack table.

Pre-Clerk and Post-Clerk. These are the iptables hooks called PREROUTING and POSTROUTING. The rewriting they do is NAT — Network Address Translation. The specific kind of rewriting in our story (an incoming connection getting redirected to an internal apartment number) is called DNAT (Destination NAT). The reason Post-Clerk has to reverse the rewrite on the way out is so the outside world only ever sees the building’s public address, never the internal apartment numbers.

The Hallway Concierge. A virtual bridge inside the kernel. On a Linux box running containers, it’s commonly named docker0 or podman0. It is, behaviourally, exactly the kind of switch we explored in post 1 — a thing that delivers messages between residents of one corridor by name, by lookup or by flooding, and does nothing else. The fact that it’s software and not a metal box in a rack is a detail; the algorithm is identical.

Frieda the Frontend and Brutus the Backend. Two containers, each living in its own isolated apartment. Frieda might run, say, an nginx server that handles incoming web traffic. Brutus might run a database or an order-processing API. They’re plugged into the same internal corridor (the bridge) but isolated from each other — and from the rest of the building — in a way we still have to talk about.

The moment Frieda decided to “write a new parcel to Brutus” instead of “forwarding the original” is exactly the moment a real reverse proxy makes a fresh request to an upstream service. It is two separate conversations stitched together, not one conversation passed through. That distinction matters more than it sounds; we’ll come back to it.

Apartment 10 and 11, the corridor, the building. These are the IPs you’ve been seeing all over post 1 and this one. Frieda’s apartment number is 10.88.0.10. Brutus’s is 10.88.0.11. The corridor is the 10.88.0.0/16 subnet. The host — Mama Kernel’s own seat at the corridor’s table — is 10.88.0.1. The building’s public address is whatever IP the outside world sees the host as; that one varies.

The conversation between Frieda and Brutus that the outside world never saw. This is the inter-container traffic on the bridge. Notice it crossed no NAT, no routing decision worth speaking of, no outside interface. Two apartments, one corridor, Concierge passing parcels. Cheap, fast, invisible from outside. This is why container-to-container traffic on a bridge is so quick — the kernel basically never has to leave the lower layers.

The two ledgers being kept in lockstep. That’s connection tracking — conntrack — the kernel feature that remembers every active connection and what rewrites were applied to it, so reply traffic can be reversed perfectly even though the rewriting was applied asymmetrically. Without this, no NAT scheme on Earth would work. With it, the entire modern internet (where every home router does this, all day, for every device on your home network) functions invisibly.

The Aha

When my friend at dinner heard “the internet,” she pictured, I think, a kind of pneumatic tube. You put a thing in one end, it comes out the other. Magic in the middle.

The middle is not magic. The middle is a relay of small, dumb, identical actors, each doing one tiny job per parcel, none of them aware of the others, none of them aware of the contents. The “intelligence” lives at the two ends — Browser-Bot wrapping the parcel with the right addresses on each layer, and the receiving server unwrapping them and replying. The middle is just a sequence of helpful strangers passing notes.

Inside the office building, the relay continues. Mama Kernel is more sophisticated than any one courier — she can rewrite addresses, route between corridors, keep ledgers — but she is still, fundamentally, a series of small mechanisms in sequence. Receive at the door. Strip the outer wrapper. Look at the address. Maybe rewrite it. Look up where it goes. Wrap it again. Send it out the right inner door. That’s a pipeline, not a wizard. You can hold the whole thing in your head once you stop expecting it to be one big magic step.

This was the second click for me, after the switch one. The switch unlocked what happens at one moment in time, in one place. The toaster story unlocked what happens across an entire journey. They go together.

What I Still Don’t Understand

There’s a moment in the story I quietly waved past.

When the parcel arrived at Frieda’s apartment, I said her “local kernel” peeled the wrappers, her own door slot received the courier sleeve, her own desk held the original packet. Frieda has, apparently, her own little internal world: her own front door, her own loopback to herself, her own apartment number, her own desk, her own everything.

But Frieda isn’t a separate computer. Frieda is just a process running on the same host as Mama Kernel. There is only one kernel in this building. There is only one set of hardware.

So how does Frieda’s apartment exist? How does it have its own internal addresses, its own loopback, its own private experience of the corridor that doesn’t trample on Brutus’s right next door? How does the same kernel run multiple parallel network worlds side by side, each believing it’s alone?

The answer is the feature I waved at in post 1 without explaining. It’s called a network namespace, and once it clicks, the entire reason containers feel like “their own little computers” snaps into focus. It’s also the foundation under VPNs, under sandboxing, under a lot of zero-trust networking — and under the bug I’m slowly explaining across this whole series.

I’ll get there. I promise.

See you next post — whatever rabbit hole I fall into first.

(Written by Human, improved using AI where applicable.)

What Actually Is ASGI? A Journey from Confusion to Clarity

Tue, 04 Nov 2025 00:00:00 GMT

The Mystery of the Invisible Scope

I was building a feature that needed real-time updates. Nothing fancy—just wanted to push some data to clients as things changed on the server. Naturally, I reached for FastAPI and started reading through the documentation. That’s when I first encountered it.

scope.

The docs mentioned it everywhere. Tutorial code referenced it. Stack Overflow answers assumed I knew what it was. But here’s the thing that drove me crazy: I couldn’t find it in any of my actual code. It was like everyone was talking about this mysterious variable that just… existed somewhere. Nowhere in my route handlers, nowhere in my WebSocket endpoints. Where was this magical scope object?

I felt like I’d walked into the middle of a conversation where everyone understood the context except me.

Then I started seeing receive and send mentioned too. Same problem—referenced everywhere, visible nowhere in the framework code I was writing. AI assistants kept explaining things in terms of these three parameters, but I couldn’t connect the abstract explanations to the concrete code I was looking at in Litestar and Advanced Alchemy documentation.

That’s when I realized: I’d been learning frameworks without understanding what they were built on. I needed to go deeper—to the bare metal of how Python async web servers actually work. Not the FastAPI way or the Django Channels way, but the fundamental protocol underneath.

This is the story of that journey. If you’ve ever felt confused about ASGI, WebSockets, or why async Python web stuff feels so different from traditional Flask/Django, this is for you. By the end, you’ll understand not just what ASGI is, but why it exists and how it creates a unified model for everything from simple HTTP requests to long-lived WebSocket connections.

The First Aha Moment: ASGI Is Just a Contract

Here’s what finally clicked for me: ASGI isn’t a framework or a library. It’s a specification—a contract for how web servers talk to Python applications.

Think of it like this: back in the day, everyone agreed on how electrical outlets should work. Any appliance manufacturer could build a device knowing it would fit the standard outlet. ASGI is the same idea, but for async Python web servers and applications.

The entire contract boils down to three parameters:

async def app(scope, receive, send):
    # Your application logic here

That’s it. That’s the whole interface. Every ASGI application—whether it’s FastAPI, Starlette, Django Channels, or something you build from scratch—is fundamentally just a callable that accepts these three parameters:

scope: A dictionary containing metadata about the connection. Think of it as the “context” or “session info”—what kind of connection is this? Where’s it coming from? What’s being requested?
receive: An async function you call to get messages from the client. Like checking your mailbox for incoming letters.
send: An async function you call to send messages to the client. Like putting outgoing letters in the mailbox.

The beautiful part? This same simple contract handles everything—regular HTTP requests, streaming Server-Sent Events, bidirectional WebSockets, even application startup and shutdown. The scope just tells you which kind of communication pattern you’re dealing with.

But here’s what confused me at first: what exactly counts as a “session”? Is it one request? A conversation? The lifetime of the server?

The answer is: it depends on the scope type. And that’s where things get interesting.

HTTP: The Simple Transaction

Let me start with the familiar: plain old HTTP requests. This is probably what you already understand intuitively, even if you didn’t know ASGI was involved.

When a client makes an HTTP request to your server, ASGI creates a scope that looks something like this:

{
    "type": "http",
    "method": "GET",
    "path": "/api/users/123",
    "headers": [...],
    "query_string": b"format=json",
    "client": ("192.168.1.5", 54321),
    "server": ("10.0.0.1", 8000),
}

The type: "http" is the key—it tells your application “this is a simple HTTP request-response transaction.”

Here’s the flow:

Client sends request → Server creates scope and calls your app
Your app calls receive() to get the request body (maybe in chunks)
Your app calls send() with response.start (status code, headers)
Your app calls send() again with response.body (the actual content)
Scope ends. Connection done.

Think of it like a vending machine: you insert money (request), press a button (the route), and get a snack back (response). Transaction complete. The machine doesn’t remember you existed.

That “doesn’t remember” part is crucial—HTTP in ASGI is stateless. Each request is independent. The scope lives for maybe 100 milliseconds, just long enough to handle one request-response cycle.

Example messages you’d send:

# First, send the response headers and status
await send({
    "type": "http.response.start",
    "status": 200,
    "headers": [[b"content-type", b"application/json"]],
})

# Then send the actual response body
await send({
    "type": "http.response.body",
    "body": b'{"user": "Shane", "id": 123}',
})

Simple, stateless, short-lived. This is the foundation—the easiest pattern to understand.

Server-Sent Events: The One-Way Stream

Now, what if your vending machine needed to keep telling you about new snacks as they were restocked? You’d want to stay connected and receive updates, but you wouldn’t be sending anything back except “yes, keep the updates coming.”

That’s Server-Sent Events (SSE).

Here’s what confused me initially: SSE uses an HTTP scope, but it doesn’t follow the typical request-response pattern. Instead:

Client makes an HTTP request
Server responds with headers (including Content-Type: text/event-stream)
Connection stays open
Server keeps sending chunks of data whenever it wants
Eventually, either side closes the connection

The scope is still type: "http", but the interaction is different. It’s like calling a restaurant for their daily specials, and instead of hanging up after they tell you today’s menu, they keep you on the line and tell you every time a new special is added.

Example of what the server might send:

await send({
    "type": "http.response.start",
    "status": 200,
    "headers": [[b"content-type", b"text/event-stream"]],
})

# Then keep sending updates...
await send({
    "type": "http.response.body",
    "body": b"data: {\"new_order\": 42}\n\n",
    "more_body": True,  # Signal that more data is coming
})

# ... later ...
await send({
    "type": "http.response.body",
    "body": b"data: {\"new_order\": 43}\n\n",
    "more_body": True,
})

Notice more_body: True? That’s the key. It tells ASGI “don’t close the connection, I’ve got more to send.”

SSE is still fundamentally HTTP—it’s just long-lived HTTP. The client doesn’t send data back through this connection (it’s one-way), but the connection can stay open for minutes or even hours.

When to use SSE:

Live dashboards showing real-time metrics
Notification feeds
Stock tickers
Progress updates for long-running tasks
Any scenario where the server pushes updates but doesn’t need responses

It’s simpler than WebSockets because it’s unidirectional, but more powerful than regular HTTP because it’s persistent.

WebSocket: The Full Conversation

Then I hit WebSockets. And this is where my understanding of ASGI really got tested.

WebSockets are fundamentally different from both HTTP and SSE because they’re bidirectional and stateful. This isn’t a transaction or a one-way stream—it’s a ongoing conversation where both sides can speak whenever they want.

If HTTP is like sending letters and SSE is like listening to a radio broadcast, WebSocket is like a phone call.

The scope for WebSocket looks different:

{
    "type": "websocket",
    "path": "/ws/chat/room-42",
    "headers": [...],
    "query_string": b"user=shane",
    "client": ("192.168.1.5", 54322),
    "server": ("10.0.0.1", 8000),
}

Notice: type: "websocket". This signals a completely different kind of interaction.

WebSocket messages include:

Incoming (from client to your app):

{"type": "websocket.connect"}       # Client wants to start a session
{"type": "websocket.receive", "text": "Hello!"}  # Client sent a message
{"type": "websocket.disconnect"}    # Client ended the session

Outgoing (from your app to client):

{"type": "websocket.accept"}        # You approve the connection
{"type": "websocket.send", "text": "Welcome!"}   # You send a message
{"type": "websocket.close"}         # You end the session

See the difference? With HTTP, you receive a request and send a response. With WebSocket, you’re dealing with a state machine: connect → accept → ongoing send/receive loop → close.

The scope lives for the entire WebSocket session—could be seconds, could be hours. You might exchange hundreds of messages within a single scope. This is fundamentally stateful—the server remembers this connection exists and can push data to it anytime.

When to use WebSocket:

Real-time chat applications
Collaborative editing (like Google Docs)
Multiplayer games
Live video/audio streaming control
Any scenario requiring true bidirectional real-time communication

But here’s what really confused me when I first learned this…

The Mystery of the WebSocket Handshake

I kept seeing references to “WebSocket upgrade” and “HTTP to WebSocket upgrade handshake.” And I had this burning question: if WebSocket is different from HTTP, how does it even start?

Here’s the part that blew my mind: WebSockets always begin as HTTP requests.

Wait, what?

Let me explain. WebSockets were designed to work with existing web infrastructure—the same ports, the same proxies, the same SSL/TLS setup. So instead of inventing an entirely new protocol from scratch, they built it on top of HTTP.

Here’s what actually happens when a browser wants to open a WebSocket connection:

Step 1: Client sends a special HTTP request

GET /ws/chat HTTP/1.1
Host: example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: X3JJHMbDL1EzLkh9GBhXDw==
Sec-WebSocket-Version: 13

This is a regular HTTP GET request, but with special headers saying “hey, I’d like to upgrade this connection to WebSocket.”

Step 2: Server responds with HTTP 101

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: <computed response>

That 101 Switching Protocols status is the magic handshake. It means: “Okay, we’re no longer speaking HTTP. From this moment forward, this TCP connection will use WebSocket protocol instead.”

Step 3: Protocol switch happens

The underlying TCP connection stays open, but HTTP is done. Now both sides start speaking WebSocket framing protocol—a completely different language.

Think of it like this: You call a restaurant (HTTP), ask to be transferred to a specific table (upgrade request), the host says “transferring you now” (101 response), and suddenly you’re having a direct conversation with someone at that table (WebSocket). Same phone line, different conversation protocol.

What Happens at the ASGI Layer?

Here’s what confused me the most: where does ASGI fit into this handshake?

The answer: ASGI comes in after the handshake is complete.

Let me break down what happens at different layers:

Server Level (Uvicorn/Hypercorn—automatic, not your code):

Receives the HTTP upgrade request
Validates WebSocket headers
Sends back the 101 Switching Protocols response
Switches the TCP connection to WebSocket framing

ASGI Level (your application code):

Server creates a WebSocket scope
Server sends you a websocket.connect message
Now YOU decide: accept or reject?

This two-layer decision-making finally made sense when I understood it this way:

Protocol layer (server): “Is this a valid WebSocket upgrade request?”
Application layer (your code): “Do I want to allow this specific connection?” (authentication, authorization, rate limiting, etc.)

The handshake is already done by the time your application sees websocket.connect. The client is already “connected” at the protocol level. Your websocket.accept isn’t completing the handshake—it’s your application’s business logic saying “yes, I approve this session.”

Example flow in your ASGI app:

# You receive this from the server
message = await receive()
# message = {"type": "websocket.connect"}

# Now you decide - maybe check authentication
if user_is_authenticated:
    await send({"type": "websocket.accept"})
    # Now the bidirectional message loop can begin
else:
    await send({"type": "websocket.close", "code": 1008})
    # Reject the connection - unauthorized

This distinction between server-level (protocol) and application-level (business logic) was my biggest “aha!” moment. The framework documentation suddenly made sense—FastAPI’s WebSocket endpoints, Starlette’s connection handling, all of it was operating at the application layer, trusting the server (Uvicorn) to handle the protocol layer.

Either side can close the connection at any time. There’s no “client is in control” or “server is in control”—it’s a true peer-to-peer conversation (well, as peer-to-peer as client-server can be). When either side sends a close message, the WebSocket session ends and the scope completes.

Lifespan: The Orthogonal Dimension

Just when I thought I understood the pattern—HTTP, SSE, and WebSocket representing different communication patterns—I encountered a fourth scope type that didn’t fit the model at all.

lifespan.

Here’s what tripped me up: I initially thought “lifespan” meant “the lifespan of a connection.” Like, the duration from when a WebSocket connects until it disconnects. That made intuitive sense!

I was completely wrong.

Lifespan has nothing to do with individual requests or connections. It’s about the application itself—the entire server process from startup to shutdown.

Think of it this way:

HTTP scope: exists for one request (~100ms)
SSE scope: exists for one streaming session (minutes to hours)
WebSocket scope: exists for one bidirectional conversation (seconds to hours)
Lifespan scope: exists for the entire application (days to months)

When your server starts up, before it handles any HTTP requests or WebSocket connections, it creates a lifespan scope and sends you startup messages. When the server is shutting down (gracefully), it sends you shutdown messages.

The scope looks like:

{
    "type": "lifespan",
}

The messages you receive:

{"type": "lifespan.startup"}
{"type": "lifespan.shutdown"}

What would you use this for? Things that need to happen once per application, not per request:

Opening database connection pools
Loading machine learning models into memory
Starting background task schedulers
Warming caches
Setting up monitoring/metrics collectors
Graceful cleanup on shutdown

Analogy time: If HTTP/SSE/WebSocket are like serving individual customers in a restaurant, lifespan is like opening the restaurant in the morning (turning on ovens, prepping ingredients) and closing it at night (cleaning up, shutting down equipment).

You don’t want to load a 2GB ML model on every HTTP request—you load it once during lifespan.startup and reuse it for all requests. You don’t want to abruptly kill database connections—you close them gracefully during lifespan.shutdown.

In FastAPI, this is what those @app.on_event("startup") and @app.on_event("shutdown") decorators are doing—they’re handling lifespan messages for you.

Lifespan is orthogonal to the other three scope types. It’s not about communication patterns between client and server—it’s about the lifecycle of the server process itself.

The Unified Mental Model

After all this confusion, trial, error, and eventual clarity, here’s the mental model that finally made ASGI click for me:

ASGI is a universal interface for describing “communication sessions” between servers and applications, where a session can be:

Scope Type	Duration	Direction	Use Case
HTTP	Milliseconds	Request → Response	Simple API calls, page loads, form submissions
SSE	Minutes to hours	Server → Client (one-way)	Live feeds, dashboards, notifications
WebSocket	Seconds to hours	Bidirectional	Chat, collaboration, games, real-time updates
Lifespan	Application lifetime	N/A	Startup/shutdown tasks, resource management

All four use the exact same interface: async def app(scope, receive, send).

The scope dictionary tells you what kind of session you’re dealing with. The receive and send functions let you interact with that session. The same contract, different behaviors.

What makes this elegant is that middleware can work across all types. Want to add authentication? Write middleware that checks the scope type and handles HTTP, WebSocket, and SSE appropriately. Want to add logging? Same deal—one interface, universal application.

This is why frameworks like FastAPI feel so natural once you understand ASGI. Under the hood, every route handler, every WebSocket endpoint, every startup event—they’re all just ASGI applications conforming to this same simple contract.

The mystery of the invisible scope finally made sense. It’s there in every ASGI application—frameworks just abstract it away so you don’t have to think about it for simple cases. But when you need to go deeper, when you need to understand why WebSockets work differently from HTTP or where to put your database initialization, understanding the bare ASGI layer gives you that clarity.

From Confusion to Clarity

When I started this journey, scope was an invisible, mysterious thing that documentation assumed I understood. Now I see it for what it is: a simple dictionary describing a communication pattern.

The beautiful part is how this knowledge transfers. Now when I read FastAPI docs and see WebSocket examples, I understand what’s happening beneath the decorator syntax. When I see Starlette’s startup events, I know they’re handling lifespan messages. When I troubleshoot why a connection isn’t staying open, I can reason about whether it’s an HTTP scope that ended naturally or a WebSocket that closed unexpectedly.

Going to the bare metal—understanding the actual protocol instead of just the framework—transformed my confusion into confidence.

If you’re building real-time features, async APIs, or just trying to understand modern Python web development, I hope this journey helps demystify ASGI for you the way it did for me. Next time you see that mysterious scope parameter in the documentation, you’ll know exactly what it means.

And maybe, just maybe, you’ll find yourself diving even deeper—implementing a minimal ASGI app from scratch, writing custom middleware, or truly understanding what your framework is doing behind the scenes.

The rabbit hole goes deeper if you want it to. But at least now you know where the entrance is.

(Written by Human, improved using AI where applicable.)