Software Engineering on Shane's Personal Blog

Temporal Time-Skipping: The Clock You Didn't Know Existed

Mon, 09 Mar 2026 00:00:00 GMT

I spent hours debugging a test that worked perfectly in production but exploded in the test environment. The workflow ran fine. The activities completed. The signals were correct. But every single test failed with a TimeoutError — the workflow would just… die, before my test code even had a chance to interact with it.

The culprit was a clock I didn’t know existed.

This post is about the mental model that finally made Temporal’s time-skipping test server make sense to me. If you’ve ever been confused by WorkflowEnvironment.start_time_skipping(), or had tests fail mysteriously with timeouts that don’t happen in production, this is for you.

What is Temporal? (The 30-Second Version)

Temporal is a workflow orchestration engine. You define your business logic as a “workflow” — a sequence of steps — and Temporal takes care of running it reliably. If a step fails, Temporal retries it. If your server crashes, Temporal picks up where it left off. It handles all the ugly stuff: retries, timeouts, state persistence, distributed coordination.

The important thing for this post: Temporal runs your workflow inside its own runtime environment. Your workflow code doesn’t just execute like a normal Python function. It runs inside Temporal, and Temporal manages when things happen.

That distinction — “runs inside Temporal” — is the root of everything that follows.

Temporal is a Timekeeper

Here’s the insight that changed everything for me: Temporal doesn’t skip your code. It only controls its own clock.

I had been thinking about Temporal as some kind of execution engine that speeds up my code. It’s not. Temporal is a timekeeper. Think of it like a kernel — a central dispatcher that keeps a timetable of scheduled events.

When your workflow calls workflow.sleep(3600) (sleep for one hour), Temporal doesn’t somehow make your Python code run faster. What it does is add an entry to its internal timetable:

"Wake up workflow ABC at current_time + 3600 seconds"

When your workflow starts an activity with a 30-second timeout, Temporal adds:

"If activity XYZ hasn't returned by current_time + 30 seconds, fail it"

When your workflow has an execution timeout of 10 minutes:

"If workflow ABC isn't done by current_time + 600 seconds, kill it"

Sleeps, timeouts, activity deadlines, heartbeat intervals — from Temporal’s perspective, they’re all the same thing. They’re entries in a timetable. Each one says: “fire event X at time T.” The only difference is what event gets fired — resume the workflow, fail an activity, kill the whole thing. But the mechanism is identical: a timestamp and an action.

This is the foundation for understanding time-skipping.

Time-Skipping: Fast-Forwarding Idle Time

Temporal’s Python SDK gives you two test environments:

WorkflowEnvironment.start_time_skipping() — downloads a lightweight Rust-based test server binary that runs in-process (no Docker needed). This server has a virtual clock that can jump forward.
WorkflowEnvironment.start_local() — runs a full Temporal server with a real clock, just like production.

The time-skipping server does one clever thing: when nothing is happening, it fast-forwards its clock to the next scheduled event.

Imagine your workflow does this:

await workflow.sleep(3600)   # sleep 1 hour
await do_some_activity()
await workflow.sleep(7200)   # sleep 2 hours

In production, this takes 3 hours of wall-clock time (plus however long the activity takes). With time-skipping, here’s what happens:

Workflow calls sleep(3600). Temporal adds to its timetable: “resume at now + 3600s.”
Nothing else is pending. The server jumps its clock forward 3600 seconds instantly.
Timer fires. Workflow resumes. Activity starts.
Activity is running — the server does not skip, because it’s waiting for a real result.
Activity completes. Workflow calls sleep(7200). Timetable entry added.
Nothing pending. Server jumps forward 7200 seconds.
Timer fires. Workflow finishes.

Total real time: however long the activity took (maybe milliseconds if it’s a mock). The 3 hours of sleeping? Gone. Skipped.

This leads to a really important corollary that solidified my understanding:

If your code never sleeps, never waits, and never sets timeouts, time-skipping gives you zero speedup.

Think about it. Time-skipping only fast-forwards idle time on Temporal’s clock. If there’s no idle time — no timers, no sleeps, no timeouts — there’s nothing to skip. Your workflow would run at the exact same speed with time-skipping as without it.

Of course, that’s a theoretical extreme. Real workflows almost always have timeouts, retry intervals, and sleeps. But the principle is clarifying: time-skipping is not about making your code faster. It’s about eliminating wait time between events on Temporal’s timetable.

The Handle: Your Remote Control

When you start a workflow, Temporal gives you back a handle. Think of it like a restaurant ticket — you placed your order (started the workflow), and now you have a ticket to interact with it while it’s being prepared.

handle = await client.start_workflow(
    MyWorkflow.run,
    inputs,
    id="order-123",
    task_queue="kitchen",
)

The workflow is now running independently inside Temporal’s runtime. The handle is your remote control. It has a few buttons:

handle.result() — “Call me when my food is ready.” You sit on the phone, waiting, until the workflow finishes and gives you the result.
handle.query() — “Hey, how’s my order coming?” A quick question. You get an answer immediately and hang up.
handle.signal() — “Actually, add extra cheese.” You send a message to the running workflow. It gets delivered immediately.
handle.cancel() — “Cancel my order.”

The handle is just temporalio.client.WorkflowHandle — nothing magical. But understanding what each button does to the time-skipping server is where it gets interesting.

`result()` vs `query()` — The Trap

This is where my intuition was completely backwards, and getting it right is what finally unblocked me.

handle.result() is a phone call where you say: “Don’t hang up until my food is ready.” You’re blocking. You’re waiting. And critically, you’re telling the time-skipping server: “I’m waiting for this workflow to finish.” The server hears that and thinks: “They want the result. Let me help by fast-forwarding to when it’s done.”

handle.query() is a phone call where you say: “Is my food ready? No? Okay, bye.” One shot. You get the current state, and then you’re done. The server has no reason to fast-forward anything — you didn’t ask it to wait for anything.

Here’s the part that tripped me up: result() is event-based (wait for the “done” event), and query() is polling (check the current state and return immediately). You’d think the event-based approach is the “better” one — and in normal programming, it usually is. But in time-skipping mode, the polling approach is safer, because it doesn’t give Temporal permission to mess with its clock.

Operation	What it says to Temporal	Triggers time-skipping?
`handle.result()`	”Wake me up when it’s done”	Yes — server tries to make it be done
`handle.query()`	”What’s the status right now?”	No — server just answers
`handle.signal()`	”Deliver this message”	No — immediate delivery

That middle column is the whole story. result() gives the server license to fast-forward. query() doesn’t.

The Race Condition: Death by Time Travel

Now you have all the pieces. Let me show you how they combine to create a very confusing bug.

The Setup

Our workflow has steps that need user feedback. After a step runs (say, analyzing a document), the workflow pauses and waits for the user to review the output and click “Continue.” In production, this might take minutes or hours — the user is reading, thinking, editing.

In the code, this waiting looks like:

# backend/src/genai/temporal/workflow_executor.py, line 149
await workflow.wait_condition(lambda: waiter.signal_received)

This tells Temporal: “Pause here until signal_received becomes True.” There’s no timeout — it waits as long as it takes. In production, that’s fine. The user eventually clicks Continue, a signal is sent, signal_received flips to True, and the workflow resumes.

The Exam Analogy

Imagine you’re a teacher proctoring an exam with a 2-hour time limit. You have a clock on the wall.

Production (real clock): A student raises their hand. “I need my calculator from my locker.” You wait. Someone brings it. The student finishes the exam. The clock says 45 minutes passed. No problem.

Time-skipping (magic clock): A student raises their hand. “I need my calculator from my locker.” You look around the room. Nobody is actively writing. Nothing is happening. So you spin the magic clock forward — 30 minutes, 1 hour, 1.5 hours, 2 hours. “Time’s up! Exam over!” The student fails.

The person bringing the calculator walks in 0.1 real seconds later. But the clock already says 2 hours. Too late.

What Actually Happened

Here’s the exact sequence in our tests:

Real time 0.00s — Test starts the workflow. Mock activities return instantly (they’re fakes — no real I/O).

Real time ~0.02s — All activities are done. Workflow enters wait_condition() — waiting for a feedback signal. At this moment, Temporal’s timetable has nothing pending: no activities running, no timers set. Just a workflow sitting at a wait_condition.

Real time ~0.02s — Test calls handle.result(): “Tell me when the workflow finishes.”

The time-skipping server hears this and thinks: “The client wants the result. Let me check what’s pending… No activities. No timers. Nothing to wait for except a wait_condition I can’t satisfy. But there IS a workflow execution timeout at now + 600 seconds. Let me jump there.”

Server clock jumps: 0s → 600s.

Workflow execution timeout fires. Workflow dies with TimeoutError.

Real time ~0.03s — handle.result() returns with an error.

Meanwhile, the test had a polling loop that was supposed to query for pending feedback steps and send signals. That loop was scheduled to run after asyncio.sleep(0.1) — at real time 0.1 seconds. But the workflow is already dead. The signals arrive at a corpse.

The whole thing happened in ~30 milliseconds of real time. The server just… jumped to the end.

The Fix

The fix is almost anticlimactic once you understand the problem. Don’t call handle.result() while the workflow is waiting for signals. Instead, use handle.query() to poll, send signals when needed, and only call result() after confirming the workflow is already done:

# backend/tests/integration/genai/test_temporal_workflow.py, lines 49-77
async def _run_workflow_with_feedback(handle):
    while True:
        await asyncio.sleep(0.1)    # real-time sleep in the test process

        # query() doesn't trigger time-skipping — clock stays frozen
        progress = await handle.query(JurorAnalysisWorkflow.get_progress)

        # Send signals for any steps waiting for feedback
        for step_id in progress.pending_feedback_steps:
            await handle.signal(
                JurorAnalysisWorkflow.submit_step_feedback,
                StepFeedbackSignal(release_step_id=step_id),
            )

        # Only call result() AFTER the workflow reports it's done
        if progress.status == WorkflowStatus.COMPLETED:
            return await handle.result()  # returns instantly — nothing to skip

Why this works:

query() doesn’t touch the clock. The workflow stays frozen at its wait_condition. No fast-forwarding.
Our polling loop runs in real time. asyncio.sleep(0.1) is a real sleep in the test process — Temporal doesn’t control it.
Signals are delivered immediately regardless of what Temporal’s clock says.
By the time we call result(), the workflow is already complete. There’s nothing to fast-forward to. It returns instantly.

The workflow’s clock never jumps because we never gave the server permission to jump it.

Takeaways

A few rules of thumb I’m keeping in my back pocket:

The mental model:

Temporal is a timekeeper, not an execution engine. It manages a timetable of timers, timeouts, and activity deadlines.
Time-skipping fast-forwards idle time on Temporal’s clock. It doesn’t speed up your code.
If there’s nothing on the timetable to skip to, there’s nothing to skip.

The practical rules:

Never await handle.result() on a workflow that’s waiting for external signals. It gives Temporal permission to fast-forward, and the workflow will die before your signals arrive.
Use handle.query() to poll workflow state. Queries are passive reads — they don’t affect the clock.
Signals work immediately in any time mode. They don’t care what the server clock says.
Only call handle.result() after you’ve confirmed the workflow is done via query().

When to use which test environment:

start_time_skipping() — great for workflows that are self-contained (just timers and activities, no external interaction). Also works for signal-based workflows if you use the query-based polling pattern.
start_local() — safer for workflows that require signals/queries during execution, since it runs in real-time. But slower, because timers actually wait.

We kept start_time_skipping() because the query-based pattern works correctly, and we get free speedup on any timer-based operations (like activity retry intervals and jitter sleeps).

Disclaimer: Written by Human, improved using AI where applicable.

Cursor Is Brilliant—And Here's Why I Still Hate Using It

Fri, 19 Dec 2025 00:00:00 GMT

Let me tell you about the $600 burnout.

Two-thirds through the month, I’d already exhausted my Cursor quota. Not because I was being careless—I was just doing my job. Eight hours a day of building features, reviewing code, tweaking implementations. The kind of work that requires a senior engineer who cares about type-strict Python, modular architecture, and maintainable systems.

I sat there, staring at the “upgrade to pay-as-you-go” prompt, feeling something I hadn’t expected: exhaustion mixed with resentment. Cursor is brilliant. Its retrieval system finds scattered dependencies across your codebase like magic. Its context understanding makes other tools feel prehistoric. It rarely fails.

And yet, in that moment, I realized I’d been fighting it for weeks.

This isn’t a typical tool review. I’m not here to crown a winner or tell you what to buy. I’m a full-stack engineer running an AI startup, with a master’s in ML and strong opinions about code quality. I need tools that respect my judgment, not override it. What I learned using Cursor, Windsurf, and Claude Code for months revealed something more interesting than “which is best”—it exposed two fundamentally different philosophies about what AI-assisted development should be.

The Problem Nobody Talks About: Cognitive Predictability

Here’s what every review misses about pricing.

Cursor shows you token burn in real-time. Seems transparent, right? Except I had no idea what it actually meant. How much does a refactor cost? A feature implementation? A complex architectural change? The numbers were visible but meaningless.

I’d hesitate before exploring ideas. “Should I ask it to try this approach, or will that burn through my quota?” That hesitation—that’s the real cost. Not the dollars, but the cognitive overhead of constantly calculating whether a thought is “expensive enough” to pursue.

Then I tried Windsurf again.

Their pricing model is almost boring in its clarity. Every model has an explicit credit multiplier. Opus uses 3x credits. The million-token Sonnet? 10x credits. You know before you ask whether something will be expensive. Within a week, I’d internalized the pattern: roughly $10 per day of normal development work.

I set up auto top-up and stopped thinking about it entirely.

The psychological difference was striking. Not because Windsurf was cheaper—at similar usage intensity, they cost about the same. But because I could predict it. My brain could offload that worry and focus on the actual work. Cursor’s opacity created a constant background anxiety that I didn’t even notice until it was gone.

Opaque pricing doesn’t just affect your budget. It affects your thinking.

When Smart Tools Fight Smart Developers

I need to be precise about something: Cursor rarely makes mistakes. Its code is usually correct. Its suggestions are often clever. This isn’t about technical failure—it’s about something more subtle.

Let me show you what I mean.

My entire codebase uses single-line docstrings. Every function, every class—just a concise description of what it does. I’m strict about type hints at the function signature level, no type decorations cluttering the docstrings. When I use Pydantic models, that’s the pattern everywhere. The codebase is screaming its conventions at you.

Windsurf sees this and follows it. New code matches existing patterns. If the codebase uses Pydantic, the new feature uses Pydantic. If docstrings are single-line, they stay single-line. It treats your codebase as the source of truth.

Cursor sees this and… ignores it.

Unless I explicitly prompt “use single-line docstrings,” it generates multi-line ones. Even when literally every surrounding function uses single-line format. It wants to write tests. It wants to generate summary documentation. It has opinions about how code should look, and those opinions override what your project is telling it.

You can fight this with prompting. Every time I carefully articulate what I want to build, I also have to remember to say: “Follow the existing code style. Use single-line docstrings. Use Pydantic. Match the type hint patterns.” It’s like having a brilliant colleague who needs to be constantly reminded that they’re not working on their own project.

This creates an exhausting loop. I word my intent carefully, expecting a thoughtful proposal. Instead, Cursor immediately jumps to code editing, applying its global style defaults. I have to stop it, reassert constraints that should be obvious from the codebase, and restart. The tool itself becomes part of my cognitive load.

Here’s what makes this particularly frustrating: this behavior makes sense for beginners. If you’re learning, opinionated defaults teach good practices. The problem is that Cursor optimizes for the median developer—someone who needs guardrails. For senior engineers with established patterns and strong preferences, those same guardrails become chains.

I don’t want my tools to have opinions about my code style. I want them to learn mine.

The One Thing Cursor Actually Gets Right

I need to be fair here, because Cursor’s retrieval system is genuinely impressive.

When you ask Cursor a question about your codebase, it shows you its search in action. That “searching…” indicator isn’t just UI polish—it’s reflecting a powerful RAG system finding relevant context. It’s explicit, visual, and frequent. You can almost watch it think.

This makes Cursor exceptional at project-wide planning. Need to implement a feature that touches multiple services? Cursor finds the scattered pieces, understands the dependencies, and gives you a coherent proposal. It’s like having a colleague who’s memorized your entire codebase.

Windsurf has “Quick Context,” but it’s rarely invoked automatically. You have to explicitly ask for it, and even then, it feels more like keyword search than semantic retrieval. Fast, yes. But less powerful. Sometimes it traces files instead of actually searching for concepts.

The frustrating part? Cursor’s strength is also its weakness. Because the search is so opinionated and automatic, you can’t always tell when to trust it. I’ve had Cursor confidently delete code because its search didn’t find a feature—even though that feature existed and was important. When I catch this happening, I have to tell it: “Read the whole file, don’t search.” But sometimes it searches anyway.

This perfectly captures Cursor’s philosophy. The system is smart enough to be useful and opinionated enough to override you. For exploration and architecture planning, that’s valuable. For careful implementation of critical code, it’s dangerous.

The Mental Load You Don’t Notice Until It’s Gone

There’s a specific kind of frustration that’s hard to articulate. Not anger—more like background static.

With Cursor, I’m always aware I’m using Cursor. Will this prompt be expensive? Did it follow my style? Is it about to apply its defaults? Should I stop it before it generates tests I didn’t ask for?

With Windsurf, I think about the code.

That’s it. That’s the difference.

Windsurf occasionally hangs on tool calls—running multiple commands in parallel sometimes causes the first few to never stop. That’s a real bug, purely in their execution engine. But you know what? It doesn’t frustrate me the same way Cursor’s behavior does.

Because when Windsurf fails, it’s failing while trying to do what I asked. When Cursor frustrates me, it’s succeeding while doing something I didn’t ask for.

There’s a principle here: reliability without respect isn’t helpful. Cursor is more reliable technically—I rarely see it actually crash or error out. But Windsurf respects my intent more deeply. Given the choice between a tool that works perfectly but fights me constantly, and a tool that occasionally stumbles but fundamentally aligns with my goals… I’ll take the latter.

What Actually Matters: The Model, Not the Wrapper

Here’s something that took me months to realize: code quality comes from the model, not the IDE.

Claude Opus generates excellent code. Clear structure, proper types, handles edge cases well. Usually one or two iterations to get exactly what you want—assuming your prompt is clear. (Nothing can save you from bad prompting, by the way.)

Claude Sonnet is faster but noticeably weaker on complex type systems. It struggles with Python generics. Makes more syntax errors. For planning and proposals, it’s fine. For implementation of type-strict, high-quality code, it’s inadequate.

The IDE’s job should be simple: stay out of the model’s way. Let Opus be Opus. Let Sonnet be Sonnet. Provide context, handle tooling, then step back.

Cursor doesn’t step back. It wraps the model in layers of prompt orchestration. Forces conventions, generates extra artifacts, enforces style. Sometimes this improves the output for beginners. For experts, it degrades what the model could have produced on its own.

This is why I find myself switching back to Windsurf when I need serious, production-quality code. Not because Windsurf is smarter—it’s not. But because it trusts the model enough to mostly leave it alone. I get closer to what Claude Opus is actually capable of, without an IDE’s opinions interfering.

Why I Use Both (And Why You Might Too)

After months of frustration, experimentation, and honest assessment, here’s what I’ve learned:

I use Cursor for exploration. When I’m starting a new feature, trying to understand how different parts of the codebase connect, planning architecture—Cursor’s RAG system is unmatched. It’s worth the mental overhead for that initial phase. I get a clear map of what needs to change and why.

Then I switch to Windsurf for implementation. When I know what needs to be built and I need it done right—clean types, consistent style, modular structure—Windsurf respects my codebase’s patterns. The code it generates fits naturally into what already exists.

And when I’ve burned through quotas on both? Claude Code becomes the fallback. All-or-nothing interaction, but at least it’s thinking is sharp. I can’t edit incrementally, but for architectural decisions or complex reasoning, it still delivers.

This isn’t indecision. It’s pragmatism. Different tools for different phases of work.

The uncomfortable truth is that neither Cursor nor Windsurf is “better.” They represent fundamentally different philosophies about what AI assistance should be. Cursor is productized intelligence—heavily orchestrated, optimized for consistency across millions of users, opinionated by design. Windsurf is model-first intelligence—thinner abstraction, codebase-driven behavior, fewer guardrails.

If you’re a beginner or working on unfamiliar territory, Cursor’s opinions help you. If you’re a senior engineer with strong preferences and established patterns, those same opinions fight you.

The IDE I Actually Want to Use in Five Years

Right now, we’re in the hype phase. Everyone’s building “AI agents” and treating prompts like trade secrets. Cursor hides its orchestration entirely. Windsurf is more transparent but still opaque. We’re supposed to trust these black boxes because AI is magic and mysterious.

But the hype will fade. It always does. And when developers start caring more about productivity than novelty, something will shift.

The winning IDE won’t be the smartest. It’ll be the most honest.

I want an IDE that completely exposes its agent prompts. Not just makes them accessible if you hack around—actually shows them to you. In the UI. As first-class configuration. “Here’s what I’m telling the model. Here’s the context I’m injecting. Here’s the style guidance I’m applying. Change any of it.”

Full customizability. Per-project preferences. Per-user behavior profiles. Even AI-assisted prompt tuning—let me tell the IDE “be more like Windsurf” and have it adjust its orchestration accordingly.

This isn’t radical. It’s just taking developers seriously. We’re the users. We understand prompts. We can handle complexity. Stop treating us like consumers who need hand-holding and start treating us like engineers who need control.

The best AI IDE won’t have opinions. It will learn yours.

What This Means for You

If you’re evaluating these tools, here’s what actually matters:

Choose Cursor if:

You’re learning or working in unfamiliar domains
You need powerful project-wide context and planning
You’re okay with opinionated defaults and can work within them
You have predictable, moderate usage patterns

Choose Windsurf if:

You’re a senior engineer with strong style preferences
You prioritize code quality and codebase consistency
You want transparent, predictable pricing
You’re willing to manually invoke context when needed

Choose both if:

You do complex, multi-phase development work
You can afford the cognitive overhead of switching
You value exploration (Cursor) and implementation (Windsurf) differently

But here’s the deeper insight: don’t evaluate AI IDEs like you evaluate programming languages or frameworks. Evaluate them like you evaluate working relationships. Does this tool respect your expertise? Does it impose cognitive overhead or reduce it? Does it amplify your thinking or fight your intent?

Because we’re not just choosing tools anymore. We’re choosing collaborators. And the best collaborators aren’t the smartest—they’re the ones who know when to lead and when to follow.

I still use Cursor. I still get frustrated by it. I still pay the bills.

But I know now what I’m paying for: powerful intelligence that comes with the cost of constant negotiation. Some days, that’s worth it. Other days, I need a tool that just… builds what I asked for, the way I asked for it.

The future of AI-assisted development isn’t about building smarter agents. It’s about building more honest ones. Tools that expose their assumptions, respect our preferences, and understand that control isn’t the opposite of intelligence—it’s a requirement for trust.

Until then, I’ll keep both IDEs open. And keep fighting for the code I actually want to write.

(Written by Human, improved using AI where applicable.)

Why AI Cannot Engineer

Thu, 20 Nov 2025 00:00:00 GMT

I asked Claude to build me a customer dashboard last month. Ten minutes later, I had gorgeous React components, a clean API structure, and authentication that worked flawlessly on my laptop. I felt like a wizard.

Two days later, when we put it in front of actual users, everything fell apart. The API crumbled under concurrent requests. The authentication tokens expired in the middle of user sessions. The database queries that felt snappy with my 10 test records took 30 seconds with real data.

The code looked beautiful. It even worked—sort of. But it was a disaster waiting to happen, and I had no one to blame but myself. I’d asked for a construction worker when I needed an architect.

The Skyscraper Nobody Can Live In

Think about building a skyscraper for a moment. You need two completely different types of expertise:

The Architects and Engineers design the building. They figure out where the load-bearing walls go, how to route all the electrical wiring and plumbing, what materials can handle the wind shear at that height, how the building responds to earthquakes. They think about fire escapes and elevator shafts and how people will actually move through the space. They create detailed specifications for everything.

The Construction Workers build it. They know how to mix concrete, how to weld steel beams, how to install windows. Give them a good blueprint with all the specifications—“use this gauge of rebar, space them 6 inches apart, let the concrete cure for 48 hours”—and they’ll build you something solid and safe.

Here’s the interesting part: construction workers understand building principles. They know that rebar makes concrete stronger, that you can’t just drill holes wherever you want in a load-bearing wall, that certain materials expand in heat. They might not be able to calculate the exact compression loads or explain the physics, but they know how to build.

So in theory, if you asked construction workers to build a skyscraper without giving them architectural plans, they could do it. They’d use their knowledge to put up walls and install wiring and plumbing. The building would stand.

For a while.

Then someone tries to actually live in it. Or an earthquake hits. Or there’s a fire. And you realize the electrical system is a tangled mess that can’t handle the load. The plumbing creates massive pressure problems on the upper floors. There are no proper fire exits. The structure can’t flex with the wind. It looks like a building, but it’s not safe, not livable, not reliable. It’s a expensive disaster wrapped in a pretty facade.

You really needed those architects and engineers.

We Accidentally Merged Two Jobs

Now let’s talk about software. In our industry, we have the same two roles—but we’ve gotten confused about which is which.

Software Engineers are supposed to design systems. They think about scalability, reliability, security, maintainability. They establish patterns and guardrails. They consider edge cases: What happens when 10,000 users hit this endpoint simultaneously? How do we handle partial failures? What’s our data consistency model? How do we make this debuggable when something goes wrong at 3am? They create the “blueprints” that make the difference between software that works on localhost and software that works in production.

Programmers write code. They understand syntax, algorithms, design patterns. They know how to implement features, how to make things work. Given good specifications and architectural guidelines, they write solid, maintainable code that does exactly what it needs to do.

But here’s where we got confused: engineers also write code. They have to—you can’t design a system well if you don’t understand implementation realities. An architect who doesn’t understand how concrete and steel actually behave can’t design a building that stands. Similarly, a software engineer needs to understand how code actually executes.

Because of this overlap, we started treating them as the same job. Most companies just have “software engineers” who do everything—system design and implementation. We stopped distinguishing between the person who decides “we need a caching layer with these invalidation rules” and the person who implements Redis with that specification.

For years, this worked okay. The engineer-programmer would design the system in their head, then implement it. Both roles were happening, just inside one person.

Enter the AI Programmer

Now we have large language models that can write code. And here’s what I’ve realized: LLMs are incredible programmers, but terrible engineers.

An LLM is like that construction worker who can build you a skyscraper without blueprints. Ask it to “build me a todo list app,” and it will. The code will be clean. It’ll have proper separation of concerns. The variable names will make sense. It’ll work beautifully on your laptop.

But it’s missing all the engineering:

No load considerations: It’ll use an in-memory array for the todo list, which works great for testing but fails when you have 10,000 users
No failure handling: What happens when the database connection drops mid-request? The LLM’s code probably doesn’t handle that gracefully
No security thinking: Authentication might be present, but is it actually secure against common attacks? Are there SQL injection vulnerabilities?
No performance engineering: That nested loop through every user’s todos on every request? Didn’t consider that case
No operational thinking: Good luck debugging when something goes wrong—there’s minimal logging and no monitoring

The code looks professional. It follows good practices. It even works, in the ideal case. But the moment reality hits—concurrent users, network failures, edge cases, malicious input—it falls apart.

Just like that skyscraper built without architects.

What This Actually Means

I used to think the distinction between “engineer” and “programmer” was just semantic nitpicking. Now I realize it’s crucial, and we’re about to learn this lesson the hard way.

When you tell an LLM “build me an authentication system,” you’re giving it a task without constraints. It’s like telling a construction worker “build me a building” without specifying:

How many people it needs to support
What kind of disasters it needs to withstand
What the fire safety requirements are
How the utilities should be routed
What the budget constraints are

The LLM will build you something. That something might even impress you. But it won’t be robust, because nobody did the engineering.

Here’s a real example from my startup, Empath Legal. We’re building jury selection tools for attorneys. I could ask an LLM: “Create an API endpoint that analyzes jury questionnaires and returns insights.”

The LLM would build me something functional:

It would parse the questionnaires
It would call an LLM to analyze the text
It would return formatted results
The code would be clean and well-structured

But here’s what it wouldn’t consider without engineering input:

Business requirements I need to specify:

These questionnaires can be 50+ pages and attorneys need results in under 30 seconds
Our customers are lawyers who need citations for everything—we can’t just hallucinate insights
The analysis must be deterministic enough that an attorney can rely on it in court
We have a credit-based pricing model, so we need to track and limit usage

Technical guardrails I need to establish:

We need to chunk large questionnaires intelligently (by section, not arbitrary token counts)
We must implement streaming so attorneys see progressive results, not a 30-second blank screen
We need a caching layer so re-analyzing the same questionnaire doesn’t burn credits
We need robust error handling because LLM API calls fail sometimes
We need audit trails for compliance reasons

System design decisions I need to make:

Should this be synchronous or async? (Async, because of the 30-second requirement)
What happens if the LLM call fails midway? (We need idempotent retries)
How do we handle rate limits from the LLM provider? (We need a queue with backpressure)
What’s our data consistency model? (We need to decide what happens if analysis partially completes)

The LLM could write beautiful code for the happy path. But without me doing the engineering work—understanding the requirements, establishing the constraints, designing the system architecture—that code would be worthless in production.

The New Division of Labor

This isn’t about LLMs being bad. They’re phenomenal at what they do. This is about understanding what they do.

LLMs are expert programmers: Give them a well-specified task with clear constraints, and they’ll implement it beautifully. They know patterns, they write clean code, they can even debug and optimize within a defined scope.

Humans need to be the engineers: We need to understand the business requirements, design the system architecture, establish the guardrails and principles, consider the edge cases and failure modes. We need to ask: What are we really trying to achieve? What could go wrong? How does this need to scale?

This is actually liberating, if you think about it. We can offload the tedious parts—writing boilerplate, implementing standard patterns, translating specifications into code—and focus on the interesting parts: understanding user needs, designing robust systems, making architectural tradeoffs.

But only if we actually do the engineering work. If we just prompt “build me X” without providing context, constraints, and architectural guidance, we get that beautiful skyscraper that nobody can live in.

How to Actually Work with AI Coding Tools

Here’s what I’ve learned about working effectively with LLMs:

Bad prompt:

“Build me a payment system for my SaaS app”

Good prompt:

“Build me a payment system with these requirements:

Support credit-based pricing (users buy credits, features consume credits)

Must handle concurrent credit deductions safely (two API calls shouldn’t both succeed if there’s only enough credits for one)

Need audit trail for billing disputes

Should fail gracefully if payment processor is down (degrade to cached credit counts, log for reconciliation)

Performance requirement: credit check must add < 50ms to request latency

Use PostgreSQL with row-level locking for credit deductions. Implement idempotent credit operations so retries don’t double-charge. Include comprehensive logging for debugging billing issues.”

The second prompt does the engineering work. It specifies:

The business model (credit-based pricing)
The critical constraint (safe concurrent access)
The compliance requirement (audit trail)
The reliability requirement (graceful degradation)
The performance requirement (latency budget)
The technical approach (PostgreSQL with row locks)
The operational requirement (debuggable)

Now the LLM can write code that actually works in production, because I’ve given it the architectural blueprint.

The Uncomfortable Truth

Most of us who’ve been writing code for years have been doing both roles simultaneously for so long that we don’t even notice when we’re switching hats. When we sit down to “code,” we’re actually doing engineering work (thinking about requirements and constraints) and then programming work (implementing the solution).

With LLMs, we can’t do that anymore. The programming part gets delegated to the AI. But if we don’t explicitly do the engineering part ourselves—if we just treat the LLM as a magic “build me X” box—we get code that works in the demo but fails in reality.

And here’s the uncomfortable part: many people who call themselves engineers are actually just programmers. They’re really good at writing code, but they’ve never had to think deeply about system design because they were implementing someone else’s architecture, or working on small enough systems that the complexity didn’t matter.

With LLMs, this distinction now matters. The code-writing skills that used to be valuable are being commoditized. What’s becoming valuable is the engineering thinking: understanding requirements, designing robust systems, making good architectural tradeoffs, considering failure modes.

What This Means For You

If you’re learning to code right now, don’t just learn syntax and frameworks. Learn to think like an engineer:

When you build something, ask: “What happens when 1000 people use this simultaneously?”
Before writing code, ask: “What are all the ways this could fail, and how should it handle each one?”
Study real-world system design, not just toy examples
Learn to write specifications that would let someone else implement your vision correctly
Understand the why behind patterns, not just the how

If you’re already working as a developer, pay attention to which role you’re actually performing. Are you mostly implementing features based on specs? You’re in the programmer role, and LLMs are coming for that job. Start developing engineering skills: system design, requirements analysis, architectural thinking.

If you’re hiring, stop asking “can you code fizzbuzz?” and start asking “design a URL shortener that handles 1000 requests/second and explain your failure modes.” The code-writing part is increasingly automated. The thinking part isn’t.

The Punchline

Here’s the irony: I wrote this post with Claude’s help. Not because I couldn’t write it myself, but because Claude is a better programmer than I am. I gave it my core idea, my architectural guidelines (the blog writing guide), my requirements (make it engaging and human), and my constraints (use the construction analogy, include real examples).

Claude wrote beautiful prose. But I did the engineering: I decided what to communicate, how to structure the argument, what examples would land, what the reader needs to understand.

The future of software development isn’t “no human involved.” It’s humans doing what humans are uniquely good at—understanding messy real-world requirements, making tradeoffs with incomplete information, designing robust systems for complex environments—while AI does what it’s uniquely good at: writing clean, correct code that implements those specifications.

But only if we remember to do the engineering work. Otherwise, we’re just building skyscrapers without architects.

And we all know how that story ends.

What’s your experience with AI coding tools? Are you finding yourself doing more engineering thinking, or just prompting and hoping? I’d love to hear your stories—the spectacular failures are often more educational than the successes.