The parts of programming that did not get automated

A developer on our team spent an afternoon getting a bug-report form to file into a client's ticketing system. It worked. The agent filed three tickets, then queried the system and read every field back: title, description, steps to reproduce, the reproducibility dropdown. Each one landed correctly. It reported success, and the report was true.

The feature was still broken.

The form was creating tickets as the wrong type. That type's workflow marks an item resolved the moment it arrives, so every incoming bug report landed pre-closed, invisible to anyone filtering for open work. The agent had noticed the symptom. It wrote that a newly filed bug sitting in a resolved status would not show up in open-work filters. Then it attributed the cause to the client's configuration and suggested raising it with them.

The client had configured nothing wrong. The issue type was ours, in our code.

Nothing here was a coding problem

Every line the agent wrote was correct. Every check it ran passed, and the passes were real. If you had graded that session on the thing we spent decades grading people on, whether the code works, it would have scored full marks.

What was missing was that nobody doubted "it works."

Doubt of that kind is the same instinct that made a good engineer in 1998, and it matters just as much now that a machine does the typing. The list of things that make someone good at this job barely moved. What moved is that the one item we always taught first and hired for is now the item an agent does well. Whole interview formats exist to measure that one item, and they are still being run.

Five things did not get automated. "Critical thinking" is not a teachable unit; these are. The examples below are all ours: one junior developer's afternoon, and two months of transcripts from the senior engineer working the same codebase.

The five

Breaking a problem into parts you can be wrong about separately. Not for tidiness: so that when something is wrong, you can tell which part. The senior version of this reads, in our transcripts, as an instruction sent before any work started: "Do phase 1 only: bump every place the version is pinned, and tell me what a v2 migration would cost here before doing it." Two pieces, one of them deferred, and a question attached to the deferral. Nothing stops an agent from working this way. Someone has to ask it to.

Trying to prove your own answer wrong. The strongest example in two months of transcripts is eleven words, typed at three in the morning and never edited:

you can check if it does. i doubt it. but check.

Two instructions there: test this, and here is what I already believe. An agent asked "how should we do this" produces something reasonable and then agrees with itself about it. An agent handed a belief and told to test it has something to push against. The output is completely different. The difference is that a human held an opinion first.

Not accepting the first plausible answer. A separate reflex from the last one, and it fires when the answer arrives rather than before. "what? that makes no sense, every sprint before this, we've been squash merging and all the details were in the PR description. why would we change that now? explain please." The person asking does not know the answer. They noticed it contradicted something and said so instead of moving on.

Saying what you expect before the work starts. The best example of this one in our material came from the junior developer, not the senior. Before touching anything she wrote: "all i need to verify is that on localhost i go to report a bug and submit and it gets submitted in jira." A clean acceptance criterion, written in advance, in plain language. She already had this one.

Knowing where the checking stops. An agent verifies the layer it can reach. In the ticket story that layer was whether the interface accepted the payload and whether the values arrived intact. Both true, both the wrong question. Whether a bug report is the right kind of object to create at all is a decision about what the feature is for, and no amount of reading the response back will surface it. The ceiling moves with the task. An agent that can run the test suite will confirm the tests pass, which says nothing about whether the tests test the right thing.

The friction was the curriculum

Here is the part I did not expect.

Nobody ever sat me down and taught me the first one. What taught me was a four-hundred-line function, written at one in the morning, that I could not debug. I felt that for two days. Doubting my own answers came from the compiler, which humiliated me daily for a decade. And where the checking stops: I shipped something that looked fine, it was not fine, and a person I respected had to tell me.

None of that was mentoring. It was friction, applied consistently for years, and the skills precipitated out of it. We called it "experience" and treated it as something that happens to people rather than something anyone delivers.

A developer four months into the job today can produce a working integration into a live client system in an afternoon. The typing is gone, and with it most of the small, daily humiliation that used to do the installing. The compiler has not gone quiet, either. It complains to the agent now, and the agent reads the error, fixes it, and moves on, all before a person ever sees the message. The lesson is still delivered daily. It just lands on something that does not need it.

They are not being taught less carefully than we were. They are being taught exactly as carelessly as we were, which worked fine when the job supplied the lesson for free and does not work at all now.

The five skills survived automation. The thing that used to install them did not.

So now someone has to teach them on purpose, which nobody had to bother doing before.

What deliberate looks like

The instinct is to write a checklist. We tried. A checklist becomes the fortieth unread document in a repository within a week.

The second instinct is a review gate, and we were tempted. It would not have caught this bug. A gate filters what it can measure, and everything measurable here was green: the ticket type was wrong in a way that every automated check we own would have called correct, which is the entire point of the story.

What has worked is smaller and more annoying than either.

Ask for the ticket back in their own words, before any code exists. Not "do you understand," which everyone answers yes to. "Say what this is asking for, and who it is for." If they cannot say it, that is the finding, and it arrives before anything has been built on top of the misunderstanding. This one is free and almost nobody does it.

Ask for a prediction before anything slow or risky runs. "What do you think happens? Guess." Then compare out loud. Being wrong at this moment costs nothing and is the fastest way to find the hole in someone's model of the system. It is also the bottom rung of proving your own answer wrong, which is otherwise the hardest of the five to teach directly. Telling a developer four months in to state their architectural hypothesis just makes them feel behind. Asking them to guess what a command will print is the same skill at a size they can hold.

Send them to look at the result where a real person would see it. Not the terminal output, not the agent's summary. Open the queue the way the client will open it. That check takes forty seconds, needs no expertise in the system, and is the only one that would have caught the broken tickets.

Make the agent do the asking. This is the piece I would not have predicted. We put the sequence into a skill file that runs on the junior developer's machine, written as instructions to the agent rather than to her. It now asks her to explain the ticket back before it starts building. It asks what she expects before it runs something slow. It will not merge anything unless she says so for that specific action, and an approval buried in a sentence about something else does not count. It says which layer it could not verify and where she should go look.

A document teaches when someone reads it. The agent is already in the room at the exact moment each of those questions matters, which no document and no senior developer can be.

The mechanics of that turned out to matter more than its content, and most of them are restraint. One question at a time, because a list of six is paralyzing. Ask, then actually wait, rather than asking and answering yourself in the same breath. Do not accept "yeah, makes sense." After two genuine attempts, stop being Socratic and tell her, because productive struggle is the point and being stranded is only demoralizing. And she can say "just do it" at any moment, at which point it complies immediately and mentions once what it skipped. A teaching tool that cannot be switched off gets routed around inside a week.

The part that does not delegate

The five skills are all versions of the same underlying job, which is deciding what correct means and then finding out whether you got it.

An agent cannot hold that. It will tell you, accurately and in good faith, that the interface accepted the payload and the fields came back matching. Whether a bug report that arrives already resolved is a useful object in the world is a question about the world, and it belongs to whoever understands what the thing is for.

That is the part worth mentoring now, and it is not downstream of writing code. It never was. We just used to get it thrown in.