Somebody at a city public works department wants to know how many blocks were missed on a particular route last week, broken out by the reason the crew logged.
That is not an exotic question. It is the kind of question a department asks itself constantly, and until recently the answer took a person, a spreadsheet, and most of a morning, because it lives across a work-order system, a fleet telemetry feed, and a route geometry that a third system owns.
We put a chat box on it. The chat box is the least interesting thing in this article.
What was already underneath
Before any of this, the department had spent about a decade on something with no demo value whatsoever.
A dozen separately procured agency systems, none of which had ever been asked for an API, sitting behind one enterprise service bus. Around a hundred million telemetry readings off a two-hundred-vehicle fleet, flowing in near real time. Route geometry, so a resident's address resolves against the shape the department actually plows or sweeps. All of it landing in one warehouse that can be queried as a single picture of the operation.
That work took four administrations and a pandemic to complete, and at no point in it was there anything you could put on a screen and impress somebody with. Weekly reporting went from most of a day to about fifteen minutes, which is the kind of outcome that gets a line in a status report and no applause.
It is also the entire reason a natural-language layer is possible here. A question in English is only answerable if there is one place where the answer exists.
The chat layer is two hundred lines
Once the warehouse exists, the interface is genuinely small: a chat surface, a model with tool use, and one tool that runs a read-only query against the warehouse and hands the rows back.
The loop is unremarkable. The question goes to the model along with the schema. The model calls the query tool. The rows come back. The model reads them and answers in a sentence, with the numbers in it.
We hand-rolled the API calls and the tool-call loop rather than adopting a framework. Two were on the table: one that would have added around thirteen dependencies and given us cleaner provider abstraction and streaming, and a full agent framework that would have added closer to forty and managed the whole loop, the schemas, and the process lifecycle.
For a proof of concept, neither earned its cost. The hand-rolled version ships in hours, has zero new dependencies, and can be debugged line by line by whoever inherits it. The framework versions are better bets for a long-lived production surface with several features sharing the abstraction, and that is the honest rule: take the dependency when a second and third feature will use it, not when the first one might.
Show the query
The single most important detail: it displays the query it ran.
Not in a debug panel, not behind a toggle for administrators. On the answer, where the person reading it will see it.
This matters more in a government department than almost anywhere else. A number from this system might end up in a council briefing, a public records response, or a performance report. The person who puts it there is accountable for it in a way that a private-sector analyst usually is not, and "the AI said so" is not a defence that survives contact with anyone.
Showing the query converts the system from an oracle into an instrument. Someone who knows the data can look at it and see immediately that it counted the wrong week, or filtered on the wrong route, or silently dropped the rows with a null reason code. That reader is not a nuisance to be designed around. That reader is the safety mechanism.
There is a second effect we did not expect. People learn the data model by reading the queries. Two weeks in, some of them stop asking the chat box and just write the query, because now they can. That is a success, even though it looks like disengagement in any usage metric you would think to track.
What this means if you do not have the warehouse
This is the part to be blunt about, because it is where most of these projects go wrong.
If your data still lives in a dozen systems that do not talk to each other, a natural-language layer will not fix that. It will produce fluent answers derived from whichever fragment it can reach, and those answers will be confidently wrong in ways nobody catches, because the fluency of the response is completely uncorrelated with the completeness of the underlying data.
The order is not negotiable. The integration comes first. It is slower, it costs more, it photographs badly, and it is the thing that makes everything after it possible.
The good news is that the integration is worth doing whether or not you ever put a model on top. Every one of those decade-old warehouse queries still runs. The chat box is a convenience layer over an asset that was already paying for itself.
The general version
When somebody shows you a natural-language interface over their data, the question worth asking is not about the model, the prompt, or the framework.
It is: what does this thing query, and how did that get built.
If the answer is a real, integrated, maintained data spine, the interface on top is a couple of hundred lines and a good idea. If the answer is vague, you are looking at a demo, and the demo is the whole product.