AI systems architecture
Most AI projects do not fail on the model. They fail on the shape of the system around it — where state lives, what happens when a tool errors, who approves an action, and what it costs at ten times the volume.
What this involves
Architecture here means the decisions that are expensive to reverse: whether a workflow or an agent fits the problem, where retrieval sits, how many models you call and in what order, what is cached, what is logged, and where a person intervenes. Those choices set your cost per request, your failure modes and your ceiling on accuracy long before any prompt is written.
We produce a design you can build against — component diagram, data flow, the interfaces between parts, the evaluation harness, and a written record of what was decided and what was rejected. You can hand it to your own team, to us, or to a third party.
- Workflow versus agent, decided on the branching the task actually needs
- Retrieval topology: chunking, indexing, reranking and what gets cached
- Model routing — a small model for the common path, a large one for the exception
- Guardrails: step limits, spend caps, approval gates and structured tool errors
- Observability: traces, evaluation runs and the metric each release is judged on
- An exit path from every provider you depend on
Workflow or agent
This is the first decision and the one most often taken by default. If the sequence of steps is known in advance, write the sequence: a workflow with model calls at three fixed points is cheaper, faster, testable and debuggable. An agent earns its complexity only when the path genuinely varies with the input and the branching is too wide to enumerate.
Choosing the boring workflow is usually the senior decision. We will tell you when that is the answer, even though the agent build is the larger engagement.
Where the state lives
Conversation history, retrieved context, tool results and partial progress all have to live somewhere, and the choice decides what happens when a run is interrupted. A system that cannot resume loses the work; a system that stores everything indefinitely becomes a privacy problem and a bill.
We specify what is held, for how long, and what a resumed run replays — before the first endpoint is written, because retrofitting resumability means rewriting the control flow.
Designing for the day the model changes
Providers deprecate models, shift behaviour between versions and change prices. A design that assumes one model's exact behaviour is a design with an expiry date.
So the model sits behind an interface, prompts and their expected outputs live in version control, and the evaluation suite runs against any candidate model. Switching provider becomes a measured decision rather than an emergency, and you can prove the replacement is not worse before it ships.
Cost and latency as design constraints
Both are architectural, not something to tune afterwards. Routing the common case to a smaller model, caching what repeats, trimming retrieved context to what is actually used, and streaming the first token early are decisions about system shape.
We model cost per request at your expected volume during design, not after the first invoice. Where the numbers do not work, that is worth knowing in week one.
What you get
A component and data-flow diagram, the interface contracts between parts, the evaluation harness with its baseline and threshold, a failure-mode table naming what breaks and what happens when it does, and a cost model at three volumes.
Plus an architecture decision record: what was chosen, what was rejected, and why. That last document is what stops the same debate reopening in six months with nobody able to remember the reasoning.
Frequently asked questions
Can you review an architecture we already have?
Yes, and it is a common starting point. You get a written assessment of what is sound, what will break at volume, and what is expensive to leave as it is — with the fixes ranked by cost against risk rather than presented as one rewrite.
Do we need this if the build is small?
Usually not as a separate engagement. Below roughly four weeks of work the architecture is decided inside the build and written up at handover. It earns its place when several systems, several models or a compliance constraint are involved.
Will the design work if we build it ourselves?
That is the intent. The deliverable is written for an engineering team to implement without us, with the interfaces specified tightly enough that two people reading it build the same thing.
How long does it take?
One to three weeks depending on how many systems are in scope, ending in a document and a walkthrough with your engineers rather than a slide deck.
Tell us what you are building.
Send a short description of the problem and we will reply within one business day with an honest view of scope, cost and whether we are the right person for it.
Or email directly: contact@hire-ai-dev.com