There is a lot of noise about AI-assisted development right now. Not much of it comes from people actually building production systems with it. Here is what a year of using Claude day to day on real client work, across financial services and government builds, has taught us: the wins, the near-misses, and the parts we have learned to be paranoid about.

Where AI actually saves us time

The clearest wins are on scaffolding: the boilerplate that has to be right but is not the interesting part of the problem.

I set up a multi-agent orchestration pipeline: dozens of agents wired across parallel groups, dynamic routing, and looped child workflows, each with its own timeout, retry budget, and failure handling. Getting that structure right from scratch would have been two days of reading Temporal docs and debugging replay behaviour. Claude got the scaffolding right quickly, and I spent my time tuning the agent logic instead.

The activity options, retry policies, child workflow wiring: that would have taken most of the time, and it is the least interesting part of the problem. If Claude did nothing else valuable, this alone would justify it.

The near-miss that changed how I work

Not every suggestion is right. The wrong ones sound just as confident as the good ones.

The example I keep coming back to: Claude suggested reading from in-memory pipeline state (accumulated prior-agent outputs) for a value that should have come from the database instead. It looked fine on the surface.

Two agents running in parallel share that in-memory state, though. Agent B's output can bleed into agent A's context mid-run, and agent A produces its output based on the contaminated input. The result gets persisted and labelled as agent A's, which is accurate to what agent A said, but wrong, because what agent A saw was already corrupted. Nothing downstream would catch it unless you knew to check for cross-agent bleed.

I caught it. Corrected it. And immediately wrote it into the project instructions file so the next session started already knowing the rule. That is the real workflow. Not "Claude wrote it, I merged it." It is more like reviewing code from a very fast, very confident junior engineer who has no memory of any previous project decisions.

How we handle the fact that AI forgets

The model does not carry learned mistakes across sessions. That is a real problem when building anything non-trivial with it.

So we maintain a CLAUDE.md file checked into the repo. Every rule I have had to correct twice gets written in there as a permanent constraint. A couple of examples that are in there right now:

• Always read from the database for audit inputs, never from pipeline state.

• Parallel activities must opt out of shared context or they will contaminate each other.

It is basically converting conversation corrections into project documentation. The model reads it at the start of every session so I do not have to repeat myself. Do I write more code now, or less?

More, but faster.

The system I am currently building orchestrates dozens of AI agents, Snowflake integration, a Temporal workflow layer, and a FastAPI layer. I would not have attempted that scope in this timeframe. What changed is where the time goes: less on boilerplate, more on whether the design is actually right.

The real risk is that AI makes ambitious scope feel cheap. The first 80% goes fast. It is easy to commit to something large because the initial velocity is intoxicating. But the last 20% is still all on you, and if you have overcommitted, that is when things fall over.

Knowing if an AI output is actually correct, not just plausible

Building AI-powered systems is a different problem from using AI as a development tool. It is not just "did Claude give me good code." It is "is Claude giving good outputs in production, to a real user?"

The trick we have settled on: put constraints in code, not in prompts. Structured output schemas reject anything that does not match, like a form that will not submit unless every field is valid and the right type. For some agents, the list of valid answers is pulled from the live database at runtime, so the model literally cannot hallucinate an option that does not exist without triggering an automatic retry.

There is a ceiling, though. Schema validation catches structural errors, not factual ones. If the model confidently picks the wrong valid answer, that is harder to catch. We have designed an evaluator agent as the intended last line of defence for exactly this.

How far bad outputs travel before something catches them

A sequential agent in the chain receives all prior outputs in a shared context. A bad label from agent 1 flows into agent 2, then 3. Parallel agents are deliberately excluded from that shared context (that is the fix from the near-miss above), so they cannot contaminate each other, but they also do not get the benefit of seeing upstream corrections.

The evaluator agent is designed to catch it at the end. So if the evaluator misses it, the bad output ships. In practice, the most reliable catch is earlier: the output schemas doing form-validation-style rejection. If a model returns something structurally wrong (wrong field, invalid value), it gets re-prompted automatically up to four times before the schema rejects it entirely.

What we still don't give AI

Data contract decisions. Which table, which column, what constitutes the source of truth. The AI does not have live system context and will confidently pick a real, plausible-looking table that is simply the wrong one for the job. Nothing in the code stops it, because the query still runs and returns an answer.

Anything where being wrong is hard to reverse, or where the model cannot see the actual system state it is making decisions about, stays with humans.

If you're new to this

• Put constraints in code, not just in prompts. A structured output schema forces a retry when the model gets it wrong. A prompt instruction can be easily ignored or missed with no signal that it happened.

• Watch out for: do not let AI make architecture decisions for systems it cannot see. It will give a confident answer based on what is plausible in general, not what is true about your specific setup.

Thinking about AI in your development workflow

The Omnia Collective works with financial services, government and enterprise teams across Australia as certified Anthropic partners, building production AI systems and figuring out how AI-assisted development actually fits into a team's workflow. See our AI and automation services or get in touch.