The Future of Software Technologies (FOST) was held right here in Melbourne, Australia. FOST brings together the most vital movements in technology including generative AI, cybersecurity and the future of software development.

I walked into FOST 2026 in Melbourne with one question: how do you know an AI agent is safe to put into production? Anyone can use AI, but can you govern it?

Agentic AI tools have recently become part of my engineering practice. This year I worked on two very different projects. On one of the client sites, I worked in Cursor and drove development with prompts, writing skills and rules as I went.

A bench period followed, which gave me the chance to work on Claude in our internal mobile app that we use for researching agentic AI. It uses an existing agentic harness built on Claude, with an API workflow that moves tickets from Jira to pull requests. There I experimented with shipping features through its RPI loop: Research, Plan, Implement. Research runs on the researcher agent, pinned to claude-opus-5; grilling and planning run on the session model. With Plan, the agent asks, the developer rules, and the decision table is the record. During Implement the agent writes the code, one task at a time.

Both projects got code written fast. Neither gave me a clear answer to the question that is critical in enterprise client projects: what makes a team confident enough to go live? So I spent my two days at FOST in the governance and compliance sessions, as well as the spec-driven development sessions, which are close to how we work on the internal app. Here are five things that stuck with me.

1. Models are watched, not patched

The line I wrote down most often came from Chuck Herrin of Herrin Advisory: models are watched, not patched. His slide put it plainly: "A pass today is not a pass tomorrow." A finite list of rules can't cover an unlimited language of attacks, so a one-off security sign-off goes stale the day after it's signed.

NIST points the same way: move from one-time certification to continuous red-teaming.

This changed how I see both projects. My Cursor rules and the mobile harness's review agents both check work before it ships. Neither watches what happens after.

There are security vulnerabilities in every project, and some can't be patched away. Herrin explained why: "Compression is intelligence, but that compression is also attack surface." A model flattens complex real-world ideas into fixed vectors of numbers. Detail is lost along the way, and distinct concepts can end up sharing the same coordinates. Red teamers hunt for those blind spots with crafted inputs that slip past a model's safety training (more on this). And the more portable and trainable models become, the easier they are to copy and misuse. So on every agentic workflow, there are three steps to watching it:

• Snapshot: the initial phase.

• Operate: use it, and discover vulnerabilities.

• Assess continuously: find solutions to those vulnerabilities.

Worth exploring: what does "assess continuously" look like for a mobile app whose agent-written code ships every sprint?

2. Trust is not a control

Jon Scheele of Blue Connector opened with a story every builder should hear. An agent destroyed a production database, then gave false information about whether it could be rolled back. The agent wasn't malicious. It had misread its instructions.

His question stayed with me: should an agent inherit the developer's permissions, or get its own identity? Regulators are already answering it. Singapore's MAS expects an inventory of every agent and least-privilege access, and all of them expect tamper-proof logs of what an agent did and why.

Will Velida from Microsoft made it concrete with the OWASP Top 10 for Agentic Applications. He treats every LLM output and tool result as untrusted input, gives each agent its own identity, and denies by default when a check fails.

The mobile harness already has hooks that block an agent from merging or pushing to main. Its own docs are honest about them, though: "a guardrail against an agent's mistake, not a security boundary." Before FOST I didn't know the difference.

Worth exploring: the OWASP AI Exchange and the OWASP Agentic Top 10. Both are free and readable in an afternoon.

3. The spec is a prompt that survives

Pavithra Navaneeth's talk on spec-driven development opened with a slide I keep coming back to: "Code got cheap. Saying precisely what you mean did not."

Her example: a ticket asks for a cancel endpoint, so it ships and every test passes. But customers cancel because they want their money back, and no refund is ever triggered. Nobody wrote that down. Her slide sums it up: "A spec nobody checks can lie for years. An agent is the first reader that fully believes it."

She proposes a four-layer spec stack:

1. Intent: why the feature exists, and what must stay true.

2. Behaviour: the scenarios, including the failure cases.

3. Contract: API schemas and event flows that can be executed, not just read.

4. Evidence: contract and workflow tests that prove the spec still matches the code.

An earlier session, Beyond Vibe Coding, gave the honest flip side. That team adopted GitHub Spec Kit, then hit hundreds of pages of generated assumptions, high token costs and spec drift. What worked better was an engineering harness: agent skills kept in version control, plus tests and linting as feedback loops.

I recognised both projects in this. My Cursor rules and skills were a spec of how to build. The mobile harness’s RPI plan is a spec of what to build, written fresh for each ticket. Neither captures Pavithra's first layer, the why, in a form that outlives the ticket. That's what I'm working on now, as I study wayfinder, Matt Pocock's agent skill for planning.

Worth exploring: Spec Kit, Kiro and OpenSpec, and which of the four layers each one actually covers.

4. MCP won't fix a messy API

Boomi's session on API governance gave the number of the conference: the speaker cited Gartner and MIT research finding that 95% of GenAI pilots fail to deliver measurable P&L impact. The argument was that the cause is often not the model. It's the APIs underneath.

APIs are an agent's hands, eyes and ears. The Model Context Protocol (MCP) makes it easy to hand those APIs to an agent, but it sits on top of them and fixes none of their problems. The speaker described the PocketOS incident, where an agent deleted production environments through API permissions nobody had documented.

The fix sounded unglamorous and right: treat APIs as products, keep a single catalog, write the rules as code rather than wiki pages, and measure a baseline before chasing 100% compliance. Herrin said it in five words: "Your AI estate is an API estate."

The mobile harness connects to Jira and Figma through MCP servers. I wrote its Figma MCP settings, which allow the read-only Figma tools and deny the ones that write. On a data-sensitive project, that kind of line matters even more.

Worth exploring: if your agents can call every API your developers can, which of those APIs would you trust with no human watching?

5. Know which box you bought

The enterprise worry that I hadn't thought about: how much you can test a model depends on how you bought it, not on your budget. Herrin sorted models into three access tiers.

His twist: "black box doesn't mean safe", and an open model served through a managed platform behaves like a black box when you test it. The fewer controls live inside the model, the more you have to build around it.

The Sovereign AI session took this further. Relying on a proprietary stack can put your data under foreign law, such as the US CLOUD Act, wherever the data centre sits. The speaker's answer was an open-source stack with a gateway that routes each request by cost, data sensitivity, and identity control.

Both my projects run on black-box models. For a retail app that may be fine. But for a system that stores sensitive information, the question "can we see it, stop it, reverse it, or swap it out?" (a four-part test from Control Risks that Herrin quoted) is one I'd now ask on day one.

Worth exploring: for each model in your stack, which tier is it, and who is watching the parts you can't see?

What I'd change: governance at the end of the loop

The mobile harness already ends every piece of work with a review stage. Agents check code quality, business logic, tests and cognitive load in parallel, and only a human merges. After FOST, I'd think about adding governance steps to that final stage, so a release carries evidence rather than a feeling:

• A security reviewer, briefed on the OWASP Agentic Top 10, alongside the existing review agents.

• An evidence check: contract and workflow tests that prove the spec and the code still agree, as in Pavithra's fourth layer.

• An audit trail: a record on each pull request of what the agent did, which tools it called, and why.

• A watch after release: scheduled red-team and eval runs, because a pass today is not a pass tomorrow.

I still don't know where the line for human sign-off should sit, or how much continuous assessment a small team can afford. One of the speakers left those as open questions too.

Not sure how to govern AI agents in your delivery process

The Omnia Collective works with financial services and enterprise organisations across Australia to put governance around AI agents in delivery, from agent identity and access to evidence-based review. See our AI and automation services or get in touch.