Building an AI agent has never been easier. Getting one into production, where real people depend on it and real consequences follow its decisions, is a different problem.
Most teams we talk to have experimented with agents. Far fewer have moved them into meaningful production use where autonomous decisions are being made.
Over four executive dinners in Seattle and Chicago this year, our team has brought together over 55 leaders across technology, finance and operations with 8th Light practitioners to compare notes on what’s working, what’s getting stuck and what it actually takes to move beyond the pilot.
These five lessons kept surfacing.
1. Start with the harness, not just the agent
Frontier models get most of the attention. It’s a highly competitive space; there are splashy press releases, and every vendor on the planet is trying to win the same race. But the model isn't what makes an agent dependable.
Reliability comes from the system around it: deterministic rules for what must never vary, simulations that test behavior before it reaches users, trusted knowledge sources, human-verified facts, and clear boundaries around what the agent can do.
We call that system the harness.
Without it, teams can build impressive demos that fall apart when they encounter messy data, edge cases, or decisions with financial or regulatory consequences. The surrounding system, or harness, is the real goal because it provides fail-closed gates and observability an operations leader can understand in plain language. No engineering degree required.
💡Try this: List the decisions your agent makes that should produce a predictable result every time. Put those in a list of unbreakable rules. Reserve the model for the decisions that actually require inference or judgment.
2. Design governance in from day one
Governance isn't the review that happens just before launch as a checkbox activity. In reality, governance helps guide what you should build in the first place through risk-oriented guardrails.
For teams working with regulated or sensitive data, AI governance can determine which use case goes first. One engineering firm we met planned its AI rollout around CMMC Level 2 and audit readiness, intentionally keeping sensitive work out of phase one. A financial services company handling sensitive financial data expects AI projects to clear security, governance, and procurement review.
The practical pattern is living automation: let the agent do the work, require a person to approve actions with meaningful consequences, and maintain a trace of what happened. If something fails, it fails loud and closed. Nothing proceeds without a human review.
That way, when an auditor, customer, or your own team asks, “What did the agent do, and why?” you have an answer and can trace the action down to the timestamp.
💡Try this: Start with a use case you could comfortably explain to an auditor today. Build tracing and human approval into the system from the pilot onward, not after you've decided it works.
3. Choose the architecture before you choose the model
The best frontier model for today’s jobs-to-be-done will not be the best model six months from now. Innovation is moving far too fast to anchor on a release number.
If the rest of your system assumes one provider, every major improvement somewhere else becomes an expensive migration. A better approach is to set up evaluation criteria so the model is a replaceable part of the architecture.
Define how context enters the system, how tools are called, where permissions live, how outputs are evaluated, and how actions are traced. Then choose the model that best fits the job.
That flexibility matters for more than model performance. Privacy, cost, and deployment requirements are changing quickly too. As capable models become viable on local hardware, teams have more choices between self-hosted and managed options.
💡Try this: Draw your architecture with the model as a replaceable box. If swapping providers means rebuilding the surrounding system, you've found a tight coupling that is worth fixing.
4. Bring finance and operations in early
Some of the most compelling agentic use cases we heard weren't coming from engineering.
One firm is using an agent to review expense receipts against federal acquisition rules and HR policy, with an estimated 70% time savings. An industrial services company is looking at AI to accelerate SAP work. A healthcare technology company is applying AI across non-technical teams.
The common thread wasn't headcount reduction. It was operating leverage: reducing repetitive work, improving throughput, and creating more capacity without adding equivalent cost.
That's also how leaders were thinking about investment. One described a staged approach: a small discovery effort, a larger prototype, and then meaningful investment only once the path to production was clear.
💡Try this: Don't pitch an agent, pitch the business case. Quantify hours saved, throughput gained, margin impact, or another strategic metric finance already cares about. Then fund the work in stages, with a clear decision point between each one.
5. Start narrow, but build for what comes next
Your first production agentic workflow doesn't need to transform the company. It needs to solve one real problem and make the second agent easier to build.
That approach changes how you scope the work.
Start with a narrow workflow where the inputs, decisions, and outcomes are understandable. But build the underlying pieces so they can be reused: the harness, permissions, tracing, evaluations, knowledge architecture, and deployment patterns.
The goal isn't dependence on an outside team, either. One leader told us they wanted a creative problem-solving partner, “not staff augmentation.” Another wanted a framework their own team could iterate on, manage, and monitor, with outside expertise available for the hardest problems to solve.
A successful first agent should leave your organization with more than working software. It should leave you with a better way to build the next one.
💡Try this: Before you start, write down what agent #2 should inherit from agent #1. If the answer is nothing, rethink the process you are following to make the architecture modular before you build.
The gap isn't the agent
Technology is moving quickly, but getting agents into production isn't primarily a model problem. It's a systems problem.
Teams that make the jump are building the harness around the model, treating governance as part of the architecture, keeping their technology choices flexible, connecting the work to business outcomes, and creating capabilities they can reuse.
That's what turns an interesting agent into infrastructure a business can actually rely on.
Keep the conversation going
These lessons came out of our executive dinner series, where leaders working through the same challenges compare notes with peers and practitioners who are putting agentic systems into production.
Our next two dinners are in Chicago and Denver. If you're working through these questions yourself, we'd love to have you at the table.
Or, if you want to start with your own workflows, our Harness Engineering Workshop is a focused way to apply these principles with your team in a live, half-day session.
