Entrazure
“We ran a pilot. Everyone liked it. It is still a pilot.”
The pattern is consistent enough to be predictable. Someone builds a prototype in a few days, grounds it on a folder of documents copied by hand, and shows it to the leadership team. The reaction is good. Then it stops.
Six months later the same prototype runs on one person’s account. The reasons given for why it has not shipped are vague: security has concerns, we need to sort out the data, we are waiting on budget. Underneath the vagueness there are always the same four questions, and nobody can answer them.
Who is this agent allowed to see documents on behalf of? What does it cost per month at real volume rather than at demo volume? How often is it wrong, measured against what? And who owns it on the Monday after the person who built it changes role?
Those four questions are the entire distance between a demo and a system. They are also the reason the second pilot usually goes the same way as the first.
Why the usual answer fails
The usual answer is an AI accelerator: a fixed-price four to six week proof of concept, built on a vendor framework, delivered as a working demo and a roadmap deck. It fails for a specific and repeatable reason. It optimizes for the demo.
A demo corpus has no permissions on it. Documents are copied into a clean index, so every retrieval returns everything, and the agent looks impressively well informed. The moment that index is rebuilt against the real SharePoint estate, retrieval starts returning the salary review, the acquisition memo and the disciplinary file, because nothing in the pipeline filters results against the identity of the person asking. Security trimming is not a feature added late. It is a property of how the index is built, and retrofitting it usually means starting the retrieval layer again.
A demo is also evaluated by vibes. It is tested with the questions the builder already knows it handles. Nobody assembled a set of real cases with agreed correct answers, so there is no number to defend when a director asks how accurate it is. “It seemed good” does not survive a risk committee, and it should not.
The second usual answer is worse: turn on Microsoft 365 Copilot and call that the AI program. Copilot is genuinely useful, but it is a different thing with a different failure mode, and enabling it on an unaudited tenant creates an oversharing incident rather than an agent. That is a separate piece of work, covered in our approach to a Microsoft 365 Copilot rollout.
Third: the framework gets chosen before the use case. A vendor with an agent platform will recommend the agent platform. The decision that actually matters, Copilot Studio versus Azure AI Foundry, has a real answer that depends on where the data sits and how much control you need over networking, orchestration and evaluation. It should be made on the merits of one case, in writing, not inherited from whoever sold the workshop.
How we build an agent that stays in production
We build the evaluation set before we build the agent. Thirty to fifty real cases drawn from your own tickets, documents or decisions, each with an agreed correct answer. It takes about a week and it is the most durable artefact of the engagement, because it outlives the model you started on. Every change is measured against it. When accuracy moves, you know by how much.
Retrieval respects the permissions the user already has. We build the index in Azure AI Search with source permissions carried through, and every query filtered against the caller’s identity. Chunking strategy, hybrid search and reranking are tuned against the evaluation set rather than by intuition.
We pick the surface on the merits. Copilot Studio when the content lives in Graph and the task belongs inside Microsoft 365. Azure AI Foundry when the workload needs private networking on inference, custom orchestration, non-Microsoft sources or an evaluation harness in your own pipeline. Document Intelligence when the use case is extraction from invoices, forms, contracts or clinical documents, which is usually where payback is clearest because the manual baseline is already measured in hours.
The boundary is written down. What the agent may do, what it may not do, what it must escalate, and what it logs. Enforced in code and configuration, not implied by a prompt that a user can talk their way around. Security review on the design is led by our certified security architects, and the output is a document you can hand to an auditor.
It runs where the rest of your estate runs. Inference inside your tenant, private endpoints, identity from Entra ID, logs into the same workspace as everything else. If that foundation is not there yet, it is built first, which is what an Azure landing zone is for. Agents deployed outside the governed estate become the exception nobody can audit.
The engagement that delivers it
One workflow, eight weeks, delivered as AI engineering on Azure AI Foundry. Weeks one and two narrow the use case and produce the evaluation set and the cost model. Weeks three to five build retrieval, orchestration, permissions and the interface, with working software in your tenant every week. Weeks six and seven are measurement, failure-mode work, security review and load behavior. Week eight is deployment, monitoring, documentation and a working session with the team that owns it next.
You keep the application code and infrastructure as code in your repository, the evaluation set, the token cost model, the monitoring configuration and the written record of what the agent may and may not do.
In regulated settings the boundary work is most of the engagement rather than a formality. A care network deploying a staff assistant spends more time on which documents may be surfaced to which role than on the model itself, which is why healthcare AI work is scoped differently, and why the controls are designed alongside the compliance architecture rather than bolted on afterwards. If the blocker turns out to be that the numbers cannot be trusted, the honest answer is a data platform modernization, and we will say so before you spend eight weeks discovering it.
Questions we get
Copilot Studio or Azure AI Foundry: how do we choose?
By where the data lives and how much control the workflow needs. If the content sits in Microsoft Graph (SharePoint, Teams, Outlook) and the task happens inside Microsoft 365, Copilot Studio ships in weeks and inherits the tenant permission model. If you need custom orchestration, non-Microsoft data, private networking on the inference endpoint or an evaluation harness you control, Azure AI Foundry is the right surface. We make that call in week one and write down the reason.
How long before an agent is genuinely in production?
Eight weeks for one narrowed workflow, from kickoff to a deployed agent with an owner, an evaluation set, logging and a budget alert. Wider scope does not compress that; it just moves the date. A second use case built on the same foundation runs faster because the retrieval, identity and cost plumbing already exists.
Will the agent surface documents people are not supposed to see?
Not if retrieval is built with security trimming, which means the index carries the source permissions and every query is filtered against the identity of the person asking. This is the step most pilots skip because a demo corpus has no permissions on it. Without it, an agent is a data leak with a chat interface.
What will it cost to run each month?
We build a token cost model during the engagement, per use case, with the model tier chosen on measured accuracy rather than on brand. Budget alerts and rate limits are configured before go-live. Most workloads do not need the largest reasoning model, and paying for one silently is a recurring cost nobody reviews.
What happens when it answers wrong?
It will, sometimes. We build an evaluation set of 30 to 50 real cases from your own history before writing the agent, measure every change against it, and design the interface so a human stays accountable for the decision. An agent with no evaluation harness is not finished, it is untested.
Our data is a mess. Do we have to fix that first?
Partly. Document-heavy use cases often work on the mess as it stands, because extraction tolerates inconsistency. Anything requiring reliable numbers does not. We tell you which of the two you have in the first two weeks, and we will not sell an agent as a way around a data platform problem.