The most expensive voice AI mistakes happen before you start building

This article is part of the Agent Architect's Digest, a series from Parloa's Agent Architects team.

There’s an old saying that an ounce of prevention is worth a pound of cure. When it comes to building enterprise voice AI agents, I’d modify it slightly: An ounce of discovery prevention is worth a pound of implementation cure.

Rework is expensive in any software project, but the multiplier is particularly high in voice. When teams change an architectural decision after implementation, they may have to rework prompts, rerun user acceptance testing (UAT), retrain quality assurance rubrics, reopen compliance reviews, and reschedule telephony or carrier work. That makes the architectural decisions made during discovery unusually consequential.

The problem is that discovery doesn’t happen in a vacuum. There are commercial commitments, finite resources, and a delivery date that still needs to be met. When discovery gets compressed, it's usually because gathering everyone's assumptions takes longer than the timeline allows, and no one prioritized which of them actually needed validating.

Effective discovery is then all about knowing which questions you cannot afford to leave unanswered. As Lead Agent Architect at Parloa, that's one part of the job I've become obsessed with. Every customer I've seen succeed long term did the hard discovery work first, and it's what let everything after go smoothly.

Here's how I determine which discovery questions matter most.

Not every assumption carries the same risk

I think about assumptions in terms of an inverse relationship: The greater the potential impact if an assumption is wrong, the less willing I am to leave it assumed.

Routing, escalation, intent coverage, authentication, application programming interfaces (APIs), and the agent’s place in the call path all fall into that category. I think of these as one-way-door decisions: choices that become disproportionately expensive to reverse because they propagate into almost every flow you design afterward.

Telephony, escalation, and authentication

Take telephony. Before designing the agent, we need to understand where it will actually sit: on a session initiation protocol (SIP) trunk or in front of an existing interactive voice response (IVR) system. We need the real IVR and routing map. We also need to know how the environment handles dual-tone multi frequency (DTMF), barge-in, and accessibility requirements.

The same applies to escalation. “It transfers to a human” sounds like a requirement until you start asking questions. Is it a warm or cold transfer? Which queue receives it? What context needs to be passed? What happens when that queue is closed or full?

Authentication creates similar dependencies: Is the caller authenticated upstream? What information is the agent permitted to receive? What is the knowledge-based authentication (KBA) policy, and who owns it? Once authenticated, can the agent perform write actions, or is it limited to read-only access?

These are the constraints that determine what we’re able to build.

Prioritize with data, not opinion

Determining these constraints also helps avoid one of the most common reasons pilots fail in production: trying to do everything at once. It's not unusual for a client to bring 20 use cases to a kickoff, but no matter how great the ideas are, you need a way to cut that list down to the four or five you can actually build well. The defensible way to do that is data: call volume, containability, integration cost. Discovery buys you the standing to say what gets built first and what waits.

Ask for the details

One of the least useful discovery questions you can ask is, “Do you have an API?” The answer can be yes while telling you almost nothing about whether that API can support the experience you’re designing. For a voice agent, I want to know the API’s p95 latency, authentication model, rate limits, sandbox availability, read and write scope, and who needs to approve changes.

Because while a four-second backend call to another system for authentication might be perfectly acceptable in chat, in a voice experience, four seconds of waiting can destroy the conversational feel. For voice, latency is a design constraint that has to be understood up front.

Compliance and data

Compliance and data carry the same requirement: know the specifics before you design around them. Where can transcripts be stored? Can they leave the tenant? What needs to be retained or redacted? Which recording and consent rules apply in the caller’s jurisdiction? Does PCI, PII, or HIPAA scope mean part of the interaction must move to DTMF or a human?

You want those answers before you try to design a experience around an architecture that cannot support it.

The manual groundwork automation requires

If an agent is built entirely on assumptions, you're off to a risky start. I's important to have everyone, all the stakeholders and the people who will be interacting with the product directly, in the same room aligning on goals and technical restrictions from the start of the build.

The same applies to documentation. Process documentation tells you how an interaction is supposed to work. Real calls tell you how it actually works. Understanding processes upfront reduce the risk of rework.

Go to the source

For every candidate use case, we listen to real recordings. Twenty to thirty is a good starting point, but it’s important not to choose only clean, representative calls. Deliberately find the ugly ones, because exceptions often define architecture requirements. A straightforward call can tell you what the happy path looks like, but a difficult one tells you what happens when authentication fails, a caller changes intent, the backend doesn’t respond, an escalation queue is unavailable, or the documented process simply doesn’t match reality.

We also shadow live agents doing the task end to end, ideally with timestamps, including the supervisors who handle escalations. This process routinely exposes steps nobody thought to document, and while frontline agents tend to know the path extremely well; supervisors know where that path breaks.

Another useful byproduct of shadowing the live agents is caller language. The exact words people use to describe their problems become valuable input for intent design and prompt language later.

A standard intake questionnaire can help structure this process, but it isn’t enough. Async forms have a tendency to capture the process people believe exists, which is rarely what actually exists in entirety. Workshop the questions live, probe the answers, and make sure the person explaining a process is close enough to the work to know how it actually happens.

Start with a slice, then expand

Eventually, discovery reaches its limit.

A stakeholder can tell you an integration should work. A diagram can show that telephony should route correctly. An API specification can suggest that the required data should be available. At some point, the best way to answer those questions is to build something.

That’s why I like to run a thin vertical slice early: one call through the real telephony environment and into the API sandbox before broad agent design begins.

That single test can resolve more meaningful unknowns than another week of workshops. It changes the question from "Do we believe this architecture will work?" to "Did it work?" Once that one path works end to end, you can expand outward from it, rather than trying to build every path at once and hoping they all hold up.

This is also why I like drafting UAT acceptance criteria during discovery rather than waiting until the agent is built. If you can't write the test for a requirement, you probably don't understand the requirement well enough yet. The gaps in your test plan become the gaps in your discovery.

Make uncertainty visible

Discovery should be written, versioned, and explicitly agreed upon. There are two things we want to make explicit before moving forward: the assumptions we’re still carrying, and the conditions that tell us we’re ready to build.

Keep an assumption register

One of the most useful artifacts is an assumption register. Every meaningful assumption gets an owner, a validation method, and an expiration date. An assumption shouldn’t be allowed to survive for months just because nobody disproved it. If a high-impact assumption reaches its expiration date without validation, it becomes a risk with a name and an owner attached to it.

We also mirror what we’ve heard back to the client in writing. People are often much better at correcting an inaccurate statement in front of them than volunteering the missing information from scratch. That written record creates a shared understanding of what we know, what we still believe to be true, and what still needs to be validated.

Define what done looks like

Before discovery begins, define what it means for discovery to end. Written exit criteria turn “Are we ready to start building?” from a judgment call into something the team can objectively evaluate. That includes agreeing on what the agent will not do. A written non-goals list establishes the agent’s boundaries and helps prevent scope creep as implementation gets underway.

That document should nail down the practical edges, too: the traffic shape you're designing for (peak concurrency, seasonality, volume per intent), and the language, locale, and accent range of the real caller population, since that's rarely identical to whatever was used in testing. It should also say who owns the agent after launch, what change control looks like, and where the authority sits to approve a change, or to say no to one late in the process.

It also means defining success before implementation starts. Teams need one agreed definition of containment, not three competing ones. Establish the baseline for metrics such as average handle time (AHT), transfer rate, customer satisfaction (CSAT), and abandonment so that there is something to measure the new experience against.

That baseline is what makes the next question answerable: What result would cause us to turn this off? If you don’t know what failure looks like before launch, it becomes very difficult to make objective decisions once the system is live.

Discovery is risk management

A good discovery process doesn’t answer every possible question. Nor should it. Enterprise teams have deadlines, budgets, and customers waiting for the thing they’re trying to build.

The goal is to use the time you have to remove the uncertainty that carries the greatest risk.

That's why the most valuable work an Agent Architect does sometimes happens before the first agent is built. We listen to what live callers do, validate the assumptions that could change the architecture, test the riskiest pieces against real systems, and make the remaining uncertainty explicit. 

There are no shortcuts that eliminate that work. You can choose whether to do it during discovery, when changing direction is relatively cheap, or during implementation, when it isn’t.