Building AI agents on shifting ground

This article is part of the Agent Architect's Digest, a series from Parloa's Agent Architects team.

In December 2024, Parloa found out that a core dependency of our product, GPT-3.5, would be retired in just eight weeks. For those who have worked with standard software products, this is a hard speed to conceptualize. 

Historically, software providers give years of notice before components or operating systems are retired. But LLMs operate on a different timeline. 

The AI landscape is evolving at an incredibly fast pace, and that means the conversational AI operating systems are as well. Mentioning GPT 3.5 feels like ancient history today, as current models are at least two generations advanced, with more already underway.

As an Agent Architect, I work at the intersection of crafting a conversation, applying advanced prompting strategies to achieve the desired outcome, and thoughtfully planning for the overall structure of the agent. Needless to say, LLM updates directly impact my work. Here’s how my team and I navigate the continuous change:

Recognizing LLM modifications as central to the agent development lifecycle

In agentic voice solutions, LLMs make sense of inputs, trigger tool calls, and ultimately bring a conversation with a caller to the desired outcome. The model makes sure guardrails are observed, handles thousands of calls per hour, and provides an extensible framework that can be maintained by business users.

In order to deliver reliable outcomes at scale, working with large language models requires three fundamental differences from standard software:

• End-of-life comes fast, and keeps coming. The GPT-3.5 retirement wasn't a one-off. GPT-4o was the next production ready model, and now its successor, GPT-4.1, is scheduled for end-of-life this October. Because the LLM technology is not yet mature, capabilities change vastly between models, and the providers retire support for legacy models. The planning horizon for a foundational dependency is measured in months. 

• A migration is never a find-and-replace. Moving from GPT-3.5 to GPT-4o, or from 4o to 4.1 is not a matter of simply upgrading from one version of software to another with minor UI adjustments. Behaviour shifts with every model, requiring significant architectural work. When comparing GPT 3.5 to 4o, for example, the new model interpreted instructions differently and was more conversational in nature, so prompts that had been carefully tuned for one version had to be reworked for the other and new guardrails added. 4.1 is much more verbose with intermediate messages, requiring new prompts to reduce this verbosity.

• "Best" is contextual, not absolute. A model's performance depends on the system around it. We made Gemini 2.5 Flash production-ready on our first AMP platform with excellent results: fast, with tool-calling that outperformed GPT-4.1 for our use cases. As we evolved our platform and introduced Subtask Agents, that same model had to be re-qualified from scratch under new conditions. You cannot rank models on a leaderboard and be done. Performance is a reflection of both the model and the architecture of the harness.

Seeing migration as an opportunity

The reality of LLM evolution requires a change in how agent builders perceive migrations. Whereas in the past, migrations had been seen as an interruption to regularly-scheduled programming, now it must be seen as routine. 

The shift in perspective is important because model change is not free. Every move requires prompts to rework, behaviours to re-validate, edge cases to re-test, and timelines to coordinate. Planning for that effort up front transforms the migration from being a surprise expense to a standard line item. More importantly, it turns a chore into an opportunity.

That’s why at Parloa, we plan for model migration at the start of every new project. We include it as one of our standing maintenance tasks. It’s as routine as security patching or dependency updates in any mature software practice.

Understanding the downstream benefits

Each migration is a scheduled moment to revisit prompts, retire workarounds built for an older model's limitations, and let the agent hold a more natural, capable conversation. Planned well, the maintenance task serves as the improvement engine. The model retirement forces a visit; what you do during that visit is where the product gets better.

Planning for the customer impact

Because retirement windows are short, readiness is a permanent state. We track the lifecycle of every model in production, keep the next candidate already in evaluation, and hold a living set of representative conversations and scoring criteria for each use case. This way, we can judge a new model on evidence and re-run the same judgment based on KPIs, like latency and tool calling reliability, whenever the platform changes. We qualify fast and adopt deliberately.

Through this disciplined approach, the model churn happens on our terms, not the provider's, enabling us to transition customers on a timeline that works best with their schedule. We place different customers on different points of the migration curve based on their specific needs, setup, and where they are in their agentic implementation. The end result is controlled change that delivers consistent or even improved quality.

Volatility as an advantage

The LLM evolution isn’t going to settle any time soon, and we know that waiting for steady-state is an illusion. We’ve learned to anticipate change and build for it, and use it as an opportunity to make our customers’ conversations feel a little bit more human every time.

GPT 4o and 4.1 are retiring soon, but we’re not concerned. We've already been preparing for GPT-5.4 mini, the next model capable of the performance our agents require. More on that soon.