How can AI agents grasp discernment when humans barely can?

This article first appeared on Forbes Business Council.

In ancient Greece, the highest form of intelligence wasn't knowledge. It was phronesis: practical wisdom. The kind of judgment that governs how we navigate tradeoffs, make decisions under uncertainty and act when the right answer isn't obvious.

Think of a general deciding whether to retreat. A doctor weighing the risks of surgery. A judge balancing competing rights. Each faces situations where there's no formula, only competing aims and imperfect information. They act not because they know what will happen but because they must decide what should happen.

At present, AI agents haven't reached that kind of reasoning. What we've built so far lives in a different world: prediction. Feed a model enough data, and it gets very good at forecasting what comes next—the next token, the next frame, the next protein structure. Nearly every major AI breakthrough over the last five years has been a triumph of statistical forecasting, and that got us very far, very fast.

However, as we now start building AI agents, the limits of prediction are starting to show. Agents don't just need to guess what might come next; they need to choose what to do, and discernment, or phronesis, is still out of our grasp.

Predictions are relatively easy.

In simple terms, prediction is statistical, while judgment is directional.

• Prediction asks: "Given everything I've seen before, what's the most likely next step?"

• Judgment asks: "Given imperfect information, competing priorities and real consequences, what action should I take?"

For example, take an AI agent of a health insurance provider. Let's say a policyholder calls in, upset that a medical procedure wasn't covered. The AI agent may correctly identify it as a billing issue and start walking through options.

Midway through the call, however, the customer mentions skipping care because they couldn't afford the last out-of-pocket bill. Now, the stakes change.

Should the agent continue with the billing script? Escalate to a human immediately? Flag this as a potential health risk and follow internal protocol? The customer didn't explicitly ask for help, but the subtext is critical. Most importantly, how the system responds could impact health outcomes, not just customer satisfaction.

There's no one 'correct' answer, but every choice has a cost.

A systematic Lamar University review from late 2024 that involved over 100 studies on AI decision making made the gap clear: While AI has excelled at recognizing patterns, it consistently fails when ambiguity, competing goals or missing data force actual decisions.

AI agents operating in real-world environments will constantly face these situations. They'll need to:

• Prioritize conflicting goals.

• Make decisions with incomplete context.

• Navigate ambiguous inputs.

• Balance efficiency, safety and ethical constraints.

• Reason about long-term consequences.

You might be thinking that this is what decision trees were made for. In a way, this is true. They were the early scaffolding for machine reasoning that were clear, structured and easy to follow. However, decision trees were built for neat problems (e.g., discrete inputs and predictable branches).

Today's AI agents play an entirely different game. Large language models (LLMs) and their surrounding architectures can adapt on the fly, weigh uncertainty and surface options that aren't hard-coded into a static tree. Still, even they start to strain when the task demands actual discernment—that is, when the problem isn't about picking the best move but deciding what "best" even means in a shifting landscape.

Moreover, the challenge isn't only about making decisions. It needs agents that can act when the data runs out, when values collide and when none of the options fit neatly into a predefined path. That's where decision trees end, and the next generation of AI agents needs to begin.

Where do statistics break down?

When AI systems hallucinate, it's not always because something has gone wrong. Sometimes, it's because everything is working exactly as designed. A model trained to predict the most probable next word will produce something that sounds plausible, even when it isn't. That's the nature of statistical inference; it fills in blanks with patterns.

Most of the time, that's enough. When it breaks, though, the scaffolding underneath is often the cause. The technical gap in judgment extends beyond needing more data or larger models; it needs abilities that current predictive systems lack.

World Modeling

World modeling is the ability to build causal representations of how a system behaves (i.e., how inputs propagate through processes, how actions lead to state changes, how feedback loops emerge).

Uncertainty Quantification

This means AI agents should act on uncertainty—whether through calibrated confidence scores, Bayesian inference, ensemble methods or external risk thresholds. This isn't optional in domains where the cost of error is high, such as insurance, healthcare, banking, etc.

Counterfactual Reasoning

Counterfactual reasoning means agents must evaluate actions not just based on observed outcomes but on plausible alternatives. This includes estimating impact differences between chosen and unchosen paths using simulation, learned models or structured reasoning.

Explicit Objective Functions

Most environments involve competing goals. AI agents need mechanisms to represent, prioritize and dynamically adapt objective functions,such as thinking through "what if" scenarios and alternative outcomes.

Value Alignment

Value alignment involves embedding domain-specific constraints and human values directly into decision policies. This requires interpretability, auditability and sometimes direct human feedback—not just pretraining on broad datasets.

Moreover, this requires fundamentally different architectures, training approaches and system designs. We're already seeing retrieval-augmented generation (RAG), tool-using agents, multiagent orchestration and simulation-based fine-tuning used in production across sectors like insurance.

Discernment is about restraint, especially in CX.

CX is often where AI agents meet their limits because the context won't sit still (i.e., the customer's stated need isn't always the intended one), and AI agents have to respond before that's fully clear. Ultimately, phronesis was never about certainty. It was about carrying the weight of action when the right answer wasn't obvious, and that's what CX demands.

Until agents can reason with that kind of depth, the guardrails protect your customers and teach the AI agent what it means to choose carefully.