Sid and Andrew explore the unique governance challenges of agentic AI systems, highlighting the compounding error rates, security risks, and hidden costs that organizations must address when implementing multi-step AI processes.
In this last episode of the series, the hosts explore governance challenges that organizations must navigate when implementing multi-step AI systems. While traditional AI governance focuses on single-step processes, agentic systems require an entirely new framework that accounts for their multi-step processes, expanded capabilities, and risks.
They discuss the mathematical reality of compounding errors in multi-step processes. Even with impressive 90% accuracy per step, a four-step agent's overall accuracy plummets to just 65%. This basic limitation should give pause to organizations rushing to implement agentic systems for critical functions without proper safeguards.
Security and confidentiality concerns take center stage as Sid and Andrew highlight how agentic systems create new vulnerabilities. Unlike traditional one-way AI systems, agents establish two-way information flows where sensitive data like credit card numbers or personal information might be processed across multiple steps with inadequate protection. For example, targeted prompting to improve awareness comes at the cost of performance (arXiv, May 24, 2025). Organizations must rethink their data handling practices when implementing these systems.
Perhaps most overlooked are the true costs of responsible agentic AI. Beyond API calls and compute resources, organizations face substantial "second-order, third-order, fourth-order" costs in governance overhead, specialized monitoring, and human expertise. These hidden expenses often make simpler approaches more cost-effective for many use cases.
The hosts advocate for modular governance where each distinct function—perception, reasoning, action, and learning—receives independent validation and monitoring. This granular approach stands in stark contrast to current industry trends favoring end-to-end black-box solutions that resist meaningful inspection.
Is your organization prepared to implement the robust logging, monitoring, and validation frameworks necessary for responsible agentic AI? Are you considering whether LLMs are truly the best tool for reasoning within an agentic system, or if more transparent and efficient mechanisms might better serve certain functions? These are the critical questions organizations must address before embarking on agentic AI initiatives.
Make sure you check out Part 1: Mechanism design, Part 2: Utility functions, and Part 3: Linear programming. If you're building agentic AI systems, we'd love to hear your questions and experiences. Contact us.
Ask the hosts!
