Milind Naphade, SVP, technology, of AI Foundations at Capital One, offered best practices and lessons learned from real-world experiments and applications for deploying and scaling an agentic workflow. Capital One recently launched a production-grade, state-of-the-art multi-agent AI system to enhance the car-buying experience. In this system, multiple AI agents work together to not only provide information to the car buyer, but to take specific actions based on the customer’s preferences and needs. With over 100 million customers using a wide range of other potential Capital One use case applications, the agentic system is built for scale and complexity. Capital One’s applications include a number of complex processes as customers raise issues and queries leveraging conversational tools. These two factors made the design process especially complex, requiring a holistic view of the entire journey — including how both customers and human agents respond, react, and reason at every step. “The main breakthrough for us was realizing that this had to be dynamic and iterative,” Naphade said. “If you look at how a lot of people are using LLMs, they’re slapping the LLMs as a front end to the same mechanism that used to exist. They’re just using LLMs for classification of intent. But we realized from the beginning that that was not scalable.” Based on their intuition of how human agents reason while responding to customers, researchers at Capital One developed a framework in which a team of expert AI agents, each with different expertise, come together and solve a problem. Additionally, Capital One incorporated robust risk frameworks into the development of the agentic system. As a regulated institution, Naphade noted that in addition to its range of internal risk mitigation protocols and frameworks,”Within Capital One, to manage risk, other entities that are independent observe you, evaluate you, question you, audit you,” Naphade said. The evaluator determines whether the earlier agents were successful, and if not, rejects the plan and requests the planning agent to correct its results based on its judgement of where the problem was. This happens in an iterative process until the appropriate plan is reached. It’s also proven to be a huge boon to the company’s agentic AI approach. “We have multiple iterations of experimentation, testing, evaluation, human-in-the-loop, all the right guardrails that need to happen before we can actually come into the market with something like this,” Naphade said. In terms of models, Capital One is keenly tracking academic and industry research, presenting at conferences and staying abreast of what’s state of the art. In the present use case, they used open-weights models, rather than closed, because that allowed them significant customization. That’s critical to them, Naphade asserts, because competitive advantage in AI strategy relies on proprietary data. In the technology stack itself, they use a combination of tools, including in-house technology, open-source tool chains, and NVIDIA inference stack. Working closely with NVIDIA has helped Capital One get the performance they need, and collaborate on industry-specific opportunities in NVIDIA’s library, and prioritize features for the Triton server and their TensoRT LLM. Capital One continues to deploy, scale, and refine AI agents across their business. Their first multi-agentic workflow was Chat Concierge, deployed through the company’s auto business. It was designed to support both auto dealers and customers with the car-buying process. And with rich customer data, dealers are identifying serious leads, which has improved their customer engagement metrics significantly — up to 55% in some cases. “They’re able to generate much better serious leads through this natural, easier, 24/7 agent working for them,” Naphade said.