Prem Natarajan, EVP and head of enterprise AI at Capital One, says the economics of gen AI costs are changing so dramatically that attempts to use traditional financial tools to calculate and project AI ROI is the wrong way to go about it. “In the last 22 months, the cost of inference has come down by more than a factor of 1,000 on a performance equivalent basis,” Natarajan tells CIO.com. “Something that cost you $10 to do inference on [two years ago] is now costing you one cent. In that environment of dramatically changing costs, any focus on near-term robust prediction of ROI as a justification for investing in gen AI” is likely to fail. Natarajan, who described the economic changes as being “on the throes of a generational inflection,” believes that CIOs taking that approach to projecting ROI “are making irreversibly bad decisions that will make them fall behind” given their “obsession of calculating ROI in the face of transformative technologies.” Natarajan joined Capital One in March 2023 from Amazon, where he spent almost five years as vice president for Alexa AI. He estimates that at Capital One he oversees “several hundred petabytes of data that will approach exabyte scale” in “a couple of years.” That data trove is a key asset for Capital One in making the most of AI, according to Natarajan, who sees data governance and accessibility as additional keys to AI success. Jason Andersen, a vice president and principal analyst tracking AI for Moor Insights & Strategy, says Natarajan’s take on the ROI issue for CIOs is valid. “Enterprises [such as Capital One] are starting to get really smart about how they are deploying AI and building AI applications,” Andersen says. “The reality is that we haven’t seen a trend in technology ever move this fast — ever.” That speed has caught many IT executives off guard as techniques that have always worked for them stop working, Andersen adds. “With this absolute velocity, you are seeing the old norms of trying to figure out how much to invest, those are no longer useful tools,” he says. “If you use traditional methods, you just don’t get it.” With almost any form of AI, he says, “your data advantage is your AI and ML advantage.” “The amount of proprietary data we had was an important asset to be brought to life in building generative AI applications and capabilities that would be differentiating for us,” Natarajan says, stressing that their evaluations showed that they “could not use closed-source models, because you cannot meaningfully customize those models.” The Capital One AI team eventually opted to use Meta’s open-source Llama LLM and set about building AI solutions atop its public cloud foundation, established prior to Natarajan’s tenure at the financial services company. “Capital One was the first bank — and to date, the only bank — that is all in on the public cloud. They shut down all their data centers over a period of two years and moved everything to AWS,” Natarajan explains. “We became cloud-native developers.” At Capital One, which employs roughly 14,000 IT specialists, talent is critical but so too is data — perhaps more so, Natarajan says.