David Silver and Richard Sutton, two renowned AI scientists, argue in a new paper that artificial intelligence is about to enter a new phase, the “Era of Experience.” This is where AI systems rely increasingly less on human-provided data and improve themselves by gathering data from and interacting with the world. While the paper is conceptual and forward-looking, it has direct implications for enterprises that aim to build with and for future AI agents and systems. According to the authors, in addition to learning from their own experiential data, future AI systems will “break through the limitations of human-centric AI systems” across four dimensions: Streams, Actions and observations, Rewards, Planning and reasoning. Buried in Sutton and Silver’s paper is an observation that will have important implications for real-world applications: “The agent may use ‘human-friendly’ actions and observations such as user interfaces, that naturally facilitate communication and collaboration with the user. The agent may also take ‘machine-friendly’ actions that execute code and call APIs, allowing the agent to act autonomously in service of its goals.” The era of experience means that developers will have to build their applications not only for humans but also with AI agents in mind. You will also need to design your APIs and agentic interfaces to provide access to both actions and observations. This will enable agents to gradually reason about and learn from their interactions with your applications. If the vision that Sutton and Silver present becomes reality, there will soon be billions of agents roaming around the web (and soon in the physical world) to accomplish tasks. “By building upon the foundations of RL and adapting its core principles to the challenges of this new era, we can unlock the full potential of autonomous learning and pave the way to truly superhuman intelligence,” Sutton and Silver write.