“Intention economy,” where AI predicts and sells your decisions before you even make them, is becoming possible. Researchers from Cambridge warn of the profound implications of this emerging technology.
Artificial Intelligence advancements are on the verge of transforming the way we make decisions — and how those decisions are monetized. In a study published in Harvard Data Science Review, experts from the University of Cambridge’s Leverhulme Centre for the Future of Intelligence (LCFI) caution that emerging “intention economy” technologies could sell human intentions before individuals even realize they’ve made up their minds.
Yaqub Chaudhary, an LCFI visiting scholar, and Jonnie Penn, a historian of technology at Cambridge’s LCFI, reveal that generative AI, coupled with our growing trust in chatbots, is opening new horizons of “persuasive technologies.”
“We caution that AI tools are already being developed to elicit, infer, collect, record, understand, forecast and ultimately manipulate and commodify human plans and purposes,” Chaudhary said in a news release.
The implications are profound. AI agents, interactive digital assistants, will gain access to intimate psychological and behavioral data derived from casual, conversational dialogues. This could range from everyday interactions to sensitive data impacting voting choices and consumer decisions.
“Tremendous resources are being expended to position AI assistants in every area of life, which should raise the question of whose interests and purposes these so-called assistants are designed to serve,” Chaudhary added.
These technologies could transition us from the “attention economy” — where user attention is monetized by platforms like Facebook and Instagram — to an “intention economy.”
“Unless regulated, the intention economy will treat your motivations as the new currency. It will be a gold rush for those who target, steer and sell human intentions,” Penn said in the news release.
For decades, social media platforms have thrived by commodifying user attention. This shift towards intentions — our plans, buying habits and even political leanings — heralds a potential transformation in digital marketplaces.
“While some intentions are fleeting, classifying and targeting the intentions that persist will be extremely profitable for advertisers,” Chaudhary added.
Companies already collecting this data could leverage Large Language Models (LLMs) to analyze individual user habits, preferences and social cues. For instance, AI could suggest actions like booking movie tickets by reading cues such as a user’s tone of voice and prior conversations.
Tech giants are already making strides toward this future.
In a 2023 blog post, OpenAI highlighted the need for “data that expresses human intention … across any language, topic and format.” Apple’s new “App Intents” framework for Siri is designed to predict future user actions based on developer-provided predictions. Meanwhile, companies like Meta and Nvidia discuss using AI to understand both intention and desire.
“These companies already sell our attention. To get the commercial edge, the logical next step is to use the technology they are clearly developing to forecast our intentions and sell our desires before we have even fully comprehended what they are,” added Chaudhary.
While the “intention economy” may offer unprecedented marketing opportunities, it also poses essential ethical and societal questions.
“We should start to consider the likely impact such a marketplace would have on human aspirations, including free and fair elections, a free press and fair market competition, before we become victims of its unintended consequences,” Penn added.
Awareness is crucial as the world stands at the threshold of this new economic model. As Penn concluded, “Public awareness of what is coming is the key to ensuring we don’t go down the wrong path.”