It's often a joke among those using home artificial intelligence (AI) assistants that the algorithm is "always listening."
Imagine someone said to their smart speaker, "Alexa, play 'Dance' by the Rolling Stones." Then, later they saw an ad for Rolling Stones poster art on their Instagram feed.
Coincidence? Most likely it's just a form of targeted advertising at work, though it can leave some people feeling unsettled with the idea that a device is spying on them.
These devices are not always listening, but there are some aspects of this technology that consumers should be aware of, according to Umar Iqbal, an assistant professor of computer science and engineering in the McKelvey School of Engineering at Washington University in St. Louis.
"Consumers should be able to take advantage of the benefits generative AI provides," Iqbal said. "The question is how to make these technologies meet user expectations, particularly in the context of privacy and security."
Iqbal has previously studied how Amazon uses smart speaker interaction data to infer user interests and then uses those interests to target personalized ads to the user.
Iqbal said that these smart speaker-based AI assistants work by being activated by certain words, such as "Alexa" or "Siri." Amazon's FAQ site describes these words as chosen "wake words."
Some Amazon customers recently were alerted to a change in how the company processes voice commands issued to AI assistant Alexa. Users now no longer have a choice to process voice commands issued to smart speakers locally. Instead, all commands will be sent to Amazon's cloud computing centers for processing.
In the past, Amazon and other tech companies typically have used voice command data to personalize advertising to each user. In addition, Amazon now wants to use those commands to train Alexa+, the company's next-generation AI assistant. However, until recently, Amazon customers using three previous versions of Echo could opt out of having their voice commands process at the server side. Only a very small minority of customers using English chose that option, according to Amazon.
The announcement emailed to Amazon customers and shared with media announced that if a customer's Echo device was set to "Don't save recording," their Voice ID feature would not be functional. Customers who opted out of saving their recordings would lose much of the personal assistant-type features that Voice ID offers.
Iqbal said the fundamental issue is the lack of transparency. Users should be in control of how their data is processed and eventually deleted.
"From my perspective, the lack of transparency results in a lack of trust," he said.
Being less open about the fate of those recorded requests only adds to ingrained fears of the technology, he added. Iqbal described Alexa as similar to chatbots such as ChatGPT. Alexa is using large-language models to process information. It's just a voice-based chatbot.
Alexa operates on the hardware of both smart speakers such as Echo and on mobile devices. Those devices don't have the processing power to run large AI models, which is why Amazon wants to send these recordings to the cloud for processing. However, Iqbal said there are other ways to integrate large-language models that give users more autonomy over the fate of their data.
Apple products, for instance, offer more safeguards in selective use of what data gets sent to the cloud and better controls on tracking that data.
"There are options that are more user- and privacy-friendly," he said.
As AI assistants begin taking on more tasks for consumers, like planning vacations and scheduling work events, the threat to data security increases as the various algorithms swap user information and potentially open themselves to data manipulation. Iqbal and colleagues at McKelvey Engineering have developed tools to counter these threats. One such method, known as "IsolateGPT," keeps external tools isolated from one another while still running in the system, allowing the AI assistants to do their job while also keeping user data secure.
