Is Your Smart Speaker Listening in on You?

Almost a third of American households have a smart speaker, according to a study from National Public Radio and Edison Research. These devices play music, turn on lights, set alarms and perform numerous household tasks, all with a simple voice command.

The Google Home, Apple HomePod and Amazon Echo, among other smart speakers, are like having a personal assistant that happens to be inanimate. But for every voiced request, it’s difficult not to wonder if anyone is listening. As Matt Day, Giles Turner and Natalia Drozdiak write in Bloomberg, for users of Amazon’s Echo speaker, with its digital assistant, Alexa, thousands might be.

Amazon, as Day, Turner and Drozdiak report, employs thousands of voice reviewers, people around the world who help refine Alexa’s responses. As they explain: “The team listens to voice recordings captured in Echo owners’ homes and offices. The recordings are transcribed, annotated and then fed back into the software as part of an effort to eliminate gaps in Alexa’s understanding of human speech and help it better respond to commands.”

Bloomberg points out that Google and Apple also use human reviewers, but their processes appear to less robust.

For iPhone’s Siri assistant, Bloomberg says the recordings “Lack personally identifiable information and are stored for six months tied to a random identifier, according to an Apple security white paper. After that, the data is stripped of its random identification information but may be stored for longer periods to improve Siri’s voice recognition.”

For Google’s Home, “Some reviewers can access some audio snippets from its Assistant to help train and improve the product, but it’s not associated with any personally identifiable information and the audio is distorted,” Google told Bloomberg.

Amazon’s process emphasizes the human role required to create effective algorithms. The Bloomberg article notes that Amazon’s marketing materials downplay that element, saying, “Alexa ‘lives in the cloud and is always getting smarter,’ but like many software tools built to learn from experience, humans are doing some of the teaching.”

Bloomberg reports that Amazon has full-time and contract employees working across the globe—from Boston to Costa Rica, India to Romania, and they’ve all signed nondisclosure agreements. Reporters learned, however, that “They work nine hours a day, with each reviewer parsing as many as 1,000 audio clips per shift, according to two workers based at Amazon’s Bucharest office.”

Day, Turner and Drozdiak call the work largely “mundane,” explaining that “One worker in Boston said he mined accumulated voice data for specific utterances such as “Taylor Swift” and annotated them to indicate the searcher meant the musical artist.”

Occasionally a worker picks up private moments—like crying. Other times, what’s heard is significantly more serious:

Two of the workers said they picked up what they believe was a sexual assault. When something like that happens, they may share the experience in the internal chat room as a way of relieving stress. Amazon says it has procedures in place for workers to follow when they hear something distressing, but two Romania-based employees said that, after requesting guidance for such cases, they were told it wasn’t Amazon’s job to interfere.

In a statement to Bloomberg, an Amazon spokesman said, “We take the security and privacy of our customers’ personal information seriously,” emphasizing that “We only annotate an extremely small sample of Alexa voice recordings in order [to] improve the customer experience.”

Users may already be aware that someone, if not an Amazon worker, might be listening. Two reviewers told Bloomberg that users frequently ask: “Do you work for the NSA?” and “Alexa, is someone else listening to us?”

Despite these concerns, according to a 2019 report from Voicebot.ai and Voicify, the Echo, with its Alexa digital assistant, is the most popular speaker, with 61% of market share, a comfortable lead over Google Home at 24%. Overall, according to a similar study by Adobe Analytics, 71% of smart speaker owners use their device “at least daily.” As TechCrunch writes, “The overall trend is that Amazon Echo remains the smart speaker to beat.”

Researchers have not yet determined whether the Bloomberg article’s claim, that humans might actually be listening to Alexa users, will have an impact on sales. As it stands, Juniper Research, a U.K.-based firm, predicts that there will be 8 billion digital voice assistants by 2023.