The Guardian:
Apple contractors regularly hear confidential medical information, drug deals, and recordings of couples having sex, as part of their job providing quality control, or “grading”, the company’s Siri voice assistant, the Guardian has learned.
Although Apple does not explicitly disclose it in its consumer-facing privacy documentation, a small proportion of Siri recordings are passed on to contractors working for the company around the world. They are tasked with grading the responses on a variety of factors, including whether the activation of the voice assistant was deliberate or accidental, whether the query was something Siri could be expected to help with and whether Siri’s response was appropriate.
Apple told the Guardian: “A small portion of Siri requests are analysed to improve Siri and dictation. User requests are not associated with the user’s Apple ID. Siri responses are analysed in secure facilities and all reviewers are under the obligation to adhere to Apple’s strict confidentiality requirements.”
With Apple putting so much marketing and company effort into differentiating itself on privacy, the company must not only do better than “the other guys,” they must be seen to be doing better than the other guys. This incident is not a good look for the company.
Incident? This can not be done differently. There must be living people who improve the service.
What I still want to know is what the scope of this “small proportion of Siri recordings” actually amounts to. Are these random samplings from A) “all Siri users/interactions”, B) “all Siri users who have not opted out of testing”, or C) “all Siri users who have explicitly opted in to testing”? If it were C, surely Apple would be leading with that. The fact that they aren’t, tacitly suggests options A or B, which runs counter to and undermines the company’s strong message on customer privacy.
They said that the information isn’t associated with an Apple ID yet to Shawn King this is still an invasion of privacy. God please go work for a non-Apple blog.
This. You can argue that this should be brought in house or you can argue that such improvement should be made in some other way (and identify that way) or you can argue that Siri quality is good enough and there doesn’t need to be further QA but otherwise I don’t see a way out of this.
Also, is privacy violated here? If you overhear a snippet of conversation as two people walk behind you on the street, have you violated their privacy? This isn’t a perfect analogy as that conversation was in public, but my point is that if someone can’t be identified, has privacy been violated?
Please point to the part of the post where I said, “this is still an invasion of privacy.” Go ahead. I’ll wait.
Totally agree.
The person reporting the issue, seems to be intent on saying specifically that Apple is not anonymizing as it says it is. my guess is that they’re overstating, to make the story juicier – I’m looking forward to more info from Apple.
First, trusting a contractor is from all perspectives a crummy thing to do.
I have to hope that Apple’s contractors are of a higher caliber than the ones Facebook hires as moderators.
Notwithstanding the seriousness of this, it sounds like the voice activated TV bit on 30 Rock. I would love to see a SNL sketch with drug dealers accidentally activating Siri…
Sure it could. You could categorize terms that are likely to be sensitive (e.g. financial and health related terms) and exclude them. Since these snippets are also tagged with location data, you can ring fence certain areas, like hospitals and doctors’ offices as places that will be excluded from the analysis. You’re not going to miss all the false wake words that would presumably be catching things like drug deals or people having sex, but it would be an improvement.
According to the whistleblower, the snippets “are accompanied by user data showing location, contact details, and app data.” That would indicate that there is a way someone can be identified.
My gut instinct here is to believe the white paper over the whistleblower, but if this is actually happening we’ll find out.
It can be done differently if quality control is thrown out the window.
It’s worth pointing out that Apple did change their stance on privacy from absolute to anonymous. They announced they would collect data that was protected by differential privacy, which would completely obscure the identity of the user. However, they are the first to deploy such a system in scale, so flaws are inevitable. Your specific ID can be obscured, but identifying data can be part of a Siri request. If you ask Siri for a location, that location becomes part of the Siri search. Same if you search for a contact or any data in an app. Eventually, Siri will be able to recognize such identifying data, but it won’t get there without human analysis.
Nonsense. As an enterprise contractor or almost two decades, I can tell you nearly every enterprise you know has staff augmentation contractors, including Apple. I’ve worked in banking (what’s in your wallet?) and government, and we all have contractors.
Then what’s the point of this handwringing? As reported on AI, Apple’s disclosures already disclose they listen to some samples for accuracy. They remove the identifying ID.
Not if it’s anonymized. They have to do checks for accuracy.