Making my way through a tidal wave of content responding to the Google voice bot demo at Google I/O. We wrote about it in this post: Watch Google Assistant make a phone call to schedule an appointment. Stunning.
If you have not yet seen the demo, click over and watch it now. It’s short.
With that demo in mind, take a minute to scan through Jason Kottke’s piece, Should our machines sound human?, which obviously was the inspiration for part one of the title of this post.
With all that under your belt, here’s my take.
Subtly weaving human queues (like “uhh”) into the conversation clearly made the AI voice believable, fooled the listener into thinking it was interacting with a human. The tech behind the voice bot was quite robust, responding quickly and on target, ultimately solving the intended problem.
Clearly, Google Duplex is a powerful tool that can solve a host of problems, including taking over voice interactions to save humans time, and providing a voice for the voiceless, or those uncomfortable with human interaction.
So far, we’ve got sophisticated, groundbreaking tech, along with a tool to provide a voice to help us.
But along with that comes a series of ethical issues. Is it OK for a machine to fool us with a human voice? And what about informed consent?
One issue with the demo was the fact that it was clearly based on trickery. The person being called did not know they were speaking with a machine. The audience applause was, in part, appreciation of how well the voice bot did its job, how convincingly it portrayed a human.
Informed consent. Does Google have an obligation here to make it clear that the voice is machine generated? Should I have the right to opt out of automated voice bot communications?
Another issue for me is the worry about this tool falling into the wrong hands. I can only imagine someone crafting a clever script that automated a typical spambot strategy.
Instead of a simple recording trying to trick me into their spider’s web, imagine a voice bot with a wealth of detailed, personalized knowledge about me at its disposal, with the goal of convincing me to hand over my banking or credit card info. That voice bot, acting at the behest of the dark side, would use machine learning to improve its technique, slowly improving the response scores and adjusting in real time to changes in customer responses.
I don’t see Google being a bad actor here. They are genuinely excited about the brave new world they are building. But the fact that they did not weigh in with any awareness of the ethical issues that come along with this sophisticated tech is worrisome.
From this Anil Dash Twitter thread about informed consent:
Any interaction with technology or the products of tech companies must be exist within a context of informed consent. Something like #GoogleDuplex fails this test, by design. That’s an unfixable flaw.
And this response from Dieter Bohn:
Google told me yesterday that it does believe it needs to do that – but doesn’t know they best way yet.
This should have been part of the preface, the lead up to the demo. These ethical issues should be worked through at the same time, presented side-by-side with the technology which opens the Pandora’s Box in the first place.
Have to wonder what the legal implications are to any agreement made between a human and the “person” they spoke to? As always, the law will be way behind the tech.
Here’s another scenario… right now legislators discriminate between kinds of feedback on an issue. Obvious form letters (where you signup on a site to have a message sent to a legislator by some organization) get one, lower priority. Emails get another. Physical letters another… and calls get a high priority. This is done because the latter are felt to show a more personal touch and a higher commitment to the issue by the voter (it’s one thing to click a button on a site, another to take the time to look up a number and call).
Now imagine I want the high consideration given a call but the numbers of a bot. I program something like Duplex to vary their script enough to be credible and I let thousands of them loose. Without some indication t hat the call is from a bot, the legislator is flooded with calls that feel like they’re from humans and…
Seems easy, correct and a branding opportunity for Google’s Duplex, et al., to identify itself. As in “Hello, I’m X’s digital assistant”.
Somehow, I don’t quite see Google’s culture considering informed consent important enough to prioritize. Google may not ever have set out to be a bad actor, but it’s proven to be a careless one often enough.
Update: Gruber raised an interesting point. These are recordings of demos, not done live. Perhaps the product isn’t quite ready.
I have so many more problems with this.
Who is the agent actually representing? It’s supposed to limit itself to the user, right? Will the agent limit itself to representing the user? (“Oh, hey, and do you have a website?”) Will Google use answers for other purposes? Will Google record the call to use the raw voice data later? (It’s bad enough the person receiving the call is talking to a bot. It’s awful if those calls are going to be re-used later.)
I just don’t trust Google to make the right decisions here. I don’t see how anyone could.
Maybe I’m just a technophile, but I truly don’t agree with many of the “negatives” put forth here, especially the one about spam robocalls. I already get these, and if they aren’t recordings then they are scripted humans, and many of them are targeted to me. My wife got one the other day claiming her immigration status was at risk, and many other immigrants have as well, so they definitely have some kind of list to target.
I don’t see anything different between a voice bot or a trained human reading a script. The way to combat that is not by banning bots; it’s to continue informing people as we do with current robocalls to not answer the phone or to not give out personal details over the phone for such things.
Informed consent? We’re already past that with human and recorded robocalls. None of them have to inform anything, and most of them even fake the phone numbers they call from. I prefer we focus on the good that assistants like Google Duplex will do for us and not just throw out this baby because its something new we don’t think we understand or that somehow current laws and actions don’t apply to it.
The digital assistant in the first call in the video linked above does identify itself as an assistant. Why does it have to do more than that? When you are getting business, does it matter if the assistant is a well-spoken, well-working digital one or a human?
I’d be disinclined to use digital assistants that branded themselves. If that means paying for the service to remove the branding, I’m okay with that.