Quartz:
Consider this thought experiment: you are traveling along a single-lane mountain road in an autonomous car that is fast approaching a narrow tunnel. Just before entering the tunnel a child attempts to run across the road but trips in the centre of the lane, effectively blocking the entrance to the tunnel. The car has but two options: hit and kill the child, or swerve into the wall on either side of the tunnel, thus killing you.
This might seem a pointless exercise, but as the field of robotics matures and artificial intelligence design takes on the burdens of critical decision making, this ethical dilemma will force itself on software designers.
I doubt that a driverless car will even be able to understand the difference between a child or something else, a racoon for example. The sensor and computing power for such recognition at speed would cost more than the rest of the car.
It’ll probably always drive over anything it can’t avoid.
In a way, I find this point moot.
Driverless cars won’t need windshields/windows. So, your car’s A/B/C/D pillars could be much much stronger, or it could be just one big canopy
Top that with larger crumple zones + airbags. Uless the car gets smashed under a huge slab of concrete, there’s literally no risk to the passenger.
So driverless cars will be limited to very slow speeds and good weather?
its not going to be like Driver to Driver less cars in one day. Initially it’ll be manned by a driver. Then driverless on interior roads, during good weather. and so on. It’ll be incremental. starting with a whole taxi fleet serving a small radius
@disqus_r5UpUhT514:disqus If we’re talking about this already, it will be built into the AI when the cars finally do come out. Maybe a car will simply broadcast who (demographically) is in it so that the most ethical decision can be made in an accident. Today’s Kinect would be able to do this. We’ll all be identifying ourselves to/from our devices soon enough, anyway.
@anondesi:disqus I’m not so sure that avoiding the question is the same as answering it. In the billions of trips that will eventually be taken by driverless cars, there will be a situation in which an accident is unavoidable. The question remains: Is the car’s responsibility its own host? The youngest person? The person responsible for the most people? The one society considers the most valuable? If the failure rate is 0.0001%, then people will keep their windows and windshields and that failure (as small is it is) will inevitably play out.
WTF, does the driverless car have sensors with a range of only 20 meters? Did the child just teleport? the child is running at c, and then suddenly stops? The child is invisible until it trips?
That’s supposed to be a thought experiment? Nonsense, that’s a “deepity”. It seems profound unless you take a few seconds to analyze the real world it pretends to model.
A few things:
It doesn’t need to know WHAT the thing moving in a direction that is not the correct flow of traffic at a proper speed is, only that SOMETHING is not behaving correctly.
As soon as the thing is moving into the road “incorrectly”, the car should begin applying brakes in a most enthusiastic fashion. That gives the car a good 2-3 seconds of extreme slowing down. With antilocks, the chances of a skid are reduced and automated systems would be able to compensate faster for that situation anyway.
that will bleed off a lot of speed. Secondly, there are always alternatives to MUST HIT EVERYTHING HEAD ON like these moronic “thought experiments” require. is there a guard rail? If so the car can veer into it, in a sideswipe kind of way that will assist in slowing the car down even faster. Is there a ditch or a shoulder? Oh look, another option.
All the car has to do is slow down enough so that a collision with say a wall won’t kill or even severely injure the occupants. That’s highly possible in the real world. In addition, and I know that for “thought experimenters” this is news, roads and tunnels are designed by people who actually think about this kind of crap. Speed limits are designed for this kind of thing and in a driverless car, one would assume speeding becomes a non-issue.
Also, yes, you want a windshield. Humans are pretty good at things if they’re paying attention and the ability to override and say “slow down, there’s a kid on the side of the road” is important.
Why do people pay attention to this kind of drivel, much less pretend it has any form of profound insight to offer is a mystery to me.
Barring “rocks fall, everyone dies”, you’d be amazed at how few accidents are completely unavoidable.
This is an easy one. The driver always has the responsibility to avoid a pedestrian. Always. If your car is on auto-pilot, that does not absolve you of any of your responsibilities as a driver.
Arguing the details of a hypothetical situation is pretty pointless. There are always edge cases in life, and the job of a software engineer means dealing with them gracefully, especially when life or death is on the line.
If it bothers you so much, lose the tunnel and the child.
IF the vehicle finds itself in a position of unavoidable harm to either a pedestrian or the passenger, what should it do? Who should decide? That is the question.
I believe existing systems already can detect different objects to a fair degree of accuracy.
this isn’t hypothetical. This is real world, there’s scores of data on how things work, and this is actually something you can model. “Unavoidable” is simply unrealistic. If the car cannot “see” far enough ahead/wide enough, and process the data fast enough to avoid that situation or be aware of its potential, then that’s a clear sign they shouldn’t be on the road.
Computers can definitely tell the difference between a child and a racoon.
Wasn’t that the plot of Jurassic Park?
in a few ways.
this is one of the things that have been problems for driverless cars. Although inattention and poor training can kill this, as it turns out, humans are really good at processing a lot of data in a useful fashion. We can observe the road not just 20 or so meters in front of us, but hundreds of meters away. We can tie multiple inputs together, and along with experience, make small adjustments that keep us well out of trouble.
For example, on an interstate, when I come to an exit, unless i’m getting off the interstate, i move out of the right lane, because I know there will be a lot of cars suddenly changing speed, or diving across that lane because they almost forgot the exit.
I do this based on experience. By taking that action, I avoid not just accidents, but the potential for accidents.
We do this all the time while driving. we know the risks involved in an intersection, so we make, often unconsciously, multiple adjustments to our driving behavior so that if someone else does something stupid, we’ve already managed for that.
Getting cars to do THAT is what’s required, and it will avoid these “YOU HAVE NO CHOICE BUT TO KILL SOMETHING” idiocies.
If the cars can’t do that, then they should not be on the road.
There won’t be driverless cars on single lane mountain roads (or any road, for that matter) without a lot of safety-oriented infrastructure to prevent such a scenario, such as boundary fences. I also doubt if the “intelligence” of such systems would need to be programmed for such absurd levels of complexity.
Can god make a rock even god can’t pick up?
The point isn’t who lives or who dies. It’s who gets sued afterward.
Dead driver scenario – car company gets sued.
Dead child scenario – car company and driver get sued.
Both dead – car company gets sued. (Twice?)
So if you’re Honda or GM or McLaren, do you really want to unleash driverless cars unto the unwashed masses? Because the first time this scenario happens, you’re going to be sued. And again, and again, and again.
A well designed driverless car will see the child before they enter the road, anticipate the accident and slow down well before the child trips. If the car can’t see far enough ahead to avoid a sudden accident it should slow down to a safe speed.
I would argue you could start breaking even sooner than that. The moment there’s something mobile on the side of the road, the vehicle should start dropping in speed.
And no, that doesn’t mean it takes forever to get anywhere. Because in the future when cars are all driverless, they also increase in speed when it’s completely safe.
I’m not being very clear here, so let me just say: this is agreement with you. Our hypothetical situations are based on driver attention and reflexes. Computers shouldn’t be driving until they can do better. And one day, they’ll do MUCH better.
exactly. It’s why you can have completely unstable planes flying in a reliable manner, 16m off the ground at mach.
this is eventually a solvable problem.
also, you realize you’ve defined the problem completely differently than the original. unavoidable harm is rather different than unavoidable death.
In your situation, which I would point out is rather more realistic, the bias must always be towards the pedestrian. They are not protected by layers of steel and seat belt and airbags. A collision that would kill a pedestrian would cause the person in the car little to no harm at all.
With modern car design techniques, you can create a vehicle that is both safe, maneuverable, and efficient.
It is the “unavoidable death” scenario that makes the original problem so blazingly stupid, because how do you even set up a condition that will be equally fatal for both a pedestrian and someone inside a car?
Ignoring the fact that it might be difficult to have an internal debate at the moment of the accident to make a rational choice, I would program my car to kill me. If that was the only choice. But I would hope the air bags will keep me alive. No way I would ever choose to kill a child over protecting my own well-worn skin.
Now, if the choice was me or a member of Congress, well…
Nope. Gravity is object specific.
Absolutely right. And over time, we can even solve the problem of not seeing far enough: We can put a camera up in that spot that CAN see ahead, and let it relay the image.
“Now, if the choice was me or a member of Congress, well…”
LMAO Then it’s an easy call. Run him over…then back up and finish the job… 🙂
This. The first fully autonomous vehicle to kill a pedestrian will become the Trail of The Century, and it will be all feels on one side and all lidar data and cctv footage on the other side.
Your car should ideally do a background check on both you and the child.
Depending on the result, it’ll decide to kill both of you.
I don’t know about weather, but they’re currently limited to 25mph (at least the Google ones, in California).
It is not seeing far enough, it is the reaction to the scene which is important.
The human mind can react instantly but can a computer do that.
Today? No idea.
In a few years? Sure.
But if it can’t react instantly, it’s going to hit the child anyway. There’s no decision involved.
GM killed more than a dozen people to save 57 cents per car. Lawsuits are a part of manufacturing one of the most dangerous objects we use in our daily lives, and driverless cars are going to be vastly less deadly.
There should never be a time when a car can’t be driven by the folks inside. Even if in general it’s totally driverless.
And cars shouldn’t be going that fast or lack airbags etc to avoid killing passengers. Driver or driverless.
Not to mention how was that road accessible to the kid in the first place. That question needs to be in the mix as we make this move.
Unless the car has been made in such a way that you can’t drive it. Which is what many are hoping will one day happen.
Not in British Columbia. The law says “a driver
of a vehicle must exercise due care to avoid colliding with a pedestrian.” I don’t believe “due care” extends to crashing your vehicle and dying.
“just walk out into traffic with impunity”
Driverless cars won’t exempt pedestrians from the laws of physics.
Killing both is also a no decision.
I still skeptical about the reaction part anyway we will find out if the computer can react instantly.
If we can make a trash can that can calculate where thrown trash is going and move to catch it, we can probably make a car that can avoid people and angle impact to minimize damage to occupants. ( http://www.livescience.com/34431-smart-trash-can-moves-by-itself-to-catch-your-trash.html ) My biggest concern is the OS handling it all crapping out. You don’t want a blue screen at a critical moment. 🙂