New work: Algorithms to give self-driving cars 'impulsive' human 'ethics' In a version of the infamous Trolley Problem, you're sitting in a runaway train on a fatal collision course with five people. You can flip a switch to change tracks, but on the other track you'd still kill one person. Now change the numbers, who the people are, pretend the trolley drives itself, and welcome to the crazy world …
<blockquote>However, he added, "I do not think that real-time, human driving decisions can provide an acceptable basis for the ethical programming of self-driving cars" because "it is not a good idea to create rules based on impulses".</blockquote>
As part of the emergency services, I was trained on how to deal with these situations from a practical point of view. The philosophical discussions are all very nice and interesting, but not useful in a real scenario. Same goes for emotion-based responses.
In our case, the general principle was "treat those to whom the treatment is most likely to be effective in terms of assuring their survival". In practice, this may mean having to treat an attacker in preference to one of your colleagues, to put an example, which can be emotionally taxing to say the least, but we were not there to play God.
Also, the so-called trolley problem and similar thought exercises are embarrassingly one-dimensional to be useful. E.g., what stops the signalman from politely asking the people gathered on the tracks to move elsewhere? Why was there people on the tracks? Why was the trolley not working as intended? Why were there not emergency brakes on the tracks? What if going for the group will likely result in multiple non-life threatening injuries as opposed to an assured casualty? Etc., etc.
That's pretty much what people do (when given enough time - 4 seconds - to think about it).
So, it's basically OK to replicate this behaviour in automation? As neither a woman or a child, I don't feel entirely comfortable about that.
What about skin colour? Or hair colour? Would most people choose to eliminate a ginger first (yes, I'm one of them too!)
I'm not sure I'm liking this RotM future.
@ Steve Button
Thinking much the same way. 4 seconds is considered opinion and would probably not match instinctive reactions. The assumption is that this is a good thing but now you may be deliberately mowing down A to preserve B which will be making lawyers salivate.
What about skin colour? Or hair colour? Would most people choose to eliminate a ginger first (yes, I'm one of them too!)
I am not convinced anyone would choose to eliminate someone first but I am sure tribalism does play a part in determining who one prefers to save. It is important to note that tribalism does not equate to racism and the like though the outcomes may often look very much like it.
The difficulty here is that there is no universal truth, no absolute answer as to who should be saved and who should be sacrificed. The best we can do is analyse whether any outcome was legitimate in the circumstances or not and even that's hard, perhaps impossible. Lab testing can be influenced by people seeking to hide or suppress their tribalism to varying degrees, consciously and subconsciously.
Has anyone analysed random choice; how 'shutting one's eyes and putting it in the hands of a higher power' works out?
That would at least absolve mere mortals being held to blame for any outcome which some may disagree with.
Time to think is the wrong approach if you want to know what people actually *do*.
Stick 'em in a driving simulator, let them drive real time for a while, and throw hazards at them in 'real life' situations... and see what they do.
Some of them won't even notice the incident and will plough straight into the victim; some will notice the ball bouncing into the road and brake before the kid follows it; most will likely try and react too late. Whichever they do, you'll have a good baseline to which the AI must merely do better than.
I am really unconvinced that people make ethical judgements in such circumstances; I do believe that they may attempt to make a choice of 'avoid hitting anything/avoid too much damage/hope the airbags work'.
The problem with the 'Trolley Problem', as usually described and as used in this article, is that only one of the two important factors is considered: the dilemma of choosing whether just one person dies or whether several people die.
The other important factor is responsibility: if you remove yourself from the scenario then the route will not be switched and multiple people will die but as you are not present then you can bear no responsibility for any outcome. If you then add yourself back in to the scenario but do nothing then the outcome is the same: the same group of people will die, just as they would when you were not present.
However, if you add yourself back in to the scenario but this time switch the route then one person, who would not have died, but for your action, will now die.
Your two options, therefore, are not symmetric; if you switch the route then you become responsible for the death of the one person because they have died as a direct consequence of your actions but conversely, if you do nothing then you can't be held responsible for the deaths of the several people because their fate would be the same as if you were not present and unable to influence the outcome.
Some countries have/are considering 'Samaritan' laws and these might be interpreted, in the 'Trolley Problem', as obliging you to switch the route and kill the one person but I don't know how they would then deal with your direct responsibility for killing the one person. And even if the Samaritan law could grant you immunity from criminal prosecution, could it also grant you immunity from civil prosecution?
"if you do nothing then you can't be held responsible for the deaths of the several people because their fate would be the same as if you were not present and unable to influence the outcome."
No, because doing nothing when you have the capability to do something is a choice in and of itself. Hypothesizing about being removed from the situation is simply wishful thinking in an attempt to abdicate responsibility, not a valid logical argument.
Being able to make a choice, and responsibility are different things. For example, some people have the ability to act, but not the knowledge.
So often, in real life, the trolley problem falls apart as people do not have any knowledge of the situation, or too little to have responsibility or to act. Such as not knowing who is in a cart/on a track or not knowing if the outcome will be worse or better... it could hit only one person, then fall off the track and hit a school of orphans. However, in many instances a "good Samaritan" law covers you if you act on the knowledge you have and act in a way to minimise damage.
So applying breaks to try and stop, if it goes wrong and derails, but you did not know it would do so, negated you from responsibility of the derailment, as you tried to slow down controlled.
@Steve Knox: "because doing nothing when you have the capability to do something is a choice in and of itself"
I believe that this is incorrect for two reasons. Firstly, having a choice does not equate to being in a position to make a choice, especially when that choice has to be made immediately and without time for analysis. Secondly, you seem to assume that everyone can make decisions easily and instantly when in reality many people find it difficult to make any decisions, let alone stressful ones; you can't simply claim that an inability to decide is a decision in itself.
Perhaps you personally find it easy to make decisions - that's fair enough for you, but if you start projecting your decisiveness, or indeed any of your personal qualities, upon everyone else you're going to end up criticising everyone else for not being you.
Firstly, having a choice does not equate to being in a position to make a choice, especially when that choice has to be made immediately and without time for analysis.
The parameters of the problem state that you are in a position to act, and most forms state that you do have time to make a choice, but not to analyze that choice. It's a snap decision, yes, but it is a decision.
Secondly, you seem to assume that everyone can make decisions easily and instantly when in reality many people find it difficult to make any decisions, let alone stressful ones; you can't simply claim that an inability to decide is a decision in itself.
This is the entire point of the Trolley Problem. It's an edge case exemplifying the line between thinking things through and snap decision making.
Perhaps you personally find it easy to make decisions - that's fair enough for you, but if you start projecting your decisiveness, or indeed any of your personal qualities, upon everyone else you're going to end up criticising everyone else for not being you.
I believe you misunderstand my point. My point is that if you are in the situation described by the Trolley Problem, then, whether you throw the switch or not, you are the only one with the capability to do so. Hence you have a burden of responsibility to do one or the other, and will personally have to live with the consequences of whatever you do.
I am not ascribing judgement on either choice, nor am I suggesting legal culpability would be a good thing in this case (as others have mentioned, the Good Samaritan laws exist specifically to prevent heaping legal trouble on top of the moral conundrum this type of problem poses.)
@Steve Knox: "I believe you misunderstand my point."
Fair enough, but I still can't agree with "...you are the only one with the capability to do so. Hence you have a burden of responsibility to do one or the other..."
The problem I have with this is the implied obligation to act: the "burden of responsibility to do one thing or the other". I'll admit straight away that my objection is more philosophical than practical but if you accept the proposition that you are obliged to act then this raises two further issues: firstly, an obligation means you have no choice, and secondly, to whom (or what) are you obliged?
It seems to me that if you are obliged to act then you're putting the need to act ahead of which act you perform: in the context of the Trolley Problem it means that it doesn't matter which action you take because the subject has changed from which action you perform to simply whether you act or not.
The second issue, of to whom or what are you obliged, is a bit of a trick question. Unless you are owned, as in slavery, then any obligation you may feel you are under can only be granted by yourself, whether it be to an individual person or to a concept, like Humanity.
The upshot is that, unless you're a slave, there can be no external obligation upon you, only the obligations you grant to others. Consequently, if there can be no external obligations imposed upon you then you can not be burdened with any responsibility to act, other than by yourself.
@Steve Knox: "because doing nothing when you have the capability to do something is a choice in and of itself"
Ah, but doing nothing, legging it out of the situation, and denying you were ever there can be the same as not being there (for legal purposes). But if you switch the points (especially if you leave finger prints/evidence) then you were there, and may face legal consequences for your choice.
And even if the Samaritan law could grant you immunity from criminal prosecution, could it also grant you immunity from civil prosecution?
It might grant you immunity from those things, but what about the human angle? In other words, the angst and perhaps guilt that you made the wrong decision even if it was made with the best of intentions? Humans are not computers and thus, these "tests" are really meaningless except to rationalize something that really can't be rationalized at an emotional level.
In other words, the angst and perhaps guilt that you made the wrong decision even if it was made with the best of intentions?
Lets take the speciest angle out of it. Imagine you're barreling along at a rate of knots, and you see a small yappy dog on the pavement, at the same time you see a rare opportunity to flatten a magpie that's on the carriageway directly in front of you, that (for once) has miscalculated.
What do you do? My money's on letting the magpie live and going up on the pavement to score the yappy dog. And bonus points if you can get the owner.
> Some countries have/are considering 'Samaritan' laws and these might be interpreted, in the 'Trolley Problem', as obliging you to switch the route
That is not correct. To quote the relevant instance I am most familiar with¹ (my emphasis):
Quiconque pouvant empêcher par son action immédiate, sans risque pour lui ou pour les tiers, soit un crime, soit un délit contre l'intégrité corporelle de la personne s'abstient volontairement de le faire est puni de cinq ans d'emprisonnement et de 75 000 euros d'amende.
Sera puni des mêmes peines quiconque s'abstient volontairement de porter à une personne en péril l'assistance que, sans risque pour lui ou pour les tiers, il pouvait lui prêter soit par son action personnelle, soit en provoquant un secours.
¹ Art. 223-6, code pénal français.
@AC: "That is not correct. To quote the relevant instance I am most familiar with¹ (my emphasis):"
Note that I said "might be interpreted" and not will be/is interpreted.
But the legislation you quote only considers one party being at risk and in the trolley problem there are two parties. As framed, this law leaves the way open for lawyers to prosecute you which ever way you act because acting to help one party means that you haven't acted to help the other party.
Of course this is nonsensical, but nonsense doesn't deter lawyers.
> Of course this is nonsensical, but nonsense doesn't deter lawyers.
Or possibly it doesn't make sense to those who are not learned?
There is no better example of nonsense than the post I am quoting. Would the gentleman like a bigger spade to continue his very illustrative endeavours?
The traditional runaway trolley-bus dilemma has it that you you see a runaway trolley-bus approaching, whilst standing near the points. You have the option of diverting it to hit one or other of two groups of people. To make it interesting, one group is usually more "deserving" than the other - possibly a choice between a group of school children or a load of drunks who've fallen asleep on the tracks.
There is, however, a third option. By throwing yourself into the path of the trolley-bus you could derail it, or at least slow it down, thus saving both groups. This option has the questionable advantage that you won't have to live with the consequences of your choice (or with anything else, come to that!). I suppose the motoring equivalent of that is where you could avoid a collision with both oncoming vehicles by steering into a concrete wall, or over a cliff. Your choice in that situation says a lot about the sort of person you are.
There is another variant of the dilemma which offers a fourth option. Here, you're still standing near the points watching the runaway trolley-bus. But this time you're a policeman who's just apprehended someone suspected (but not convicted) of committing a particularly nasty crime. You thus have the option of throwing him in front of the trolley-bus instead of yourself. (I can't think of a motoring equivalent for that)
Would have loved to take part in the experiment purely for the troll value of it.
I have not looked at the research so I do not know how they controlled for the fact that people knew they were in a simulator¹, not in a real-life situation, but just to teach them to pay attention to that, I would have mowed down the poor first, then children, then women, then minorities, then vaguely communist or homosexual looking people (unless rich), backing up as many times as necessary, while avoiding white anglo-saxon protestant males at all costs².
Come on, what next? Make political decisions based on the points of view expressed in YouTube comments?
Oh, hang on...
¹ Although simulators can be impressively convincing.
² Wasn't one of the columnists here whose mother was a psychiatrist, that used to troll psychology students in his campus by giving the "wrong" answers to personality tests?
From experience (my family were nearly wiped out) people's reactions are not always in favour of the 'minimise damage' ones. When a car pulls out from a turning on your side of the road, for example, many people will instinctively swerve to avoid it, ignoring oncoming traffic with double the impact speed. Surely the test of the decision-making algorithms is how much better do they do than the average human driver?
When a car pulls out from a turning on your side of the road, for example, many people will instinctively swerve to avoid it, ignoring oncoming traffic with double the impact speed.
It is counter-intuitive but impact speed isn't doubled, not when one sits down and works it out. Draw two cars driving head-on into each other. Erect two immovable walls between the two cars, have them both drive into those rather than each other. Where's the difference? There is none.
Of course, if the car one is following is moving rather than stationary a head-on will increase the impact speed, but not double it.
The problem with your logic is that the car that pulls out from a turning on your side of the road is NOT an immovable wall. It is all about the energy that needs to be dissapated in the crash.
Your car: E=m*V^2
Other car on the road: E=m*V^2
Car on side of road (very low speed): E=m*0^2
Thus hitting the car on the side of the road means that half the energy must be distributed over both vehicles, when compared to impacting the one in the opposite direction.
Rather than giving the cars knowledge of ethics, I'd prefer the car is given knowledge about physics.
Lots of energy can be dissapated in controlled crunching and spinning, rather than into acceleration of meatbags.
> Lots of energy can be dissapated in controlled crunching and spinning, rather than into acceleration of meatbags.
In our advanced driving training we were constantly reminded of the need to avoid a head-on at all costs, swerving off the road if need be.
And from experience as a first responder, I will take a rollover, even a drop off a cliff, over a head-on any day! Rollovers are impressively survivable in modern vehicles, as long as the occupants are wearing their seat belts properly. You should see what the victims of a head-on look like.
Well... You're correct in saying that colliding with a car coming the other way is much the same as crashing into an immovable wall. But crashing into a stationary car is not the same as crashing into an immovable wall: the car that is crashed into will move which will reduce the impact.
The first-order physical approximation of the "moving/stationary" collision is the zero-momentum frame where you model it as two cars heading towards each other, each travelling at half the speed of the original car. (Of course the moving road complicates things.)
What most people care about is reducing injuries and we would need an RTA expert to tell us if the increased injury in the "stationary" car from the lower side protection and the sideways movement of the passengers exceeds the reduction of injuries from the smaller impact. And I would guess that braking may be more effective in this case.
"When a car pulls out from a turning on your side of the road, for example, many people will instinctively swerve to avoid it, ignoring oncoming traffic with double the impact speed. "
Depending on the road the oncoming car has the possibility of swerving out the way.
What's quite possible, of course, is that the car pulling out will accelerate and catch them both. I've seen something like that except that it was the car that was pulling away from the lights accelerated towards a car that had, through no fault of the driver's, been stranded in the middle of a cross-roads.
This type of work is an exercise in intellectual masturbation (useless process that just makes those doing it feel good).
When AI really is driving cars, the AI's will eventually be able to access a boatload of data and identify who they may be about to run over (LinkedIn, Facebook, people's CV's, facial recognition, etc.) and use that as part of the value proposition on who is most expendable in these situations. In which case, the AI decisions will be based on all sorts of evaluations as to the value the potential road kill has to society, as opposed to "save the baby" or "save the most people" propositions.
And of course, society & industry being what they are, for those with the money, there will be a service offered that will instruct the AI to crash/run-over anyone else first.
@The Man Who Fell:
What if the driver AI was developed or acquired by Facebook or LinkedIn owner Microsoft? (Hmmm...I compute I cannot run that person over--they have a couple Premium services on their LinkedIn account. That toddler is looking pretty expendable though.)
"That toddler is looking pretty expendable though"
Don't worry, toddlers would kill you in a heartbeat...
After watching that video, I tried the same experiment with my 4-year-old grandson. I didn't have a toy train or Lego people handy, so I improvised with a large wooden bus and four small toy cars. I lined up three of the cars in a row and a fourth car on its own to one side. I then explained to him that the bus was going to crash and had to hit one car, or group of cars. He carefully removed the car sitting on its own, explaining "That's MY car". He then ran the bus over the other cars, singing "The wheels on the bus go round and round".
Not sure what that proves, but I suppose a claim of ownership over one of the "victims" could tip the moral balance.
Any algorithms would of course need to take account of local laws, which means in the UK the car would have to be able to tell whether the potentially colliding object was a horse, cow, ass, mule, sheep, pig, goat or dog before framing a course of action.
If we're talking about hitting people, though, I'd go for maximising the QALYs. A bit tough if the car swerves to avoid a baby in a buggy and then as a result mows down a small group of pensioners, but that's life. Or death.
I think the easiest way to sort this is with the handbrake.
Get the car sideways and take out both lanes or just drive down the middle with both side doors open.
That's that particular ethics problem solved though to be fair it then raises another more serious ethical dilemma. It's hard this self driving car stuff, I hope they get it right.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
