nav search
Data Center Software Security Transformation DevOps Business Personal Tech Science Emergent Tech Bootnotes BOFH

back to article
Kill animals and destroy property before hurting humans, Germany tells future self-driving cars

Silver badge

Lost in translation?

> Ultimately, drivers will still bear responsibility if their autonomous charabanc crashes, unless it was caused by a system failure

A system can either be autonomous, or it can have someone in control (possibly one or the other at different times). But to say that a person is responsible for an autonomous vehicle, or that a vehicle is autonomous if it has a "driver" is contradictory.

26
0
Anonymous Coward

Meat defeat module?

..driverless cars should prioritize the protection of human life, unless that life is from the International Council on Clean Transportation...

Nein?

4
0
Silver badge

Who

OK, if a crash is inevitable the car cannot discriminate on race/gender/age etc. But does it prioritize its passengers?

15
0
Silver badge

Re: Who

Not sure I see a scenario for an inevitable crash with autonomous vehicles. A human would probably have to go out of their way to make that happen. Generally the computer will slow to suit conditions and then stop if anything approaching a bad situation occurs.

4
12
Silver badge

Re: Who

That would depend very much on whether the car drives "by the book" (i.e. only at a speed for which it can stop within its sensor range) or along the lines of human drivers where it assumes the road continues beyond sight and road flow is 'normal' (which is faster and probably causes less accidents by acting in an unusual/unpredictable manner to the meatbags).

10
1
Silver badge
Terminator

Re: Who

> But does it prioritize its passengers?

One could argue that that would be unethical - They chose to ride in the killing machine, the other party is deemed to be innocent.

Car comes round a blind bend to find a human standing in the road - crash into the wall or kill?

Car comes round a blind bend to find another auto-car coming the other way (with passengers) - crash into the wall or kill?

Traffic lights broken, both cars think they have right of way - crash into the wall or kill?

Add black ice, oil on the road, "invisible" trucks and other undetectable hazards, and even the best driver or computer will get caught out eventually. So Asimovs Third law applies - the car should sacrifice itself (its airbag-protected contents are at lower risk anyway)

18
1
Silver badge

Re: Who

Like I've said before, I think autonomous cars should give priority to people who are "where they're supposed to be". So if there's a pedestrian on the sidewalk, the car should not be permitted to deliberately leave the road and kill them to avoid a worse accident. But if a person is in the road not in a crosswalk, the car may plow them over if there no other alternative (i.e. cars in the oncoming lane, pedestrians on sidewalks to either side) OK not "plow them over", it'll try to stop, but if it can't that's on the pedestrian for being where he shouldn't be.

14
5
Silver badge

Re: Who

The automatic cars fails at the fundermentals of the problem.

The problem is not driverless cars. It's not drivers. It's not automation.

It's having an "on rails" system for the vehicle. We don't have one currently. Converting to it is too expensive.

All these questions and ethics were asked and solutions (to a degree) found when the first cars appeared. The answer? Get people off the roads.

Now, the option may appear again, with people having to use bridges/crossing areas, pedestrians kept at a distance from roads etc. Such infrastructure change would not be cheap, and is comparable to trams/trains.

6
2
Silver badge

Re: Who

"Like I've said before, I think autonomous cars should give priority to people who are "where they're supposed to be". So if there's a pedestrian on the sidewalk, the car should not be permitted to deliberately leave the road and kill them to avoid a worse accident."

The US has a perverse set of regulations which were imposed by the motor lobby.

The civilised world doesn't criminalise walking along rights of way. Pedestrians are absolutely entitled to be on any section of the highway (with the exception of some roads, motorways).

A pedestrian crossing a road is an expected hazard around the civilised world (not that there are exceptions for pedestrians who step off the kerb whilst within a few metres of your vehicle, but you should still be driving slower around places where that might happen).

The US (implied from your use of the word 'sidewalk') is far from civilised when it comes to transport.

20
4
Anonymous Coward

Re: Who

"The answer? Get people off the roads."

Over my dead body.... Oh, wait a minute, bad choice of words there... :-)

But more seriously, "Get the people off the roads" is a terrible thing to do. I was horrified when visiting the US that when after following instructions to drive to a restaurant (from a rented apartment) I realised that allowing for mucking around in car parks, I could have got there just as quickly had the person giving the instructions told me to walk rather than drive, but, it being my first day there, I *assumed* the locals knew best.

For self driving cars, we need a mandatory "black box" accident recording system, which is independently engineered from the car manufacturer but which records sensor data, control inputs and all around 360 degree video. *Then* we can use that to help identify whether a cyclist, pedestrian, or someone driving an old fashioned manually driven car is at fault for doing something dangerous / suicidal, or whether there is a problem with the sensors or underlying software of a self driving vehicle. In the former case, one could easily argue that being hit by a fast moving vehicle is more than enough punishment.

In the case of a software fault being identified in a self driving car, it might mean a few million people having to drive themselves to work until a patch is agreed with the regulators...

5
1
Silver badge

Re: Who

An early experience when I first went to work in the US was to park in front of a big store then, after persuing their produce, cross the road to look in a store on the other side. I was pulled up by a policeman who considered this suspicious behaviour. Apparently "normal" people get back in their car and drive to the parking lot of the store on the other side of the road.

22
1
Silver badge

Re: Who

You are assuming all cars are autonomous and they have faultless programming. As soon as you have non-autonomous vehicles in the mix the problem of avoiding crashes magnifies enormously. As for flawless programming, well you are in an inevitable crash situation when your AI realises too late that the patch of blue sky aheah is actually the side of a light-blue truck.

7
0
Silver badge

Re: Who

"For self driving cars, we need a mandatory "black box" accident recording system, which is independently engineered from the car manufacturer but which records sensor data, control inputs and all around 360 degree video."

Why are you limiting this to autonomous cars?

It should be the case in *all* motor vehicles.

8
3

Re: Who

Not specified... I guess that if collision with pedestrians is unavoidable, the passengers will in any case be much safer.

Another question raised... Does another car count as being property or is it assumed that people are inside? It *should* prefer to crash into other cars rather than pedestrians because at least the other cars' passengers are a bit protected.

3
1

Re: Who - "invisible" objects

"your AI realises too late that the patch of blue sky aheah is actually the side of a light-blue truck."

The Tesla accident with the white truck as with other "invisible" cars could be easily avoided by mandatory markings/GPS location, like reflective bands or rear reflectors are today (but adapted for autonomous viewing systems). This will go a looong way towards making autonomous driving easier and safer.

Come to think of it, pedestrians should also wear visible means, something that is sadly overlooked today in the name of fashion. When I see bicycle riders and pedestrians crossing in all black clothes at night in the rain I'm asking myself if they are intentionally suicidal or simply do not care to live.

5
8
Silver badge

Re: Who - "invisible" objects

"When I see bicycle riders and pedestrians crossing in all black clothes at night in the rain"

Can I just point out the irony... "When I *see*". People in suits aren't instantly invisible, I don't lose my feet because I've put black socks on...

There is no requirement in law (or sense) for a pedestrian (or a fallen tree, or a concrete block, a landslide or a cow) to be wearing something which you might consider a convenience to you. The requirement for cyclists is that the vehicle (not the rider) shall have a red reflector to the rear, as well as a legally compliant light. Note that these can be as dim as 4 candela, and flashing, 50% cycle, at 2-4Hz. Pedal reflectors are *sometimes* mandatory (older bikes are exempt).

Although as you have already pointed out - you can see these people dressed in black, so what's the problem... The problem is that you don't want to drive in such a manner that you can stop in the distance you can see to be clear...

18
7
Silver badge

Re: Who

My exact thought.

An accident on the main street where our cul-de-sac leads comes to mind. I saw it with my own eyes around 10 years ago. Two teenage tw*ts were so engrossed in the conversation with each other that they pulled out less than 5 meters in front of a brand new E-class Merc. They were in a late 80-es (J reg I think) vintage Ford Fiasco rusty banger so if the Merc would have tried to brake it would have hit the driver door straight on. That would have been the end of at least the driver if not both of tw*ts, despite the Merc keeping to the posted 30 mph speed limit. The Merc driver threw the Merc onto the sidewalk where it ended up smashed onto a lamppost. The two idiotic tw*ts ignored him and continued to drive off (someone stopped them a mile down). The Merc driver got out visibly shaken, but without a single scratch.

Would a computer make the same life (or possibly death) decision facing the same situation? I doubt it. It will most likely attempt breaking and kill a person as a result.

5
1
Silver badge

Re: Who - "invisible" objects

"There is no requirement in law for a pedestrian (or a fallen tree, or a concrete block, a landslide or a cow) to be wearing something which you might consider a convenience to you."

True, there is no requirement in law, but you are going against guidance in the 'Rules for pedestrians' section of the Department for Transport Highway Code if you are not making an effort to make yourself visible whilst walking beside a public road. So it's not just a case of it being for vehicle users' convenience:-

Rule 3

Help other road users to see you. Wear or carry something light-coloured, bright or fluorescent in poor daylight conditions. When it is dark, use reflective materials (eg armbands, sashes, waistcoats, jackets, footwear), which can be seen by drivers using headlights up to three times as far away as non-reflective materials.

8
0
DJO

Re: Who - "invisible" objects

I see bicycle riders and pedestrians crossing in all black clothes at night in the rain I'm asking myself if they are intentionally suicidal or simply do not care to live.

Imagine if you will a curving narrow road with huge trees and bushes on either side making the road effectively a green tunnel with mottled sunlight.

Now imagine you are employed to cut the hedge.

What to wear while standing in the road? Nice reflective high-visibility jacket? Well in that exact circumstances I came across a trendy moron who thought that a camouflage jacket and matching trousers were the ideal outfit, if his hedge trimmer hadn't been bright red I would never have seen him and I would definitely have squished him.

7
0

Re: Who

Would a computer make the same life (or possibly death) decision facing the same situation? I doubt it. It will most likely attempt breaking and kill a person as a result.

In that instance, the computer guided system removing a death trap from the road, and converting the driver into a Darwin award contender, could be seen as progress!

0
0
EBG

really ?

So the AC control system is gong to be perfect - just like the rest of IT.

2
0
Silver badge

Re: Who

The natural progression of self driving cars is that in the future people will own cars without having driving licenses.

For me, there's no point owning a self driving car until it can legally drive me home from the pub when I'm absolutely steaming.

3
0
Silver badge

Re: Who - "invisible" objects

TitterYeNot

It's in there because it is purely for the convenience of motorists - as is much of that document. The UK is a pretty heavily car biased society (not quite as bad as the USA yet, but we're getting there).

Have you checked out Rule 126:

Drive at a speed that will allow you to stop well within the distance you can see to be clear.

That's not - can't see that there is an obstruction - it's *can* see that there *isn't*.

Around most towns and cities, and most major roads there is sufficient street lighting to make people visible whatever they are wearing. On minor roads the addition of your own headlights, and the fact that the pedestrian would be walking in such a way as to see you coming, change things a bit - but you should still be able to see a pedestrian from quite a distance.

The issue is almost never one of visibility - it's one of attention but the person with the lethal weapon.

My observation of the behaviour of a significant minority of motorists is that they treat the speed limit as a minimum speed, with anywhere up to ten miles and hour above being considered 'reasonable'. If they come across anyone doing even 'only' the speed limit they will overtake whether or not they can see far enough to complete the manoeuvre safely. They will ignore the double white lines on the road in order to do so...

In the context of people who simply don't look down a road when they take actions behind the wheel of a car it is pure victim blaming to suggest that 'they weren't wearing stuff' when the reality is that (many) motorists just don't look - or rather that they look, but only for other motor vehicles, not for clear tarmac.

The attention video is well worth a watch...

https://www.youtube.com/watch?v=vJG698U2Mvo

3
2
Silver badge

Re: Who

I was pulled up by a policeman who considered this suspicious behaviour. Apparently "normal" people get back in their car and drive to the parking lot of the store on the other side of the road.

I run into this all the time. A preference for walking is highly unusual in the US.

On a related topic, you should have seen my parents faces when I told them I was excited about moving closer to town because it would put me within cycling distance of a lot of places I frequent. They thought I'd lost my mind when they realized I considered a store just under mile away to be within "cycling distance". And I thought they were going to have me committed when I told them that it was actually within walking distance if I wasn't in a hurry.

4
0
Silver badge

Black boxes

I'd be very shocked if this isn't a requirement, and even if it isn't I expect that every vendor of autonomous cars would install it anyway because they'd want to make sure their product doesn't get blamed for something that was someone else's fault.

My point above about prioritizing people who are "where they're supposed to be" wasn't intended to say "people don't belong on the roads". If in a given locale it is legal to walk in the road then the car will need to take that into account as that person is "where they're supposed to be". It all depends on the laws/customs where the car is - just like traffic laws, speed limits, etc. are different in different places. The reason I brought it up was to suggest it doesn't have to be complicated - you don't need the car to make moral decisions about killing three people in the car versus one person walking alongside the road. If they are where they're supposed to be, the car isn't allowed to go there and run over him to save the three people in the car. Simple.

2
0
Silver badge

Re: Who

"Generally the computer will slow to suit conditions and then stop if anything approaching a bad situation occurs."

I tend to agree. There are _very_ few "unforseeable accidents" on the road. It usually takes at least 3 bad decisions on the part of multiple people to cause a crash, even when one of the decisions may be bad roadway engineering.

Humans have a tendency to "press on regardless" and vastly underestimate both their reaction time and stopping distances whilst vastly overestimating situational awareness and driving ability.

As one example, kids never "just appear from nowhere" despite many claims to the contrary. A properly trained driver will not only spot the movement of feet under a parked car ahead but be slowing down long before the child steps out into the traffic lane. Computers don't suffer from psychological tunnel vision and will be in a better position to see hazards coming from all angles.

2
3
Silver badge

Re: Who

"Traffic lights broken, both cars think they have right of way - crash into the wall or kill?"

In most parts of the world when "traffic lights are broken" you are required to either treat it as a 4-way stop (USA) or as a roundabout. The fact that older drivers get confused shows they don't know the road rules.

3
1
Silver badge

Re: Who

"It should be the case in *all* motor vehicles."

It's mostly there already. All cars with airbags have significant levels of black-box functionality in the airbag computer. This data can and has been used for prosecutions and by insurance companies to deny payouts.

1
0
Silver badge

Re AC: "Is a horrible thing to do"

I don't agree with it. But it is the only method short of perfect programming AI. How well do you think we will do in programming AI vs making safer roads?

If "get people off roads" is not a nice idea, then "make roads specifically for AI cars". But Elon Musk already knows this, and is planning on moving them all underground.

1
0
Silver badge

Traffic light failure

When they fail they flash red in all directions, it is hardwired into them. If the power is lost and the battery backup is exhausted they'll be off, that's really the only potentially dangerous time (i.e. if you don't see them / don't know they're there and think you have the right of way) Cars would be programmed with maps that know where the traffic lights are, so if they can't "see" the lights they would know something is up and could stop and "look both ways" before proceeding. They'd likely handle that particular situation better than some people.

0
1
Silver badge

Re: Traffic light failure

To be fair it'd be pretty damned hard to handle it worse than some people.

0
0
Silver badge

Kill animals and destroy property before hurting humans

Glad you cleared that one up. From the headline I thought it might be a methodical instruction by German car manufacturers to kill everything else first and then the humans.

19
0
Anonymous Coward

"If an accident is unavoidable, the self-driving ride must not make any choices over who to save"

How do you program that?

First you must assign an identifier to the meat sacks, so are we saying put a random number generator in there? I think it will done using probability of saving life but then what if it's a black or white person are you supposed to adjust the numbers so you are not picking one group over another more often?

Also best to keep this quite from Peta.

3
1
Silver badge
Facepalm

Well if Google are behind the identification algorithm then sadly any black pedestrians are out of luck:

http://www.theregister.co.uk/2015/07/01/google_photos_app_machine_learning_fail/

6
0
Bronze badge

Next....

Calls for vegan cars that change those rules....

3
2
Silver badge
WTF?

Picture

Nice picture; injured or possibly dead person lying unattended in the road, while a police officer takes a statement. Presumably with a fully automated vehicle the officer wouldn't even bother attending, and just download the logs.

7
0
Silver badge

Re: Picture

Given the obvious amount of damage to the vehicle from the parts strewn about, I'm wondering what that person must have been made of.

2
0

"If an accident is unavoidable, the self-driving ride must not make any choices over who to save – it can't wipe out an elderly person to save a kid, for instance. No decisions should be made on age, sex, race, disabilities, and so on; all human lives matter."

Great! Now what about amounts? Does 2 adults trump a child? Does say the POTUS or the Queen trump a us ordinary folk? What about the car driving itself off of a cliff to avoid a crash with a pedestrain, is that OK as there is a chance the occupants might survive if there are lots of airbags? What about it there is a horse and rider? Will the car plough into the horse, kill the animal and hope the rider lands on a soft crumple zone?

"Ultimately, drivers will still bear responsibility if their autonomous charabanc crashes, unless it was caused by a system failure, in which case the manufacturer is on the hook."

Sorry? Am I responsible if I am a passenger on the underground? I'm pretty sure I'm not. If it is an autonomous vehicle then I have zero control and should not be held responsible if there is an accident. If the car is driving itself then by definition I am not the driver.

13
0
Silver badge

Does 2 adults trump a child? Does say the POTUS or the Queen trump a us ordinary folk?

By my calculations five monkeys riding dogs is exactly equal to a street mime being attacked by a bear. You should be able to extrapolate from there.

20
0

I think the El Reg standard units need updating.

7
0
Silver badge

"Great! Now what about amounts? Does 2 adults trump a child?"

Not sure why you bring up things like the Queen, given that's already covered by the "not discriminating against anyone" part - if it's a choice between hitting the Queen or hitting a single other human, the car isn't allowed to choose based on the fact that one of them is the Queen. But as you note, there are an awful lot of situations that aren't a simple 1 vs. 1 choice, and automated cars will inevitably be required to discriminate between who they kill based on some criteria. Blindly legislating that they're not allowed to make a choice over who to save can't prevent situations arising that will require exactly that.

Of course, the bigger problem is that this isn't a question people agree on in the first place. It's all just variations of the Trolley Problem - a runaway train is heading towards 5 people on a track, if you pull a switch you can divert it to a different track with only 1 person standing on it; do you pull the switch or not? In this bare problem, most people will say you should pull the switch, but not all even agree on that. Throw some complications into the mix and things get, well, complicated. Basic complications such as changing the number of people or saying the lone person is your partner, the 5 are clones of Hitler, and so on, can be relatively easy to deal with if you agree on the base problem, and the latter two are the sorts of things that would be covered by the recommendations given here. But things like removing the lone person from the track and instead saying you have to personally murder them in order to reach the switch can make huge differences in how people view the problem, and that's exactly the sort of scenario that's most analogous to the issues autonomous cars will face.

Long story short, we're trying to decide how autonomous cars should behave without actually agreeing on how humans would or should behave in an identical scenario, and in the absence of a universally agreed objective morality, it's not going to be an easy issue to solve.

2
0
Silver badge

My problem is with the "destroy property" first bit since it would imply that it would be able to differentiate between a person pushing a shopping cart and another pushing a baby carriage/pram. Does it take out the leading "property" in an effort to save the obvious person in each case? What about instances like cos-play where someone may be dressed as a horse; does it save superman and hit the "horse"?

If an accident is unavoidable...

Sorry, this is one that irks me a bit so pardon the tangent. I question whether the vast majority of "accidents" are actually accidents. Certainly when the miscellaneous deer, turkey, etc. darts out in front of your car it is an accident but in general when another human is involved it is far more likely that one party is being negligent. It could be the prat who is over tired and falls asleep at the wheel, is too busy playing pokemon on their phone to look up, or is otherwise simply being a dumbass whether inside the car or not. I'll even allow that since we expect some humans to be dumbasses such as the small child who rashly chases a ball into the street because they don't know better and so would fall into the deer category because they're aren't yet fully aware of concepts like responsibility and negligence although perhaps a case could be made against their parents.

A crash resulting from mechanical failure could be an accident if the failure is of the type that isn't foreseen such as an unnoticed defect in a control arm allows it to separate and the wheel falls off, a highly unlikely event. Mechanical failure would be negligence if it was a wear item that wasn't replaced in a timely manner such as the brake pad that's been screaming for several months and has now ejected the cylinder from the caliper resulting in a total loss in hydraulic pressure in the circuit; something I've actually seen when I worked at a service station1 in my youth.

Essentially if we can eliminate the negligence parts we could greatly reduce the number of crashes and we'd be left with the very small number of true accidents that need to be addressed. Unfortunately autonomous vehicles can't do anything about the haphazard dolt who is engrossed with their current texting session and either driving a current vehicle or even walking.

1. For the younger folks, a service station is where someone would rush out to your car when you pulled up to the fuel pump and they put fuel in your car, washed the windows, and checked the oil. The were also capable of performing other repair services on automobiles ranging from anything as simple as a tire repair or changing the oil to rebuilding a transmission. Most of them now only sell soda, chips, coffee, etc. as well as leave you to filling your car yourself unless you find yourself in New Jersey where, if you're lucky, the "attendant" might remember to reinstall the filler cap.

0
0
Silver badge

"If it is an autonomous vehicle then I have zero control and should not be held responsible if there is an accident."

Yep, was reading about the new Audi A8. Audi chappy said....

"If you're offering Level 3 (autonomy), we are in charge of the driving part. That means, of course, if we have driven, then we are responsible."

So Audi bloke reckons it can be their fault if it crashes. Which I think is totally fair. If I'm still responsible when the 'puter is driving, then I'm going to be doing the driving thanks, not the computer.

5
0
Bronze badge
Unhappy

@Eddy Ito

And they would check your tyre pressure, top them up if necessary; top up your washer fluid (in later years); take your cash and give you proper change without you having to count it out for them.

0
0
Anonymous Coward

According to British laws concerning driving, cats aren't animals. Only dogs, goats, horses, cattle, donkeys, mules, sheep and pigs are recognised.

That said, I once rang 999 after hitting a deer, and the coppers who turned up thanked me because at that time of year deer were often found in pairs: if one had jumped in front of my van then its mate night do the same to a smaller, more vulnerable car. My motives were that I hated seeing it suffering but its antlers dissuaded me from getting close enough to brain it.

8
0
Silver badge
Go

From somewhere on the Internet (possibly even here some time past, I didn't record the source, sorry).

Just need to tweak the wording a bit:

" I found the 3 Laws of Robotics of interest but David Langford’s version captures my position more accurately:

" 1. A robot will not harm authorized personnel but will terminate intruders with extreme prejudice.

" 2. A robot will obey the orders of authorized personnel except where such orders conflict with the Third Law.

" 3. A robot will guard its own existence with lethal antipersonnel weaponry, because a robot is bloody expensive. "

15
0
Bronze badge
Terminator

" 4. All robots will have a copy of the original robocop movie and all the terminator movies in order to guard its own existence better. "

10
0
Anonymous Coward

" 5. Robots must at all times use the phrase "bidi bidi" to throw humans off the scent."

9
0

6. All robots must complain about being cleaned with a brillo pad, and express a preference for a car wash .

4
0
Bronze badge
Happy

7. All robots must reply with "by your command" when responding to human direction.

0
0

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

The Register - Independent news and views for the tech community. Part of Situation Publishing