back to article Should a robo-car run over a kid or a grandad? Healthy or ill person? Let's get millions of folks to decide for AI...

The question of the infamous trolley problem for self-driving cars has finally been answered – by humans. The people have spoken. Neural networks, take note... Imagine a robo-ride is about to crash into either a kid or a bunch of elderly people. It cannot brake in time, nor swerve out of the way. Where should it go? Who should …

Page:

  1. khjohansen
    Black Helicopters

    Important 'cause...

    In the time a human driver goes OH F*** <bump>...

    Your average self-driving car will have time to run face-recognition,

    browse thru your credit record & rating on social networks??

    1. GlenP Silver badge

      Re: Important 'cause...

      Exactly my opinion having been there (and you can leave out the asterisks, I got as far as Oh F <crash>).

      The reality, I believe, is that in most cases either there is insufficient time to make a decision that will materially affect the outcome or the vehicle will be out of control anyway.

      1. Spazturtle Silver badge

        Re: Important 'cause...

        Yeah, if the car has time to actually take any action it has time to break and not hit anyone. The only time a self driving car should hit a pedestrian is if they have literally jumped in front of the car. The 2014 Corvette has a stopping distance of 90ft / 27m when traveling at 60mph, or 1 second if you prefer that measurement.

        This is not actually a thing that needs to be programmed into the car.

        1. Naich

          Re: Important 'cause...

          It's pointless trying to guess what has the least harm. A child is lighter and more resilient so hitting it instead of an older person might do less harm overall. And that child might grow up to be the next Hitler or Nigel Farage, so the car might be doing us all a favour by killing it anyway. Why not just calculate the action with the lowest speed of impact and do that?

          1. jake Silver badge

            Re: Important 'cause...

            And as any parent knows, kids are built to bounce.

            1. Version 1.0 Silver badge

              Re: Important 'cause...

              And as every kid knows, Adults are always asking stupid, impossible to answer, questions. The only safe answer here is really, "Dad, should you be 'driving' a little slower?" - and you know what the retort would be...

          2. Nick Kew

            @Naich (and others)

            You're rationalising. Yes, a child (or younger adult) is much more likely to survive with no long-term harm than an older person. "Life begins at 40" is all about the stage of life where your body starts really noticeably to lose its capacity to recover from adversity.

            But these kind of rationalisations are altogether excluded from a survey posing binary questions. At best, your rationalisations put you into a survey's "don't know". Or get lost in a middling number in an "on a scale of" answer.

          3. Gordan

            Re: Important 'cause...

            "And that child might grow up to be the next Hitler or Nigel Farage"

            I call Goodwin's law. All meaningful debate endeth here.

            1. This post has been deleted by its author

          4. harmjschoonhoven
            FAIL

            Re: Important 'cause...

            And that child might grow up to be the next Hitler or Nigel Farage, so the car might be doing us all a favour by killing it anyway.

            Or the parents of the dead child decide to make another one and that child grows up to be the next Hitler or Nigel Farage.

        2. Mark 85

          Re: Important 'cause...

          Yeah, if the car has time to actually take any action it has time to break and not hit anyone.

          The "evil" pedant in me wonders about usage of the word "break" here. "Break" as in fall apart? Or perhaps you meant "brake" as in stop the car? The first was a rather amusing muse of a car disassembling itself as part of the avoidance process so have an upvote.

        3. jmch Silver badge
          Headmaster

          Re: Important 'cause...

          " if the car has time to actually take any action it has time to break and not hit anyone"

          Surely breaking to avoid hitting anyone is pretty drastic. Maybe it could brake instead :)

          1. Michael Wojcik Silver badge

            Re: Important 'cause...

            Surely breaking to avoid hitting anyone is pretty drastic.

            Explosive wheel hubs, perhaps. Or just explosive bolts everywhere.

            As I recall, this approach was tested by Top Gear and seemed quite successful.

      2. Ogi

        Re: Important 'cause...

        > Exactly my opinion having been there (and you can leave out the asterisks, I got as far as Oh F <crash>).

        Interesting, as I am currently recovering from a crash I had less than a month ago. My experience was the opposite. As the crash was happening it seemed time slowed down immensely, and I had all the time in the world to make decisions.

        My problem wasn't making a decision. As you mentioned, the problem was that I couldn't really make the car react fast enough to do anything apart from pick where the inevitable impact was going to hit (the car was already skidding so there was limited grip to do any corrections). I managed to slow it down a bit, and managed to position the car so that the impact would be as far from me as possible (front passenger side, which was unoccupied), and that is about it.

        Although I do agree, the exact words used when I realised what was about to happen was "oh F***", then had the crash. Not the most eloquent of potential final words, I admit, but it isn't like anyone would have heard it anyway.

        I only have one anecdote about having a crash (and I don't really want to deliberately put myself in that situation in order to get more datapoints), so it is interesting to hear others stories. Although I will say I never realised quite how fast 60mph actually is until I was approaching a wall at that speed with limited control. When normally driving it always felt quite slow.

        1. Davidcrockett

          Re: Important 'cause...

          My one experience of running my car off the road went like this:

          "Oooh, the back end's coming out, this will be fun"

          "I've got it, I've got it, oh s*** I haven't got it"

          "I think the car's going to stop before I hit something"

          "Oh dear, is that a wall approaching?"

          "Ahh, well that appears to have stopped me"

          I was fine, the car wasn't. All entirely my fault, and very lucky there wasn't any cars coming the other way. I suspect if the car was equipped with stability control (this was a while ago) I would have got away with it.

        2. Lee D Silver badge

          Re: Important 'cause...

          I have experienced the "time slowing" thing. People think it's an exaggeration but it's not.

          I was driving through a rainstorm at night. Had navigated to a random point on a map, so literally had no idea where I was or where I was heading.

          Emerged from a forest, into a little village, miles from anything. Only the pub was actually lit up, the rest was just houses and incidental lighting. Passed the pub, 20-25mph or so (it was seriously belting down), followed the road, and ended up with a bridge in front of me.

          Literally, I can remember my entire thought process. A sign on a pole appeared and passed the front of my bonnet. Through the rain-soaked windscreen it was tricky to make out but I saw it and my brain processed it. It was a little car. Going downhill. Into some wavy lines. I *know* I know what that means, but I can't think of it. Literally - from my brain's point of view - many, many, many seconds of debating happened as I tried to reason what the sign was. Meanwhile I drove up onto the "bridge"... Very steep this bridge. I wonder why they have a bridge in the middle of nowhere.

          And then brain finally decided that it had thought long enough and brought back reality to me. Not bridge. Harbour. Not "the road is made of bacon" but... this is the end of the harbour and you're about to plunge into the ocean. Amazing, considering I had *zero* idea I was near the ocean at all. Never pressed the brake so hard in my life and it appeared to take forever to stop - I can remember at least "ten seconds" of me just pushing the brake to try to hasten the stopping, and it not happening... after the long internal conversation to do so.

          I literally spent the next ten minutes with my car at a 25-30 degree up angle on the ramp, full beams shining off into the sky, the bottom of the beam just catching the top of 12-foot rolling waves as they smacked against the ferry-docking-ramp I'd just driven up.

          1) I can't swim.

          2) I did not know I was near the ocean, so would have been utterly unprepared.

          3) It was 12-foot-waves. No exaggeration.

          4) Because it was a ferry port / harbour there was no easy way back up to dry land even if I could get out a car that fell into water bonnet-first.

          5) It was pitch black, middle of the night.

          6) Because of the huge rainstorm, nobody would have heard a thing. The pub was shut, it just had lights on.

          7) I'd just split up with my wife and gone on a drive to escape... so nobody was coming to look for me even if I was missing.

          I sum those to equal "death", personally. It's the closest I've ever come to it.

          However, when I recovered from the more-than-slight shock, I realised several things. Including that the sign I "passed" was parallel to the passenger door. I'd barely encroached a few feet up the ramp. Given the conditions, that tells you how slow I was being anyway, but there is NO WAY I had time for the internal-conversation that took place.

          I can remember the length and detail of that internal conversation, which must have been literal fractions of a second, and it far exceeds reality. Either your brain massively overclocks in an emergency, to get more done in a short time, or something weird happens to your perception of time.

          "Although I will say I never realised quite how fast 60mph actually is until I was approaching a wall at that speed with limited control. When normally driving it always felt quite slow."

          I like to do this to people (my kid especially). Drive along normally. Pick a landmark. A lamppost. An old lady. Whatever. Now, in your head, picture what it's like to hit them as you drive... literally see how quickly they would go from being "in front of the car" to "up in the air behind you before you could even really brake". The distance you cover at motorway speeds is stupendous, but even driving along a side road.

          There's the old lady... here we go... BANG-CONTACT-FLING-SPLAT as the front/windscreen/roof/back of your car passes the point she's standing at. It's amazingly conducive to realising quite what speed does.

          1. doublelayer Silver badge

            Re: Important 'cause...

            Some studies of this effect attribute it to the brain not discarding memories from a point of intense emotion, especially fear. They have done some tests by having people fall off buildings (they were OK with it) and the brain does not appear to overclock. I don't know if anyone's found something different, but that's what I read a while ago.

            I'm glad you did not fall into the harbor. That sounds like a terrible experience.

          2. jmch Silver badge

            Re: Important 'cause...

            @Lee - great post!

            "your brain massively overclocks in an emergency, to get more done in a short time"

            Most definitely this.

            Also, we humans don't directly measure our stream of consciousness in seconds, minutes etc, but by experiences/events happening. That's why we get the impression that time flies when having fun or when highly concentrated on a piece of work.

          3. Brangdon

            Re: Either your brain massively overclocks in an emergency...

            > Either your brain massively overclocks in an emergency, to get more done in a short time, or something weird happens to your perception of time.

            From your own account, the amount of time it took you to recognise the sign suggests the second one. If your brain was faster, you'd have recognised it sooner. In fact, your brain's emergency mode did not lead to you reacting quicker. As others have said, you just remembered more detail afterwards.

          4. Anonymous Coward
            Anonymous Coward

            Re: Important 'cause...

            "Either your brain massively overclocks in an emergency, to get more done in a short time, or something weird happens to your perception of time."

            I strongly believe it's the latter. The conscious parts of our brain are a bit delayed, and often our "rational thinking" is just rationalizing things that have already happened rather than truly coming to any decisions. The conscious brain will also lie harder than Trump that this isn't the case at all and that it's totally in control at all times.

            I suspect what happened in your case is that the lower parts of your brain recognized the mortal danger you were in, and started screaming "stopstopstopSTOPSTOPSTOPDEATHSTOPDEATHSTOPYOUMORON". Your foot was on the brake while your conscious mind was still going, "Huh? What's that noise? What're you going on about now?" Then, when the car stopped, the screaming subsided into something it could understand - a sign, harbour, water, danger, death, must stop. Already stopped? What? Fake news! I'm the decider! Clearly I did the research, analyzed the situation, and decided on the best course of action. It took a while, it sure was a hard job, but I did it! Nobody else could!

            And your lizard brain rolls its eyes and says, "Sure, buddy, whatever you say. I reacted in 200ms and saved us cause that's something I was evolved to do. But you can't do SHIT in 200ms so it must've really been half a minute. Sure."

          5. Anonymous Coward
            Anonymous Coward

            Re: Important 'cause...

            I have experienced the "time slowing" thing too but I never drive when I've taken those little pieces of blotter paper.

          6. Michael Wojcik Silver badge

            Re: Important 'cause...

            Either your brain massively overclocks in an emergency, to get more done in a short time, or something weird happens to your perception of time.

            Or your mind retrospectively creates false memories of what you experienced during the moments of extreme stress. It's just as likely that your conscious cognition isn't doing much of anything useful in the moment.

            I've had a couple of near-death experiences, a couple of auto accidents, and perhaps a few other similar episodes of brief, unanticipated, highly stressful stimulus. I'm very suspicious of my memories of them. Even where there were witnesses to corroborate my recollection of the basic events, I suspect I wasn't doing much in the way of conscious thinking at the moment.

            A long tradition of psychological and neurological research (Helmholz, Libet, Damasio, etc), much of it pretty methodologically sound, casts cold water on the idea that our conscious thought processes have much opportunity to influence our decisions and actions when first responding to a stimulus. They're just too damn slow. We can hope - I do - that conscious thought processes help condition our preconscious / unconscious ones, so that future decisions and actions will tend to be what we'd do if we did have time to think about our responses. But that's about it.

        3. Anonymous Coward
          Anonymous Coward

          Re: Time to react.

          In my case this was also inconsequential. As all the time in the world won't help when my car was not on the ground to have any braking or steering!

        4. David Woodhead

          Re: Important 'cause...

          Although I do agree, the exact words used when I realised what was about to happen was "oh F***", then had the crash. Not the most eloquent of potential final words ...

          But certainly the most popular in the history of the world in those circumstances.

    2. JDX Gold badge

      Re: Important 'cause...

      Yes a computer can do many of those things. Your reaction time is measured in tenths of seconds. Not to mention, the car will ALREADY be tracking every person/object in view so it doesn't have an "Oh ****" moment.

      I do sort of agree though. A human doesn't try to decide who to hit so for this tiny edge case SHOULD the aI be making these decisions? It should probably work out who it can hit in the least damaging fashion and how many people to take out, nothing more.

      1. Flocke Kroes Silver badge

        Re: Important 'cause...

        the car will ALREADY be tracking every person/object in view so it doesn't have an "Oh ****" moment.

        Almost... More likely:

        the car will ALREADY be tracking every person in view and looking up their purchase history, credit rating and recent search topics so it can select irrelevant adverts to display on the car doors.

    3. Valerion

      Re: Important 'cause...

      The computer won't be going "Oh F**" like the human driver would.

      The computer would probably not have got into that situation in the first place. If there are bad road conditions it would have slowed down.

      It won't be distracted like the human who is changing the station on the radio or reprogramming the satnav - it will always be paying attention.

      If there are pedestrians it would already have scanned and judged them whilst they were walking along the side of the road and determined which would top its kill-list should they step out - it wouldn't do it only when they became a problem...

      1. Anonymous Coward
        Anonymous Coward

        Re: Important 'cause...

        I hope the computer in these self-driving cars is better than the one on my Toyota.

        Its road sign recognition system often detects speed limit signs down side roads and displays them as if they apply to the road I am on.

        Several times it has "detected" speed signs of 80 and 90 mph in the UK.

        It's best trick is to suddenly decide that all the UK speed signs are now in kph and warn me that I am going too fast. Then, after a few minutes, they are back in mph again.

        Recalibrating the camera made no difference and other owners have reported the same issues.

    4. Persona Silver badge

      Re: Important 'cause...

      Self driving cars are really good at maneuvering and braking the car, better than almost all human drivers. They are relatively terrible at identifying things, current worst than most drivers so we have a way to go before making value judgements about what to hit. Once we get there it's time to apply Steinbach's Guideline for Systems Programmers - "Never test for an error condition you don't know how to handle."

      1. Adrian 4

        Re: Important 'cause...

        And likely to stay that way for a long while, judging from the fragility of current automatic driving machines (I avoid the term AI as they can't really be classed as such).

        A Waymo would very likely kill the woman in a large flappy coat pushing a buggy (unrecognised shape) to protect a poster of tommy robinson ("A loathsome, obnoxious, repellent individual").

        1. werdsmith Silver badge

          Re: Important 'cause...

          The time slowing down in an accident situation is very real, I have experienced it more than once in cars and on motorcycles.

          As for self driving cars, my last car - a Nissan had some autonomous features, including forward emergency braking.

          A lady not looking my way stepped into the road in front of my car, she then looked my way and saw a car and stepped back. I had gone for the brake but the car had already started braking and pulled up about 2 metres short of the pedestrian.

          I've since changed to a Toyota with even more autonomy and love the safety stuff.

          1. onefang

            Re: Important 'cause...

            "The time slowing down in an accident situation is very real, I have experienced it more than once in cars and on motorcycles."

            Hmm, last time I came off my bike I remember thinking "You take a long time to slide to a stop when you come off at 110 km/h". It might have been time slowing, or it might just be that you take a long time to slide to a stop when you come off at 110 km/h.

            Actually, it was likely more than 110 km/h, I was doing the speed limit, the guy behind me wasn't, I may have been sped up before actually hitting the road.

          2. Mark 85

            Re: Important 'cause...

            The time slowing down in an accident situation is very real,

            Indeed it does. I rolled a drag racer at half track... managed to think and tried stepping on the brake (while upside down... brilliant), pulling the ripcord for the parachute (straightened the car out and actually slowed it down pretty quickly), and playing with the steering wheel and stomping on the brake pedal amongst other thoughts like "why is everything upside down?" and "why aren't the brakes working?". Actual time of the "event" was maybe 5-10 seconds. The mind is a strange thing at times. Luckily only damage was to the car and my sense of immortality (I was in my 20's).

            Sidebar... I stopped racing after that season.

            1. jake Silver badge

              Re: Important 'cause...

              "I rolled a drag racer at half track"

              Did you get anything good? All the drag racers I have ever known (self included) spend all their loose change on spare parts ...

              I high-sided a bike at Thunder Hill once. At about 140 MPH ... a friend & I were practicing drafting, and swapping the lead back and forth ... He cut in a trifle early and my front tire hit his rear as we were accelerating out of a sweeper. Not good with hot sticky race rubber. I remember thinking "Well THAT was a daft thing to do! This is going to hurt. Pull in your arms & legs & get ready to roll. Shit, the wedding is in a week, SWMBO is going to be PISSED! I wonder if I'll be able to get a beer in the ER? Hopefully Doug will get the bike back to the house for me." and then I hit the deck. I wasn't in the air for more than a tenth of a second or so.

    5. PassingStrange

      Re: Important 'cause...

      Not on any network near me. Signal's awful.

  2. Anonymous Coward
    Anonymous Coward

    My take from that graphic is don't steal cats.

    1. Anonymous Coward
      Anonymous Coward

      >My take from that graphic is don't steal cats.

      If stealing cats is unavoidable - make your escape in scrubs and stethoscope with the moggies securely strapped into a pram.

    2. David Nash Silver badge

      Can't steal a cat.

      Cats belong to no one but themselves.

  3. NiceCuppaTea

    Who's gonna buy it?

    So once the "moral" decision has been made and "we" have decided that if the car has a single occupant and there is a choice between killing 6 cats or the occupant of the vehicle the occupant gets it. Who's gonna buy the car that will actively choose to kill them?

    Will that be an additional feature like alloy wheels or airbags, e.g. if you buy this add-on when you spec your new vehicle the car will always try to save your life and always mow down push chairs.

    When its not your life on the line then your obviously more willing to throw the "driver" under the bus but when it comes to the choice of killing yourself or mowing down a family that you have no vested interest in then will the decision be the same?

    1. Anonymous Coward
      Anonymous Coward

      Re: Who's gonna buy it?

      That makes a good case that there needs to be some sort of standard for how it should behave that all cars must obey. You can't choose a Volvo because it'll always mow down school children to save your life, nor can you hack your Toyota to do the same.

      1. jake Silver badge

        Re: Who's gonna buy it?

        Not me.

        I know too much about large companies & programming to trust a self-driving car with my life. Fact is, they will never be safe. At least if I'm driving myself, I know who to blame. And I'll happily take that bet.

        1. AMBxx Silver badge

          Re: Who's gonna buy it?

          They dodged the really difficult question - what to do when they find that the cameras are better at detecting paler people.

          1. a pressbutton

            Re: Who's gonna buy it?

            If the car has radar, it will favour people with metal implants / fillings?

            1. Nick Kew

              Re: Who's gonna buy it?

              If the car has radar, it will favour people with metal implants / fillings?

              Tinfoil hats?

              1. Michael H.F. Wilkinson Silver badge

                Re: Who's gonna buy it?

                IR cameras would probably see all skin tones equally, but they would probably see scantily dressed people (and hotheads) more clearly. Wearing a tinfoil hat would make you less visible in IR too.

        2. Anonymous Coward
          Anonymous Coward

          Re: Who's gonna buy it?

          I'm not worried about you trusting a self driving car with your life. I'm worried about the number of people who think they are good drivers and are not, and their possible effect on other people's lives. Like that man who drove an SUV and caravan the wrong way down a motorway and killed an innocent person driving the right way.

          Some years ago a study showed the majority of US drivers thought they were above average, whereas a much smaller minority of Swedes did. The Swedish death rate per mile on the roads is less than one third of the US rate. These things may possibly be related.

          1. FrogsAndChips Silver badge

            Re: Who's gonna buy it?

            @Voyna i Mor

            Bang on.

            No 1 root cause for accidents lie between the wheel and the brakes. Eliminate that and you will make the roads statistically much safer.

            An autonomous car follows the road rules, doesn't drive when tired or under the influence, doesn't think it drives better than the other cars, and can react much faster than a human to an obstacle appearing in front of it.

            All these trolley problem variants are extremely marginal situations, and whatever their outcomes the number of victims will be peanuts compared to the lives saved by self-driving cars.

            1. Charles 9

              Re: Who's gonna buy it?

              "All these trolley problem variants are extremely marginal situations, and whatever their outcomes the number of victims will be peanuts compared to the lives saved by self-driving cars."

              Two problems. First, edge cases don't STAY edge cases. Second, these "trolley problems" raise serious questions of priority, which can never get a satisfactory answer on account of there always being a loser (and a dead one at that) at the end, and NO ONE wants to be that loser.

              1. Nick Kew

                Re: Who's gonna buy it?

                NO ONE wants to be that loser.

                So it's down to who gets the choice. Back to familiar territory now.

                But that's not quite the whole story. Not everyone who can afford a Chelsea Tractor uses one to drive little Quentin and Aurora to the school gates. And taking risks turns out to be good for you: cyclists have longer life-expectancy than non-cyclists despite a few of them getting killed on the roads.

              2. FrogsAndChips Silver badge

                Re: NO ONE wants to be that loser

                and NO ONE wants to be the victim of a drunk driver.

                Over the last decades, road casualties have largely decreased, thanks to better cars, tighter regulations on speed and drugs and a general awareness that the road is not a jungle. Now we can achieve another step by removing the human factor. There will still be casualties because no algorithm is perfect, but we can get pretty close to 0.

                As others have pointed out, stick to a few simple rules and don't try have a solution to all situations: protect the passengers (they have put their trust in the car), stay on the road, brake, don't swerve.

                Doctors face difficult choices every day, and our societies as a whole have accepted that not everyone can always be saved, however hard it is for the dead's families. Why wouldn't we accept it for cars?

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like