back to article Fatal driverless crash: Radar-maker says Uber disabled safety systems

Uber reportedly disabled safety systems on the autonomous Volvo XC90 that killed a pedestrian Stateside last week, according to the makers of the car's sensors. "We don't want people to be confused or think it was a failure of the technology that we supply for Volvo, because that's not the case," Zach Peterson, a spokesman for …

Page:

  1. Daggerchild Silver badge

    Cause of Death: Ostrich Algorithm

    I'm wondering if the bike frame reflected a funky signal the software decided was impossible, and discarded, leaving a 'perfectly empty' road ahead...

    1. Anonymous Coward
      Anonymous Coward

      Re: Cause of Death: Ostrich Algorithm

      Or perhaps it was because Uber (reportedly) disabled safety systems.

      1. Steve K

        Re: Cause of Death: Ostrich Algorithm

        I think they disabled the standard Volvo safety systems because they were using their own and - as they are not integrated - did not want them to interfere with their sensors.

    2. Amos1

      Re: Cause of Death: Ostrich Algorithm

      I'm thinking it occurred because SOMEONE WALKED IN FRONT OF A MOVING CAR AT NIGHT THAT HAD ITS HEADLIGHTS ON.

      1. Amos1

        Re: Cause of Death: Ostrich Algorithm

        a.k.a. "You can't patch stupid."

        1. Someone Else Silver badge

          Re: Cause of Death: Ostrich Algorithm

          Stated another way: You can make it foolproof, but you can't make it damnfoolproof.

      2. Alister

        Re: Cause of Death: Ostrich Algorithm

        I'm thinking it occurred because SOMEONE WALKED IN FRONT OF A MOVING CAR AT NIGHT THAT HAD ITS HEADLIGHTS ON.

        I'm thinking you're an idiot.

        She was more than halfway across the road, which means she started crossing when the car was a long way away. The car was exceeding the speed limit, and so she probably misjudged the time she had to safely cross. She also probably assumed that the car would slow enough to let her get to safety as a human driver would do.

        1. Anonymous Coward
          Anonymous Coward

          Re: Cause of Death: Ostrich Algorithm

          She might have been stupid to cross the road when/where she did, but it was inexcuseable for self driving car to hit her. Even if it only could see as well as the front facing camera (where she seemed to appear out of nowhere) it should have been able to stop. It didn't even apply the brakes!

          I think anyone testing self driving cars involved in an accident so ridiculous as this one should be banned form testing self driving cars on US roads for a period of two years. That would set them far back, thus giving them incentive to be REALLY sure they have things working well in internal testing before they decide they're ready to move onto public roads.

          1. Anonymous Coward
            Anonymous Coward

            Re: Cause of Death: Ostrich Algorithm

            "I think anyone testing self driving cars involved in an accident so ridiculous as this one should be banned form testing self driving cars on US roads for a period of two years. That would set them far back, thus giving them incentive to be REALLY sure they have things working well in internal testing before they decide they're ready to move onto public roads."

            Your 'solution' will only lead to many more deaths as people are killed by drunk, tired, high, distracted, or incompetent drivers - people whose lives could have been saved by an earlier introduction of autonomous vehicles.

          2. Sven Coenye
            Boffin

            Re: Cause of Death: Ostrich Algorithm

            As to why she was crossing there, see the overhead of the accident spot: https://www.google.com/maps/@33.4362155,-111.9424977,163m/data=!3m1!1e3 . The big X in the median makes it very enticing to cross there if your in that area as otherwise you have walk to the stop light, cross, then double back; or pedal up W Lake View Drive, then to the intersection, and then down.

            The original police report re. the vehicle's speed (doing 38 in a 35 mph zone) was not correct. If you walk Google Streetview back from the accident scene, you'll find a 45 sign just before the overpass. The limit before the overpass is indeed 35 (walk Streetview back to before the river to find that sign), so Uber's car was accelerating but within the limit as it struck Herzberg.

        2. Anonymous Coward
          Anonymous Coward

          Re: Cause of Death: Ostrich Algorithm

          "I'm thinking you're an idiot.

          She was more than halfway across the road, which means she started crossing when the car was a long way away. The car was exceeding the speed limit, and so she probably misjudged the time she had to safely cross. She also probably assumed that the car would slow enough to let her get to safety as a human driver would do."

          =

          1. The car was not greatly exceeding the speed limit. The speed was over the limit by a few miles an hour - probably less than a human driven car on an empty arterial road.

          1b. You cannot estimate the exact speed of an oncoming car, so you don't count on being able to do so.

          2. If there is one car coming on a mainly empty road, and there is any question of crossing in time, you wait for it to pass. If there are several cars you wait for a gap. If there are many cars you find an inherently safe crossing or become very patient. The video did not show a lot of cars.

          3. If you misjudge the speed of the car, and it arrives faster than you expect, you don't walk in front of it, you stop, or even back up.

          4. You never assume a driver will see you on an unlit road at night.

          5. You never assume what a driver will do if they see you unless they give some indication to you of their intent.

          6. You never assume a car will yield the right of way, unless mandated by a light, sign, crosswalk, or police officer. Even then, be ready for something else.

          1. AdamWill

            Re: Cause of Death: Ostrich Algorithm

            These are all good principles to apply if you're a pedestrian interested in staying alive. However, in the case that a pedestrian *doesn't* apply them, that does not automatically excuse the driver (human or robot) that hits them, if they could have reasonably avoided doing so. Death is not the legally mandated punishment for jaywalking, nor is Joe Random Person/Robot Driving By the legally mandated agency of punishment for jaywalking.

            1. Kevin McMurtrie Silver badge

              Re: Cause of Death: Ostrich Algorithm

              Jaywalking is ignoring a red light, short-cutting a crosswalk (painted or implied), and not yielding to cars/bikes already in your path. Crossing the middle of a road is, of course, legal if it's clear when you started. You can't fault somebody in AZ for walking in the middle of the night either, given that some days are hot enough to kill you.

              Others driving the route at night (without artificially dark video) have shown that a pedestrian should have been visible long enough to make a graceful stop. Besides, it's against the the law (and common sense) to drive at a speed where can't avoid an obstacle in the road.

              1. Alister

                Re: Cause of Death: Ostrich Algorithm

                Others driving the route at night (without artificially dark video) have shown that a pedestrian should have been visible long enough to make a graceful stop.

                Exactly. See here

                There seems to be an erroneous belief centered on the Uber video that the accident happened in the dark, whereas the fact is it the road was well lit.

                1. John Brown (no body) Silver badge

                  Re: Cause of Death: Ostrich Algorithm

                  "Exactly. See here

                  There seems to be an erroneous belief centered on the Uber video that the accident happened in the dark, whereas the fact is it the road was well lit."

                  Thanks for that. It's a very well lit area and I can no reason for the "superior" sensing ability of this car to have not seen the victim in time to stop unless there was something very badly wrong with the sensors or the data processing. She was about 2/3rds of the way across the road when she was killed. It was obvious she was a "hazard" from the cars point of view. It almost seems as though, as someone mentioned elsewhere on these forums, that the cars systems can't anticipate a collision, merely react to an imminent one (and it doesn't even seem to have reacted in this case)

              2. Alan Brown Silver badge

                Re: Cause of Death: Ostrich Algorithm

                "Jaywalking is...."

                A uniquely American concept created by a century of lobbying by the motor industry, along with a raft of other state-level laws that mandate that pedestrians shall yield to motor traffic other than at designated crossing points. (Ie, the best laws that money can buy)

                The car didn't react to the pedestrian because it wasn't expecting to see one at this point and it wasn't expecting one because it wasn't programmed to expect one. In any other country it would be expecting pedestrians or other obstacles at all times. Uber's level of FAIL is more epic than a Cecil B de Mille Bible movie.

                Humans might not expect to see an obstacle here, but they will process the exception and react to the error condition.

            2. Dodgy Geezer Silver badge

              Re: Cause of Death: Ostrich Algorithm

              No one said it EXCUSED them. It was a feature of the accident - nothing more...

          2. This post has been deleted by its author

        3. This post has been deleted by its author

        4. Miss_X2m1

          Re: Cause of Death: Ostrich Algorithm

          Actually, the speed limit was 45 mph, and the car was doing 40 mph however, at that speed, in near total darkness, using its lowbeam headlights, it was still ovedriving it's headlights and even a human driver in full control would not have been able to stop in time to avoid hitting the woman. It's all about timing, braking distance, road conditions, etc. Essentially, the car was driving blind and a human driver would have the same issue.

          1. Anonymous Coward
            Anonymous Coward

            Re: Cause of Death: Ostrich Algorithm

            "Essentially, the car was driving blind and a human driver would have the same issue."

            Or a human paying attention would have bothered to put the main beams on.

          2. John Brown (no body) Silver badge

            Re: Cause of Death: Ostrich Algorithm

            "Essentially, the car was driving blind"

            So all this stuff about night vision and LIDAR is a crock from Uber and it;s partners?

      3. a pressbutton

        Re: Cause of Death: Ostrich Algorithm

        IMO

        all the posters going into great detail about how it was the human's fault for crossing the road in front of an autonomous car

        - and -

        all the posters going into great detail about how it was NOT the human's fault for crossing the road in front of an autonomous car

        all miss one big thing:

        In a public environment, Humans should not be killed by autonomous things ever.

        Autonomous cars are there for the benefit of humans, not the other way round.

        1. Dodgy Geezer Silver badge

          Re: Cause of Death: Ostrich Algorithm

          ...In a public environment, Humans should not be killed by autonomous things ever.

          Autonomous cars are there for the benefit of humans, not the other way round....

          Very good. Now consider this:

          National Highway Traffic Safety Administration (NHTSA) 2016 data shows 37,461 people were killed in 34,436 motor vehicle crashes, an average of 102 per day, in the US.

          That makes an average of one death per 94m miles driven. I understand that autonomous cars currently have one death per 130m miles. So if we changed all cars to autonomous ones at the current state of the technology, we would drop the deaths to 27,087 - saving over 10,000 deaths per year.

          Why do you think saving 10,000 lives a year is not for the benefit of humans?

          1. Anonymous Coward
            Anonymous Coward

            Now consider this:

            An Ars Technica article pointed out that based on miles driven, Uber's self driving cars now have a much higher fatality rate than human drivers. With a sample size of 1, I know that is meaningless statistically.

            But consider that their drivers are having to manaully intervene evry 8 miles, and you have a system that 1) Doesn't appear to even close to those of Waymo and Cruse(Sp?) and 2) Requires an alert and involved human behind the wheel - which seems to be lacking in this case.

    3. Daggerchild Silver badge

      Re: Cause of Death: Ostrich Algorithm

      Wow. Okay that didn't get the response I expected..

      The 'discard the impossible' coping strategy is something that satellites, spaceships and rockets have to do, because they *can't* just blue-screen and sulk when something goes funky - they *must* try and continue. They are forced to make a realtime judgement call over which source of contradictory sensory data is 'wrong', before they can decide how to react.

      The problems come when they guess wrong..

  2. Anonymous Coward
    Anonymous Coward

    Public roads are not test tracks

    From the referenced Bloomberg article: "the ability to detect and classify objects, is a challenging task"

    Then those vehicles should not be on the road. If competent government were at work, a number of Uber managers and engineers would be perp-walked by now.

    1. Anonymous Coward
      Anonymous Coward

      Re: Public roads are not test tracks

      "...a number of Uber managers and engineers would be perp-walked by now."

      No. Uber has a sterling reputation of adhering to only the highest ethical standards.

      1. Sgt_Oddball
        Coffee/keyboard

        Re: Public roads are not test tracks

        You sir, owe me a new keyboard......

    2. skeptical i
      Thumb Down

      Re: Public roads are not test tracks

      re: "If competent government were at work"

      Nope, it happened in Arizona at the behest of Governor Doug Ducey.

  3. sjsmoto

    Uber wrote the software? I'm surprised the car didn't erase the video and hide the body.

    Dear Uber: Do you want to make sure your driverless cars are really reliable? Then have a dozen of them circling your executive parking lot 24x7 at 100 mph.

    1. Anonymous Coward
      Anonymous Coward

      Then you'd read an article about how Uber execs are now commuting to work via helicopter.

    2. Anonymous Coward
      Anonymous Coward

      Uber wrote the software?

      "Uber wrote the software? I'm surprised the car didn't erase the video and hide the body."

      No, but it did increase the Uber using percentage of the population. The AI is working a little better than expected, that's all.

  4. Whiznot

    Self-driving cars will always be inferior to good human drivers because they cannot think. Humans are able to anticipate potential risks. Self-driving cars can only react.

    1. Francis Boyle Silver badge

      Better version

      Self-driving cars Uber executives will always be inferior to good human drivers humans because they cannot think. Humans are able to anticipate potential risks. Self-driving cars Uber executives can only react to the lure of profit.

    2. John H Woods Silver badge

      Re: "Self driving cars can only only react"

      Reacting may well have been enough. One in three adult pedestrians die in a 40 mph collision with a modern car, on in eight at 30, one in 25 at 20.

      A volvo xc90 can stop from 100km/h in 36 metres in good conditions. This car was I 2/3 of that. It looks from the video that the pedestrian was directly in the path of the car for a full 2 seconds, I reckon 40m. A good driver would have been able to brake for at least one second, with every 0.1s reducing the speed by 1-1.5mph. A self-driving car should have been almost stationary on impact.

      This was not a failure of anticipation, but one of detection and reaction... If an autonomous car cannot detect an object in its path it is absolutely unfit for purpose.

      1. Noel Morgan

        Re: "Self driving cars can only only react"

        I timed the video and from when the pedestrian appears to when the video stops seems to be closer to 0.75 seconds rather than 2.

        Not sure I would have noticed, reacted and braked to a 'safe' speed in that time.

        The failure is not in autonomous cars per se, something went wrong here in that the 'extra' senses that the car should have did not operate or were turned off.

        1. Alister

          Re: "Self driving cars can only only react"

          I timed the video and from when the pedestrian appears to when the video stops seems to be closer to 0.75 seconds rather than 2.

          You cannot base any judgement on the video, as it is of such poor quality that it is in no way representative of reality. Human vision would have detected her much, much earlier.

          Don't forget, the lady didn't just suddenly step out in front of the car, she had left the median strip and already crossed one lane, and was nearly half-way across the second lane before she appears in the video.

          However, I agree that this shows that the either detection systems on the car were inadequate, or the software inexplicably decided not to brake or avoid the obstruction.

          1. Mark 85

            Re: "Self driving cars can only only react"

            Human vision would have detected her much, much earlier.

            Human vision would have detected her much, much earlier if the driver had been paying attention to the road in front of the car instead of fiddling with whatever....

            FTFY as this happens a lot without a self-driving car involved. Driver distraction is a serous problem and will get worse with driverless cars. Once us old geezers have died off and the generation raised around these cars takes over the car will need to be 100% perfect or accidents will continue.

          2. Anonymous Coward
            Anonymous Coward

            Re: "Self driving cars can only only react"

            "You cannot base any judgement on the video, as it is of such poor quality that it is in no way representative of reality. Human vision would have detected her much, much earlier."

            You have absolutely no way to know that.

            You do not know the sensitivity of the camera - some cameras see much better at night than the human eye, some do not, and some are about the same. In addition, the person becomes visible as the illumination increases. You do not know the pattern of the light emissions in relation to the movement of the person, which means the illumination fractions of a second before the person appears in the video could have been much lower - too low for human or machine perceptions set up to register moving objects while travelling to see someone - remember that time exposure and motion do not play well together.

            You are just making assumptions, when facts are needed.

            1. Alister

              Re: "Self driving cars can only only react"

              You have absolutely no way to know that.

              The facts are that contrary to the widely held belief, the place where the accident happened was not a dark country road, it was a well lit urban street. The video footage released by Uber shows a very misleading view of the available light levels.

              If you look here then you might begin to understand that the pedestrian would have been in plain view for a long time before the accident.

              You are just making assumptions, when facts are needed.

              No, I'm actually looking at the available evidence instead of accepting things at face value.

            2. Anonymous Coward
              Anonymous Coward

              You have absolutely no way to know that.

              Yes, he does. Other people have posted video of the same location at the same time of night from their dash cams. All show a well lit stretch of street with good long range visibility.

        2. Daniel 18

          Re: "Self driving cars can only only react"

          "I timed the video and from when the pedestrian appears to when the video stops seems to be closer to 0.75 seconds rather than 2.

          Not sure I would have noticed, reacted and braked to a 'safe' speed in that time."

          You could not.

          It is generally accepted that it takes about 1 second to initiate braking once an obstacle becomes visible.

          1. Boo Radley

            Re: "Self driving cars can only only react"

            I'm certain that I react in well under one second upon becoming aware of something unexpected in the road. But then, I have had advanced driver training and have generally quick reflexes.

        3. Tom 64
          Windows

          Re: "Self driving cars can only only react"

          > "Not sure I would have noticed, reacted and braked to a 'safe' speed in that time."

          Perhaps not, but you would have damn well swerved in order to avoid taking the life of another human being. Wouldn't you?

          1. Yet Another Anonymous coward Silver badge

            Re: "Self driving cars can only only react"

            but you would have damn well swerved in order to avoid taking the life of another human being.

            If it was a Volvo driver an the human being was on two wheels?

        4. This post has been deleted by its author

        5. John H Woods Silver badge

          Re: "Self driving cars can only only react"

          "I timed the video and from when the pedestrian appears to when the video stops seems to be closer to 0.75 seconds rather than 2." - Noel Morgan

          You need a bigger screen or spectacles! Seriously, though, I suspect you are looking with human eyes....run the video frame by frame and note the frame when you can see the first hint of her shoes on the road: 2.96 seconds. Now if it really were that dark, a human might be excused not recognizing that as a person but she is presenting a full radar image at that point in the lane of travel. Impact is later than 4.28 seconds.

          In the following 32 frames (1.3 seconds) she has progressed from a quarter of the way across the lane to three quarters of the way across the lane when she is hit. Therefore it is reasonable to assume it took her at least 2 seconds to cross the left lane and get to the middle of the road. If the radar can see the entire roadway, it had a minimum of 3 seconds to react, if it can only see the lane of travel there was 1.5 seconds minimum. At an entirely reasonable 0.8g braking, that's enough to drop from 40mph to 20mph.

        6. MrXavia

          Re: "Self driving cars can only only react"

          "I timed the video and from when the pedestrian appears to when the video stops seems to be closer to 0.75 seconds rather than 2."

          True but emergency braking as soon as the obstacle was detected would have reduced the impact speed and increased chance of survival, I saw no braking at all..

          But as other dash cam show, that area is well lit, even in the darker section, so the cyclist should have been seen earlier and the car should have been slowing down already anticipating the hazard and not accelerating towards 45...

          Seeing someone crossing like that I would have taken my foot off the accelerator and covered my brake (but not actually applied the brake until she crossed the middle line, so I would have been braking from the moment her wheel started to enter my lane and stopped in time to avoid her death...

          But even if I was not paying attention my 5 year old cars radar would have braked automatically as soon as she stepped into my lane even if I could not see her.. and that would have reduced impact to a survivable speed....

    3. veti Silver badge

      Ah yes, the good old "artificial intelligence is impossible" line. I wondered how long we'd take to get to that.

      Brains are not magic. Everything that goes on inside a human mind is something that can (in principle, if you really want to) be replicated in another environment. Of course mosttimes we don't really want that, but the point is that any generalised statement about what one thing or the other "can" do is just - magical thinking.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like