Uber's disturbing fatal self-driving car crash, a new common sense challenge for AI, and Facebook's evil algorithms It’s been a grim week for AI. The deadly Uber crash and fallout from the scandal between Facebook and Cambridge Analytica are a reminder of the ways algorithms can fail, and how they can be used against us. Fatal Uber self-driving car vid - The video footage capturing the last moments just before one of Uber’s self-driving car …
The footage also makes clear two other important factors:
1. The car was driving on dipped headlights under conditions where the full beam would have been appropriate.
2. The car was driving at a speed where it was unable to stop within the distance lit by its dim and poorly adjusted headlights.
So whilst we do have to ask what the rider was doing to not notice the car, it is clear some basics of driving have been missed by the Uber developers.
Someone else has probably already said it but... a video camera might have a dynamic range of 10 stops, compared to about 20 stops, for the human eye. The reason the road looks so dark is because the headlights are bright and the aperture of the lens is adjusted so that the centre of the field is not over-exposed. If the headlights had bean turned off, I suspect that the woman would have been visible on the video recording much sooner.
However, this is all irreleveant, because the car (hopefully) wasn't using the footage shown, it was almost certainly using LIDAR - it is interesting that Uber have not been forthcoming with that data, only with the video blaming the driver* and the misleading passive camera footage.
* don't get me wrong the driver isn't blameless, but the software developers are ultimately responsible.
The only clear point the video shows is thid accident was going to happen driver or driverless.
No way would a human behind the wheel have changed the outcome.
It just highlights that whilst driverless cars will for sure reduce road deaths, it's never going to get to the magical 100%. It with current technology.
> No way would a human behind the wheel have changed the outcome.
If I had a dashcam for every time I've proven you wrong....
Driverless cars MAY ultimately reduce road deaths, but will the lives they save be worth living? They WILL monopolize personal transportation, milking riders for money, making the super-rich richer, and stranding even more lower/middle class people in slums. They're a wet dream for accidentally-wealthy binge-drinking douchebags. Also, they're beyond Orwellian.
OTOH, if self-driving cars stop for any asshole who runs out in front of them, expect a MASSIVE rise in road robberies, kidnappings, rapes, and murders.
Tractors MAY ultimately reduce farm deaths, but will the lives they save be worth living? They WILL monopolize equipment haulage, milking farmers for money, making the super-rich richer, and stranding even more agricultural labourers in slums. They're a wet dream for accidentally-wealthy binge-drinking douchebags. Also, they're beyond Orwellian.
And don't get me started about building houses to live in rather than inhabiting nice dark caves....
"OTOH, if self-driving cars stop for any asshole who runs out in front of them, expect a MASSIVE rise in road robberies, kidnappings, rapes, and murders."
You don't stop for people who run out in front you?
"If I had a dashcam for every time I've proven you wrong...."
Given the above... how?
> Tractors MAY ultimately reduce farm deaths, but--
Try harder, smart-ass. You replaced "self-driving cars" with "tractors". If you want to talk about self-driving tractors, be my guest.
> You don't stop for people who run out in front you?
I honk, swerve, brake, and sometimes I even stop - i.e. for little kids - then I yell at them if it's a busy street. But I try very hard to avoid stopping for people who deliberately try to stop me. Particularly in shithole neighborhoods. Yes, I have seen some shit.
The other WTF you missed was that even when the pedestrian came in to camera view the car did not brake even though an AI system should have a reaction time an order of magnitude faster than a human, and should out-brake most humans by knowing full well what is the lock-limit of the wheels (i.e. reaching anti-lock point),
Also as others have pointed out the car was clearly driving beyond its sensor range, again that is something humans do, but is actually against the highway code. Drive to your visible range stopping-distance we are instructed.
So who is going to fall for this: The software engineer(s) who developed the sensor/stopping code? Those who did a safety analysis? The executives up top?
The only clear point the video shows is thid accident was going to happen driver or driverless.
No way would a human behind the wheel have changed the outcome.
It just highlights that whilst driverless cars will for sure reduce road deaths, it's never going to get to the magical 100%. It with current technology.
I'm not sure why the downvotes. Based on the video just of what's happening on the road, I tend to agree. The distracted driver doesn't help. Black clothing at night on the roads is a serous problem. I agree that a human driver would have had issues also.
But are there other factors at play here... was the car going too fast for conditions? Even humans drive too fast. There's just too many unknowns here for one to make a judgment call from watching a video on a computer screen.
I'm going to add to my previous... The video itself. Is this a recording from the LiDAR or from a separate system? IF it's the LiDAR feed it's a piss-poor camera.
I'm not a fan of Uber and rather despise them due to their manglement's attitudes and actions. But I'm trying to be fair as to this. My local roads are a bitch to drive at night due to the inconsistent and uneven lighting.
It's not a recording from Lidar. Lidar provides something more like a point-cloud or just a single flat plane. It doesn't produce recognisable images. It produces something far more useful : an accurate array of distances in multipole directions.
It also works better at night, because the sensor is less often overpowered by sunlight. It's true that a black object (something with low reflectivity) will not be detected at long range, but a decent lidar should manage at least 30m for any normal clothing.
It's more likely that the decision to brake is based on a mixed weighting of visual detection, lidar and a number of other factors, and that the algorithm used chose to reduce weight given to the lidar. Or it was misaligned and didn't see her at all.
That is what I think. The program weighted the probabilities, and decided that no way that thing could be a woman crossing.. as it was in "road mode", and no humans/animals would cross.. and also, no detection on regular video.. so yeah, that must be an error in the lidar.
Had than been a deer, well, the car would have crashed too.
>Based on the video just of what's happening on the road<
(1) The video is not relevant to the LiDAR
(2) The reduced dynamic range of video gives a very poor representation of what the eye sees in the dark
(3) Actually, as shown elsewhere and reported by people who drive there, it's not that dark anyway: it's a well lit main road in a busy district.
(4) Because the video is so dark, you can't see that the district is actually built up, with crossings, includes places for pedestrians to cross the median.
(4a) Which explains why the woman is on foot: she's crossing as a pedestrain at a median break, so that she doesn't suprise drivers expecting pedestrians, not bikes.
> No way would a human behind the wheel have changed the outcome.
I strongly disagree. Even in the low-res video you can clearly see "something" before the cyclist comes into view, but there's no indication that the vehicle starts to slow even at that point. Here in Berkshire, we have many suicide cyclists who ride around in the dark, dressed in black with no lights. One night driving down an unlit road, I could see ?fireflies? twinkling in the distance. I worked out that they were the reflectors on a pair of pedals going up and down and was able to slow down enough to avoid the cyclist *before* I ran him over.
AI is not intelligent because it still only understands what it's seen before, and doesn't yet appear able to put together a hypothesis like the one that enabled me not to kill a cycilst. Personally, given that self-driving cars still can't cope in the relatively benign environments they are being trained in, I think we are decades away from genuinely autonomous vehicles.
I agree. Humans have inputs, even though their mechanism maybe not fully understood, that I can't see AVs matching in the foreseeable future.
Many years ago, on a dark, wet night, I was driving my 6V VW Beetle (the one with the candle-power headlights) along an unlit part of an almost deserted highway. Maybe at around 60kph because of the terrible conditions. I got a sudden, urgent intuitive prompt "STOP". I immediately slowed to walking pace and out of the mist in front of me loomed a cyclist wearing only (wet) black, no lights, no white showing anywhere. He had possibly been crossing the road in front of me. Had I not heeded the prompt, I'm certain I would have got him.
I bet many other readers have had comparable experiences with intuituve flashes.
AI is not intelligent because it still only understands what it's seen before, and doesn't yet appear able to put together a hypothesis like the one that enabled me not to kill a cycilst. Personally, given that self-driving cars still can't cope in the relatively benign environments they are being trained in, I think we are decades away from genuinely autonomous vehicles.
I live in Cornwall, where roads are distinctly un-uniform in arrangement. There is absolutely no way a system such as this could cope with Cornish lane-driving, ever. If a lone cyclist went undetected, imagine its behaviour when going around a steep corner to be presented with a horse-rider, flock of sheep on the road, etc...
I’m pretty sure the LiDAR registered the woman crossing. It doesn’t rely on the head lights but has its own infrared laser, and it should get returns from a human sized object well beyond 50 m. If I were to guess, I would guess that it’s a logical error in the programming. They seem to be on a freeway or similar where pedestrians are very unlikely to be in the middle of the road, and then the detections were simply considered as noise, and disregarded.
Isn’t it odd that the police publishes the face of the driver like this, before any trial and sentencing...?
See the video below that gives an indication of the LiDAR view. I agree, Uber (or the police) has been disingenuous releasing only the low resolution dash cam footage. The cam's field of view is much narrower that that of a, presumably attentive driver with reasonable peripheral vision.
Using human performance as the benchmark is also stupid. Cars have significantly better sensors, though the magic is clearly in making sense of that data.
LiDAR scanning view
* https://360.here.com/giving-cars-their-superpowers
I wonder why the Lidar did not pick up the poor woman. Perhaps they do it on the cheap - the ones used for mapping the ground can do 30,000 samples a second over a reasonable viewport. Should have picked up her shoes at least. Or perhaps they dont want the car to break on every moth or bird in front and have set a 'only if bigger than' to larger than shoe size. I guess wider ranges of laser frequencies might be necessary to ensure that people and cars dont cloak themselves.
Rights of way are often superseded by instantaneous risk analysis.
For example, the aggregate lorry flying through the junction at 30mph against the flow of traffic caused me to relinquish my right of way in favour of not being squashed like a bug.
That had me wondering as well. A point I haven't seen anywhere is whether the car was petrol engine or electric (and therefore making little noise)? Why did the pedestrian not detect the car is as much a question as why did the car not detect the pedestrian. Don't these self-driving cars carry normal headlights?
..when he realised how important self-driving cars were to the business.
QED. So there you have it - Uber is indeed a taxi company and NOT a ride-sharing website.
I am also struggling to understand the economics here - all this LIDAR (particularly the sort required here) and real-time processing power is not cheap and then integrating that with a vehicle cannot be either.
This cannot drive Uber profits in the short-term at least simply because of the time required to scale up and also the cost of doing so - especially if a safety driver is required (equivalent to a man with a red flag in car history).
The real-time processing and sensor technology impressive as a demonstration BUT the 80/20 rule is in play here. It work up to a point BUT the remaining 20% will take a significant amount of time to solve - and probably won't be solved entirely.
"By the end of the year" is a pipe dream - what sort of pipe is left as an exercise for the reader...
Others have posted videos of their $50 dash cam videos, in the exact same location and under the exact same sort of night conditions. It's like Day and Night.
Just because Uber decided to use cheap-ass cameras that can't see anything at night certainly doesn't give them any excuse to escape from what seems to be Criminal Negligence.
We've all seen dash cam videos taken at night. Why isn't this obvious rebuttal to "it was night time" perfectly and immediately self-evident? Who wouldn't instinctively realise this as an instant rejection of this inexcusable excuse?
True, but as I understand it the woman was dressed in black. That's where the problem lies; the lidar signal showed only small objects (part of bicycle, shoes) and did not realise they were connected.
That said, a pedestrian walking across a road at night wearing black - how many average drivers would have seen her?
"how many average drivers would have seen her?"
If they were travelling 85 metres without looking at the road (the speed they said the vehicle was doing at impact for the duration the video shows him staring at his phone) probably none.
And this is the issue with the self driving car, people will zone out and not be prepared when the car cuts back to them or, in this case, needs to be overridden.
The idiot who hit the truck in the Tesla is a great example of that and his car had adaptive cruise, auto breaks and lane adhesion, not even self driving (though calling it autopilot seems to have confused some users into thinking it's more, such as the guy found asleep and drunk at the wheel as "the car was driving itself").
So, even having it as driver assist is dangerous, should leave people to manage the difficult bit, keep them alert, keep the busy. Where's my sabots...
"If they were travelling 85 metres without looking at the road"
This week I have seen:
*Driver turning right onto main road in town while trying to get attention of someone on the opposite side of the road and to his left.
*Driver turning straight across the road into oncoming traffic in order to reach a roadside parking spot before someone else could.
*Van driver doing at least 20mph in a car park between rows of parked cars.
*Obvious drunk weaving and doing 15mph at about 3p.m.
And I do a fairly low mileage. FWIW I am not defending Uber; what I am suggesting is that for self driving cars to work there is going to need to be a culture change. Road markings need to be maintained. Bad drivers need to be caught and deterred. Pedestrians and cyclists need to understand defensive behaviour. The rise of the machines could be a (welcome?) corrective to human sloppiness and laziness.
And for people who MUST use the car to work and feed the spouse and kuds, we just kick ALL of them to the curb?
Or as put in an episode of Dragnet regarding abused wives staying with their abusing husbands, "Who's going to bring home the gtoceries?"
how many average drivers would have seen her?
Leaving aside the the problem that the human wasn't actually looking, if this had been a normal car with a human driver I would have thought that the chances are high that an average driver would have seen her.
The video shows that the camera didn't pick the lady up until the last minute, when she appeared in the full beam of the headlights, but then we know that video cameras are poor at resolving contrast in dark conditions, and the human eye is much, much better at resolving and identifying movement in those conditions.
When I first read of this incident, I sort of assumed that the lady had dashed across in front of the car, or had suddenly appeared from behind an obstruction. It is clear from the video that neither of these was the case, and I think an average driver would have seen her much earlier and taken avoiding action.
This very much looks like a failure of the car's detection systems, and not an unavoidable accident.
From what I saw she was wearing blue jeans and a light coloured top or jacket. But hey..
I'm reminded regarding the cops in this of a scene from Squidbillies. The two cops should be arresting Dan Halen for serial murder. Dennys all for it until Sheriff reminds him "We can't arrest Mr Halen, he's our boss.. "
Or to put it bluntly "They're as bent as a 9 bob note"
The woman wasn't dressed in black. Lidar can see black object perfectly fine, if it couldn't, it is completely useless for range finding and mapping uses.
On that road it would have been very easy to see her. The road is no where near as dark as that crappy dash cam that uber have installed (if they didn't alter the video before showing it).
And don't cyclists normally wear reflective vests as well?
Where I live, there's cyclists and bicycle users. The cyclists wear the form fitting bike outfits with helmets, etc. The bicycle users wear whatever they have on. Many are homeless, many have lost their licenses for alcohol related offences. Others just are using it to commute. It's the bicycle users who create problems with visibility. The cyclists create their own set of problems as mentioned in other topics.. like their attitude of infallibility.
This post has been deleted by its author
Obligatory WIWAL:
We were supposed to have cycle lamps on our bikes (Front & Rear) I had a dynamo powered set installed on mine rather than suffer the inevitable flat battery in the dark while cycling home.
Over the years I have seen a trend in cyclists sporting tiny flashing sets of three red LED's at the rear & a single (Or three, which purports to be ultra-bright) tiny white LED('s) usually strapped to the head (Putting me in mind of Jasper Carrott with a torch strapped to a shotgun "whats 'e doing? I dunno imitating a lighthouse) & again usually flashing as viable substitutes to aid night time riding & to be more visible to motor vehicles than the great big lights which I grew up with.
I once almost had a new fixture to my car bonnet some years ago driving up the A303 with a cyclist in the middle of the slow lane in the pitch dark & barely visible blinking rear LED's.
Something similar also occurred on the M4 only with a classic Morris Minor (As far as I could tell, as I swerved around it) pottering along at 25mph with tiny red & very very dim original rear lights.
Those of a certain age will recall the following very well:
https://www.youtube.com/watch?v=1MFuSMz1zh0
https://www.ebay.ie/itm/Ever-Ready-Vintage-60-70-s-White-Red-Front-Rear-Bike-Cycle-Bicycle-Lamps-Lights-/401512553487?hash=item5d7c035c0f
https://www.youtube.com/watch?v=1MFuSMz1zh0
'Let the world see your life as something that you treasure - get yourself seen!' - Quite.
Riding a motorbike made me realise just how invisible we can be, even when lights are on, reflective jackets etc.. - if you don't ride like every other road user is liable to kill you at a moments notice you will end up as a statistic. This also goes for when you are a pedestrian or a car driver or a cyclist etc.
Who cares? The issue here us not to apportion blame but that a so called autonomous car did not react to an object directly in its path for at least 2 full seconds (you an see her shoes even with mark 1 eyeball, even on the dashcam footage, at 0:03 -- and impact is 0:05 or later), from 40m or so to impact.
Jesus, the front assist on the car I'm currently driving is better than that, I shan't be getting in a self driving car with worse detection abilities.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
