back to article America's top maker of cop body cameras says facial-recog AI isn't safe

America's largest manufacturer of body cameras – and the biggest supplier to police forces across the United States – says today's facial recognition technology is not safe for making serious decisions. Speaking during its second-quarter earnings call with investors this week, the CEO of Axon, Rick Smith, answered a question …

  1. Anonymous Coward
    Anonymous Coward

    'The AI told me to do it'

    Firms making body cams are showing restraint while police forces aren't. What could possibly go wrong? ... We all know where this is going. US cops already routinely shoot innocent people for no reason. Can't wait for the clusterfuck excuse of: 'The AI told me to do it'!

    https://en.wikipedia.org/wiki/Shooting_of_Justine_Damond#Aftermath

  2. JohnFen

    Wow!

    I'm am incredibly, and pleasantly, surprised by this. Facial recognition is nowhere near good enough to base any serious decisions on, but there seems to be a willful ignorance of that fact amongst the sellers of the technology.

    Good on Axon for being honest and realistic!

    1. a_yank_lurker

      Re: Wow!

      Facial recognition and surveillance videos are useful not so much to positively identify the someone but eliminate people and provide time lines. If a crime occurred at a known time and you resemble the perpetrator caught on video you might want check how solid your alibi is if the flatfeet come knocking. The real problem is when the technology is pushed to beyond its limits by the ignorant and thus misused.

      1. Wzrd1 Silver badge

        Re: Wow!

        Alas, the shitware, for no other name exists for it, misidentified sitting US Senators and Representatives, of a somewhat swarthy complexion as criminals.

        Repeatedly.

        Yes! Let's field this kit! It's ready for horror films!

    2. Anonymous Coward
      Anonymous Coward

      Re: Wow!

      Axon used to be called Taser. Historically, ethical decision making skills havn't been among their strong points.

      1. Wzrd1 Silver badge

        Re: Wow!

        A Taser device is perfectly safe, right until it destabilizies your cardiac conduction system and kills you.

        When your family objects, you get litigated into homelessness.

        Welcome to US justice.

        1. Voland's right hand Silver badge

          Re: Wow!

          When your family objects, you get litigated into homelessness.

          That may be a factor here. They have themselves on public record that the tech should not be relied upon. So anyone who wants to sue them will most likely fail.

        2. Anonymous Coward
          Anonymous Coward

          Re: Wow!

          A Taser device is perfectly safe, right until it destabilizies your cardiac conduction system and kills you. When your family objects, you get litigated into homelessness. Welcome to US justice.

          You'd rather they used the traditional lead bullet approach to law enforcement? I wouldn't.

          I'd accept that tasers are merely "less lethal" rather than "non-lethal", but if being taken down by a cop I'd choose to be tasered rather than pumped full of lead. I can't quickly find a good reference for the lethality of tasers, but subjectively it looks to be in the order of 1 per 1,000 uses. I'll wager that is a whole lot better than firearms, and probably not much worse than pepper spray's unintended lethality.

          If anyone can come up with a truly non-lethal device that can incapacitate, I think you'll find the cops would happily use it - if only to avoid the bad publicity and paperwork of killing suspects.

          1. Anonymous Coward
            Anonymous Coward

            Re: Wow!

            @Ledswinger

            Actual use of tasers is 99+% *NOT* as a replacement for gun fire. So the comparison of tasers to guns is nearly 100% specious.

            Actual taser use is to impose (extreme) pain compliance, either from a "safe" distance, or to impose some extrajudicial light torture up close.

            The overwhelming non-applicability of the gun lethality comparison is the very first tidbit that needs to be hoisted aboard.

    3. Nick Kew
      Alert

      Re: Wow!

      Good on Axon for being honest and realistic!

      No possibility they might've had a vested interest? Perhaps in implicitly discrediting a competitor who makes claims for facial recognition?

    4. Zangetsu

      Re: Wow!

      the taser company CEO was not saying that because he wants civilians to be protected from over zealous cops, he knows that it would damage the brand name with so many false positives.

      as greedy as he is, he knows the way it would be misused.

  3. Mad Chaz

    Axon doesn't need

    to hide the tech is crap. Politicians and cops want it, so it works as far as they care

  4. Fruit and Nutcase Silver badge
    Black Helicopters

    Money talks

    Make it a condition that before facial recognition systems are used against ordinary citizens, a facial recognition system is put in place to recognise MPs, once a month for a trial period of 1 year. If the system recognises the MP correctly, then the MP will get paid for that month and reimbursed any expenses and any other money that is due for that month. Alternatively, impose the same trial on the Top Civil Servants

    1. JohnFen

      Re: Money talks

      "If the system recognises the MP correctly, then the MP will get paid for that month"

      That's testing the wrong thing, though. False positives are a more problematic error than false negatives, and that's where facial recognition tends to do the worst. So the test shouldn't be if the MP is correctly recognized, it should be that it didn't incorrectly identify anyone else as that MP.

      1. Fruit and Nutcase Silver badge
        Joke

        Re: Money talks

        @John Fen

        That's testing the wrong thing, though. False positives are a more problematic error than false negatives,

        Fair point. How about an entry system at Buckingham Palace?

        The Queen will be stuck out in the rain whilst Helen Mirren will be able to get in

    2. Voland's right hand Silver badge

      Re: Money talks

      Wrong idea. The right idea is "Drop down your weapon. You have 30 seconds to comply".

      Now assign it to do security for the [mps|congesscriiters|etc].

      Optional: fix the cannot navigate stairs bug first.

  5. kain preacher

    Cops want this tech cool give it to them but. When it fucks up the cops have strict liability. The police and their superior are finally liable . This can not be discharged. Cops can by charged with false arrest

    1. Anonymous Coward
      Anonymous Coward

      Not now or ever. Police are well shielded by multiple layers of invincibility. The rare exceptions are notable.

      1. kain preacher

        I know but I wished liability was attached . It's just a pipe dream

    2. Wzrd1 Silver badge

      "Cops want this tech cool give it to them but. When it fucks up the cops have strict liability. The police and their superior are finally liable . This can not be discharged. Cops can by charged with false arrest"

      Largely false in the US.

      Oops, we fucked up is the excuse and it's excused.

      Eventually, the POTUS will get shot by police, "by mistake" and they'll end up excused.

      The US has a massive gap between federal and state systems. Police are local, not quite state, not quite not. Federal officers, like Secret Service and FBI are federal.

      Each have their own Constitutional boundaries.

      So, it's entirely possible for a local police officer to conduct the behavior I suggested in the US.

      God Save The Queen if such a system misidentified her, as US police don't think, as they have a "war on crime" mentality.

      With them literally thinking that they're soldiers in a literal war.

      I know, I know enough of them.

      Not very mentally flexible at all.

  6. Anonymous Coward
    Anonymous Coward

    Axon, formerly known as Taser

    They've spent the past decade ever so slowly inching away from their previous claims that Tasers were somehow inherently cardiac safe. A deleted 'Cardiac Safety' webpage here, a changed adjective there. It's all on file.

    I'm not convince that they have a strong Corporate Ethics program. Best that you count your spoons quickly.

    1. Lee D Silver badge

      Re: Axon, formerly known as Taser

      Anything, literally anything, that incapacitates a person can not be guaranteed "safe". Not even a giant pillow made of goose down that weighs the opponent to the floor.

      If you're firing a taser, it's because you're under physical threat (if it's not, you need better police... don't expect technology to solve your training issues). At that point, the risk of ANYTHING you do to avert that threat is that it might be a threat to the guy posing it. Might.

      However, the alternative is... physical confrontation, or shooting them. At that point, I'd use a taser every time, thanks. If they were legal, I was trained, etc. No, I wouldn't necessarily use a gun at all unless my actual life was in immediate threat, I abhor guns. But that a taser might hurt even an otherwise healthy person? Yeah, sure, sorry about that. Tough luck.

      The problem you have is not Tasers, not guns, and not some unspecified health risk. It's that someone is either under physical threat and needs to respond (in which case, tasers are safer than knives, gun, batons or even fists), or that they use the weapon when they shouldn't.

      Sure, maybe their marketing claims are bogus and they have removed them. No different to anything else commercial. But tarring them with the evil brush just because they make a device you're supposed to use in self-defence and that device *might* hurt someone... what a damn shame.

      1. Anonymous Coward
        Anonymous Coward

        Re: Axon, formerly known as Taser

        @Lee D

        Actual use of tasers is 99+% *NOT* as a replacement for gun fire. So the comparison of tasers to guns is nearly 100% specious.

        Actual taser use is to impose (extreme) pain compliance, either from a "safe" distance, or to impose some extrajudicial light torture up close.

        The overwhelming non-applicability of the gun lethality comparison is the very first tidbit that needs to be hoisted aboard.

        1. JohnFen

          Re: Axon, formerly known as Taser

          "Actual taser use is to impose (extreme) pain compliance, either from a "safe" distance, or to impose some extrajudicial light torture up close."

          Yes, that's the primary problem with Tasers. They're used as means of torture.

          Funny, though, that when their use was being sold to the public, that wasn't their selling point. They were offered as a safer alternative to guns.

  7. Anonymous Coward
    Anonymous Coward

    Surely all it has to do

    Is determine whether the suspect is Black or White in order to decide whether to shoot them or not.

    They could it extend it for the UK to detect Brazilians.

  8. Awil Onmearse

    Cressida Dick?

    The one who murdered a Brazilian electrician and got promoted to head of the Met for a job well done? That Cressida Dick? Maybe best not to trust her judgement when it comes to facial recognition.

    1. Spacedinvader

      Re: Cressida Dick?

      Looks like she saw your post...

  9. John Smith 19 Gold badge
    Gimp

    "with a "worrying vacuum" in governance and lack of oversight."

    Yes that's a pretty good description of all UK efforts in this area.

    Facial recognition. Sure it's got a 98% false positive rate. We're rolling it out anyway.

    Linking FR to the backend police databases. We're doing it.

    Enabling legislation for Automatic Number Plate Recognition? Whatever for?

    1. Pascal Monett Silver badge

      Well, the UK government does have a long history of continuing to throw money into failed IT projects, so it's just government as usual.

  10. I3N
    Flame

    HELL, SAFE, SAFE? ... I've been waiting 8 months

    for my body camera request and what finally arrives is my original request ...

    cop: yelling and screaming at me about fence permit ...

    me: is that thing on. ... cop: 24/7 ... me: mine too ... cop out back taking pictures ... me waving ....

    cop writes me up for permit violation - cameras on utility pole

    City wants nothing to do with fence permit, none required and have zoning approval

    City really doesn't want utility pole issue ... not their problem or pole ... Pai got it set up so I can just declare I'm an Internet service with expedited approval

    Out checking property markers two weeks later.

    Cop shows up. Told him I'm filing a harassment complaint ...

    cop: so what ... me: I want the video on what was said by that Trump loving, Hispanic woman to get you riled up ...

    cop orders me out of street and then pulls over with lights flashing.

    AND NOW THE FUCKING VIDEO CAN"T BE FOUND!!! WHO CARES ABOUT AI!

  11. Nick Kew

    For what purpose?

    OK, quoting your actual words from the opening paragraph:

    today's facial recognition technology is not safe for making serious decisions.

    Is anyone seriously trying to claim otherwise? There's a world of difference between making a serious decision and flagging something for human attention.

    A couple of years back, I had a nasty incident with police, who thought I was someone else (who I've never met, let alone know why they wanted to arrest him) and wouldn't accept I'm me[1]. Facial recognition technology might have helped there, and - crucially - couldn't have made things worse!

    [1] Their evidence? I opened the door at my home, where the man they wanted had previously lived. I had never thought an estate agent could be so useful as the one who manages this place and eventually was contacted to confirm my identity - and when the occupant had changed - in a manner they'd accept!

    1. Nigel Sedgwick

      Re: For what purpose?

      Nick Kew writes (WRT facial recognition not being safe for making serious decisions): "Is anyone seriously trying to claim otherwise?"

      I was rather under the impression that the UK Government was making some such claim with its incoming checks on UK and other EU passports, and its facial scanning booths and 'biometric' passport photographs.

      Best regards

    2. Pascal Monett Silver badge

      Re: Is anyone seriously trying to claim otherwise?

      Well you've got the guy responsible for security at the Tokyo Olympics who really doesn't seem to have gotten that memo.

    3. JohnFen

      Re: For what purpose?

      "There's a world of difference between making a serious decision and flagging something for human attention."

      If that human attention involves detention (even for a brief time for questioning) or worse, then it's a serious decision.

  12. martinusher Silver badge

    Its the legal implications.

    Making a body camera is a trivial exercise these days. Making a system that can collect evidence quality video and audio, manging it through a chain of custody, is a whole different game. These companies sell systems, its how they can justify selling the individual units for silly prices.

    They're also aware of the legal implications of getting things wrong. No facial recognition system is going to be 100% perfect so it will make mistakes and those mistakes could cause legal blowback, including a liability issue for the manufacturer. Hence the need to be super-cautious. However, in real life these systems only need to work as well as, or preferably somewhat better than, human recognition to be useful because they're not going to be used instead of humans but as an assistant.

    There's a lot more to this technology than just matching a face to a name. If you arrive at Los Angeles airport at the international terminal and you're a US passport holder you'll be processed automatically by a kiosk. Image processing is also being used to identify people who are acting oddly or looking stressed (or rather "more stressed than normal"). This stuff is intrusive, is potentially a civil liberties killer but then its really only ANPR for people and everyone in the UK's been living with ANPR and its consequences for years, haven't they?

  13. Anonymous Coward
    Anonymous Coward

    Oh putain !

    Talking about this Cressida Dick, I actually met her.

    It must have been ten or twelve years ago, in a pitch dark alley in a non-public area of Gatwick airport (I was a first officer at the time). I recall distinctly because I was lost and saw a plod in the shadows so I came up from behind¹ and boomed out an "Excuse me Sir!" It turned out it was a she, of course, so at first I was a bit embarrassed for the mistake but then I realised she was a capital-D Dyke² if I ever saw one (and I've seen quite a few) and she seemed almost proud of having been called "sir". Very well educated too, judging by her demeanour and pronunciation.

    I didn't know at the time that she had been responsible for the assassination of Mr de Menezes.

    Anyway, I'd totally forgot about it but I saw her name in the subheading and the episode came back to me.

    ¹ Possibly not the most comforting thing in the world to have some bloke chase you up a dark alley shouting at you in a Scottish accent, I thought afterwards.

    ² As in a lesbian that looks and acts masculine.

  14. Danny 2

    Trial and error

    Burr-Brown threw away serious money at this in the '80s. DSP over the VMEbus.

    It kind of worked. Their system was 100% accurate at recognising the four design engineers, utterly useless at recognising the rest of the factory staff.

    It mistook me for a girl, which was wrenching because she was a very pretty girl. Didn't know whether to feel insulted or complimented. It was a conversation starter at least, but not a conversation that led to positive results.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like