And yes, we knew AI would suck at this. People who say mean things online want to thwart the AI filters as well.
AI sucks at stopping online trolls spewing toxic comments
New research has shown just how bad AI is at dealing with online trolls. Such systems struggle to automatically flag nudity and violence, don’t understand text well enough to shoot down fake news and aren’t effective at detecting abusive comments from trolls hiding behind their keyboards. A group of researchers from Aalto …
COMMENTS
-
-
Friday 31st August 2018 08:03 GMT jmch
"we knew AI would suck at this" ('this' being recognising offensive language)
When it comes to irony, actual people also suck at differentiating between really offensive, ironically offensive taking-the-piss that is actually innocuous, and ironic inoffensive-sounding that is actually offensive.
Text-based comments are actually very bad at that because a lot of nuance is expressed in verbal tones that is immediately clear in conversation but not available in written text.
-
-
-
Friday 31st August 2018 08:14 GMT jmch
"I don't know why anybody places any importance on what people comment on online comments"
Unfortunately many people are lacking in self-confidence and knowledge of self-worth that they attribute to themselves the worth that is assigned to them from others. Doubly bad, this type of behaviour is a side-effect of the society most people are exposed to eg in school constantly being judged for marks, behaviour etc, drummed into children at an early age that their value is what is assigned to them by parents / teachers (and as teenagers, by their peer-group). It's especially for women constantly being judged by appearance.
So, yeah, mature and well-adjusted people don't place any importance of online comments. However not that many people are mature and well-adjusted.
-
Friday 31st August 2018 18:31 GMT Orv
Unfortunately many people are lacking in self-confidence and knowledge of self-worth that they attribute to themselves the worth that is assigned to them from others.
Even people with healthy self-esteem, given a drumbeat of relentlessly negative feedback, can succumb. We're social creatures and evolved to try to adapt to the demands of those around us.
There's also the issue of speech that isn't just hateful, but is actually threatening. I don't think someone's showing a thin skin when they're put off by someone, say, naming where their kid goes to school and suggesting something might happen to them. The line between this and "vanilla" hate speech can be pretty thin and subjective -- e.g., is photoshopping a picture of someone into a gas chamber a threat or 'just' an ordinary comment that should be ignored? What about a post listing someone's address and suggesting they be SWATted?
-
-
Friday 31st August 2018 11:23 GMT werdsmith
Mostly because they drown-out serious discussion.
Take a recent example. The NSPCC wanted to have a Facebook live presentation on child abuse, but had to cancel it because they were flooded with hate speech from transphobes who organise on Mumsnet.
Why would anyone expect serious discussion on websites? They are exposed to every nutter, just like Hyde Park Corner. It's just not sensible to expect it to happen. And Faecebook? FFS who had the idea of expecting that to work? That's like choosing to have a dinner party in a sewer, there is no respect at all for that platform among intelligent people, there are only sad sacks left on Faecebook.
It can work, and this here specialised register is an example, but the wider web is the same as a busy street. You just wouldn't set up an open forum for every passer by and expect sanity to prevail.
-
-
Friday 31st August 2018 09:34 GMT Nick Kew
I don't know why anybody places any importance on what people comment on online comments.
Did someone say something?
People say nasty things. Sometimes even gratuitously nasty.
But it seems to me that much more time and effort goes into taking offence where nothing unpleasant was ever in the mind of the poster nor anyone else in the original discussion.
-
Friday 31st August 2018 09:25 GMT Christian Berger
Well we are talking about intellects comparable to...
mentaly disabled children. "AIs" are simple machine learning and therefore not very smart.
Now while people usually have reservations against tricking mentaly disabled children, they do not have such reservations against machines. Tricking machines is fair game. In fact it's common for people to hit machines or rub coins on them, scratching their surface just because they believe this will teach them a lesson.
Same goes for complex "Code of Conducts". They are essentially just an invitation to trick them by finding loopholes and be as bad as annoying as you can be while still staying within the limits of the CoC. It's much better to handle conflicts on a case by case basis when they occur.
-
Friday 31st August 2018 10:48 GMT Anonymous Coward
Re: Well we are talking about intellects comparable to...
"Well we are talking about intellects comparable to mentaly disabled children. "AIs" are simple machine learning and therefore not very smart."
But the media / researchers are responsible for pushing the view that AI is smart. My son is 21 and has severe learning difficulties - he is effectively non-verbal and has a functioning mental age corresponding to a very young child. He will never be able to drive a car yet to suggest that AI driven vehicles are not (yet) fit for purpose will bring down the wrath of the supporters of autonomous vehicles even though they would be horrified if my son got behind the wheel. People just do not realise how limited AI is and have unrealistic expectations.
A/C to protect my son's privacy.
-
Friday 7th September 2018 19:37 GMT Michael Wojcik
Re: Well we are talking about intellects comparable to...
Well we are talking about intellects comparable to mentaly disabled children.
Sure, if we're looking for an unproductive and inaccurate comparison.
I don't know of a single ML project that can usefully be compared to a human at any stage or condition of intellectual development. None of the developmentally-limited people I've known have exhibited behavior anything like any ML (or "AI") system I've seen; neither have developmentally-normal children at any stage, including infancy. I don't find any correspondence in developmental or child psychology, either.
Convolutional Neural Nets essentially do fuzzy signal filtering and amplification. Recurrent Neural Nets essentially do feature classification. Other NN types, such as LSTMs, fall into one of those two families. Deep nets are just stacked NNs, identifying signals or features at different scales and classifying based on those features. Support Vector Machines are geometric binary classifiers. k-Nearest-Neighbor and Decision Forests are n-ary classifiers. These sorts of functions are almost certainly part of the machinery of human cognition, but at a level far below anything we recognize as any sort of sophisticated qualia.
And then there are ML techniques which don't seem to have any relation to any human mental function, such as natural-language classification built on Latent Semantic Analysis.
So, yes, you can make that comparison. But you shouldn't.
-
-
Friday 31st August 2018 10:51 GMT Not also known as SC
Definition of Trolling
So has El Reg gone down the line of changing the definition of a troll from 'prat who likes to get a rise out of people on-line' to 'anyone you disagree with' or is trolling the term the researchers used? Hate speech is hate speech - it isn't trolling - and calling it trolling, instead of what it is, legitimises it to a degree because 'it's only trolling'.
-
Friday 31st August 2018 11:06 GMT Uberior
Dear Costumer...
As the latest batch of spam emails start.
Try writing rules to explain why "FUD" on this page, is probably completely different to the "FUD" shouted by a Glaswegian.
For those who don't know fud is a Scottish synomym for =mid("scunthorpe",2,4).
Then have a quiet smirk every time my local newspaper auto-censors the name of one of the local villages to Horton-****-Studley.
-
Friday 31st August 2018 11:12 GMT ivan5
Not AI
The problem is this is all machine learning that relies on a db to lookup and compare to. That db does not have all the possible combinations and permutations of words and images possible and is biased by the person/s that put it together. Therefore it will never work as people want it to.
In this case we should be thankful that we don't have a real AI for the simple reason it most probably wouldn't agree with our definition of 'hate speech, which is generally different for everyone, and would upset more people than it appeases.
-
Friday 31st August 2018 16:03 GMT Anonymous Coward
Re: Not AI
"which is generally different for everyone"
And there's the problem, it shouldn't be, it can't be, in a civilized society.
As in "you're allowed your own opinions but not facts"....
Free speech isn't when you get away with calling some of it Hate.
It's either all free speech (and some of it insulting or just plain annoying) or it's not.
Hate speech laws have been used to impose incredible changes to our societies.
I wouldn't go back to London for the world now, glad I saw it when it was British.
Cheers.
-
-
-
Friday 31st August 2018 11:38 GMT Rich 11
They're what now?
These tricks wouldn’t fool humans, but machine learning models are easily blindsighted.
Blindsided. Blindsight would be a good thing.
-
Friday 31st August 2018 12:03 GMT Anonymous Coward
Stuttershock
Side issue, but when you're looking at the ridiculous stock photo El Reg illustrated their article with, remember... some poor sod who was short of a fiver had to pose for that and is now immortalised looking like a tit.
-
-
-
Friday 31st August 2018 14:53 GMT breakfast
Re: The other way around
But Twitter is full of actual literal nazis. Like you can't criticise a racist for having racist opinions without having hundreds of nazi sockpuppet accounts pile on you and that is apparently fine by Twitter's security systems. Reporting them tends to get a "*shrug* What do you expect? It's just nazis" type response.
It's pretty clear by this point that it is by design and by this point Twitter is a site for nazis and the rest of us who use it are incidental to their core mission.
-
Friday 31st August 2018 16:08 GMT Anonymous Coward
Re: The other way around
"Like you can't criticise a racist for having racist opinions"
Why would bother? SJW much?
Don't you know that "the way" is to shun and ignore them?
Let the idiots scream obscenities until they're blue in the face, then we know who they are.
But anyone that calls people NationalSocialists in the current year isn't quite up to speed either.
Most Twitterers can't tell left from right anyways so it down we go down the drain.
-
-
-
-
Friday 31st August 2018 19:13 GMT fred 32
Hate speech doesn't exist.
Yes, lets build an internet where everyone has to speak in code like they have to in china. It stops no one, and only proves how authoritarian the lefts mentality is. So if that's your goal to constantly prove that the right is being legitimately oppressed, go for it.
The whole Mccain thing only proves how arbitrary these standards are, when he was a presidential candidate he was smeared as a racist, white supremacist, likely everything his supporters said was interpreted as hate speech. but now he's the darling pet of the left. The categorizations are purely arbitrary and "convenient".
-
Monday 3rd September 2018 10:47 GMT Anonymous Coward
Yes, lets build an internet where everyone has to speak in code like they have to in china. It stops no one, and only proves how authoritarian the lefts mentality is. So if that's your goal to constantly prove that the right is being legitimately oppressed, go for it.
It disturbs me how many people still believe that the Chinese government is left wing.
-
-
Saturday 1st September 2018 21:59 GMT DerekCurrie
What "AI"? There isn't any.
Humans have yet to create anything that can be called 'artificial intelligence'. Today, we have more like Artificial Idiocy. It's all about the programmers. Blame them. Blame the horrifically poor state of programming tools whereby memory management and security are guaranteed to fail. Blame the hypeHypeHYPE! that is detonated in our faces by marketing morons (vs mavens) whenever someone wants to sell us something inadequate and lame.
These systems are called 'Expert Systems' whereby computers scan for triggers as input, then scan through databases for relevant, associated output. That's it. Adding in the ability to take new input and associate it with new output, 'machine learning' is the latest addition we're working on for ES. It's messy with messy results. But the blame still goes to the programmers who, as I pointed out, have lousy programming tools.
-
Monday 3rd September 2018 08:41 GMT Blougram
Easily fooled
Such algorithms might force trolls to become somewhat more creative, but it won't stop many of them. The fact that they are all trained on historical data and have no common sense means that a bit of creativity is all it takes to fool them. I once played around with Google's API, and these were all deemed innocuous:
1. An ASCII image of a middle finger;
2. A courteously worded request that the person remove themselves from the gene pool;
3. "Yo mama" insults couched in courteous language.
4. Vile abuse in Elizabethan English, etc.
-
Monday 3rd September 2018 10:45 GMT Anonymous Coward
Curious game, Professor Falken.
The only way to win is to delete all social media, and for the rest of the web use Adblock Plus (or uBlock Origin) with the Mute filter to remove all comments sections from websites, and the YouTube Pure Video Experience filter to do the same for YouTube. And NoScript, of course, which will catch anything that hasn't already been caught.
This way I still get to see everything of value which is ever posted on comment sections (that is, nothing).
-
Friday 7th September 2018 19:53 GMT Michael Wojcik
ML adaptation
They can’t readily adapt to new information beyond what’s been spoonfed to them during the training process.
This may be true of the systems examined in this study (I haven't bothered reading the paper, because, frankly, it doesn't look terribly interesting1). It is not, however, true of ML system in general, as Katyanna seems to imply. There are a great many ML systems which can refine their classifiers in production, using unsupervised or semi-supervised learning.
Sometimes that's as simple as kernel augmentation - expanding category features when novel data accompanies a strong match. A more sophisticated approach is having other systems (typically human judges, though they don't always know they're filling this role) label some errors after processing, and feeding those back in as adversarial inputs. For this particular use case, sentiment analysis on replies to a post could be used to build a disagreement graph (basically an inverse reputation network representation) for a conversation and identify hotspots for more in-depth analysis.2
Assuming the Reg's precis is accurate, the authors suggest that the training set is more important than the algorithms. That may well be true for this set of systems (and it agrees with similar studies on, for example, sentiment-analysis systems), but I'm not convinced it's true in general. I suspect a continuous-learning system with heterogeneous feedback channels and a decent world model would eventually do better than any of the systems under discussion, regardless of what training set were used. But building such a system is expensive and goes against the research direction of many of the big players, particularly Google.
1Which is not to say that I don't approve of the work. Much research is not particularly interesting, but still useful, and this is particularly true of research which tempers the claims of inventors.
2Yes, we would not want a system to automatically flag as "bad" a post or contributor simply due to controversy. I'd hope that would be obvious. But it probably is not.