From self-harm to terrorism, online recommendations cast a deadly shadow

4/3/2019 18:06:00

My eye was caught by a headline in Wired magazine: “When algorithms think you want to die”. Below it was an article by two academic researchers, Ysabel Gerrard and Tarleton Gillespie, about the “recommendation engines” that are a central feature of social media and e-commerce sites.


Everyone who uses the web is familiar with these engines. A recommendation algorithm is what prompts Amazon to tell me that since I’ve bought Custodians of the Internet, Gillespie’s excellent book on the moderation of online content, I might also be interested in Safiya Umoja Noble’s Algorithms of Oppression: How Search Engines Reinforce Racism and a host of other books about algorithmic power and bias. In that particular case, the algorithm’s guess is accurate and helpful: it informs me about stuff that I should have known about but hadn’t.


Recommendation engines are central to the “personalisation” of online content and were once seen as largely benign. Sure, they tended to lock people into their own filter bubbles and might therefore contribute to political polarisation or fragmentation, but on the other hand, they were giving people what they wanted, so where’s the harm in that?


Quite a lot, as it turned out, but it took us a while to realise that. Gerrard and Gillespie open their article by revisiting the tragic case of Molly Russell, the young teenager who sought out images of self-harm before she took her own life in 2017. It was later discovered, the authors report, “that these images were also being delivered to her, recommended by her favourite social media platforms. Her Instagram feed was full of them. Even in the months after her death, Pinterest continued to send her automated emails, its algorithms automatically recommending graphic images of self-harm, including a slashed thigh and cartoon of a young girl hanging.”


In the wake of scandals and tragedies such as Molly Russell’s death, social media companies are regularly excoriated for “hosting” such material on their sites. (They’re actually publishing them, but a legal loophole allows them to pretend that they’re not and thereby escape legal liability.) Faced with rising public concern, the companies have ramped up their “moderation” (ie censorship) operations, maintaining large teams of contractors to monitor and, if necessary, remove offending content from their sites. Given the scale of online content, this task makes that of Sisyphus look like a doddle, but at the moment the companies are in denial about that. In the meantime, many of the people contracted to carry out the impossible task continue to be traumatised by it.


The sad truth about the internet is that it holds up a mirror to human nature and some of what we see reflected in it is pretty repulsive. (One of the videos reportedly used in training sessions for Facebook moderators in Arizona, for example, shows a man being murdered. “Someone is stabbing him, dozens of times, while he screams and begs for his life.”) The social media companies, which like to portray their mission as enabling people to express themselves (“Broadcast yourself” was an early motto of YouTube, for example), protest that they can hardly be blamed if people are horrible, express themselves in unacceptable ways and generally foul the online nest.


Their reliance on recommendation engines rather undermines this comforting narrative. As Gerrard and Gillespie point out: “Social media platforms not only host this troubling content, they end up recommending it to the people most vulnerable to it. And recommendation is a different animal than mere availability.”


Spot on. And there’s a growing body of evidence to support this. In areas such as self-harm, misinformation, terrorist recruitment or conspiracy theories, social media platforms don’t just make this content easy to find – their recommendation engines help to amplify it.


YouTube’s recommender, for example, sometimes seems to boost conspiracy theories and extremist content, to the point where (as the “alt-right” discovered years ago) it might be the most powerful “radicaliser” on the planet.


If recommendation engines are indeed significant generators of online harms, what might be done about them? One approach involves trying to make them less crass and more sensitive to context. Just because somebody suffering from depression searches for information about the condition doesn’t mean that they’re interested in suicide, for example. Improving the inferential processes of software is not easy, but it’s feasible, given the will and resources to do it.


“Given the will...”. The bigger problem is that recommendation engines exist for a reason – to keep users on a site and increase their “engagement” with its content. The surest way to keep people engaged is to lure them down wormholes filled with intriguing things.


Which is precisely what these engines are good at – and why social media companies won’t give them up. Nobody who ever had a golden goose voluntarily wrung its neck.





John Naughton-The Guardian | PUKmedia 


Commemoration of 3 Kurdish activists shot in Paris

Thursday, Kurds saluted the seventh anniversary of the assassination of 3 Kurdish women activists in France.


The crime took place on January 9, 2013, targeting 3 female Kurdi...

»  3 Kurdish girls die in a fire in Norway
»  Kurdish teenager plays for several European football clubs

Quarantined citizen in Sulaymaniyah: services provided are impossible for major countries

One of the citizens inside a quarantine center in the city of Sulaymaniyah said that what the medical and administrative staff provide is not possible for major countries to provide, and that the q...

»  Susan Rice: U.S. has ‘sold out the Kurds’ with Syria move
»  INTERVIEW: Kurdish leader Ilham Ahmed on security in North and East Syria

‘Wash Our Hands? Some People Can’t Wash Their Kids for a Week.’

In northwestern Syria, where a million people who have fled the civil war are sheltering in muddy tent camps and abandoned buildings, the spread of the coronavirus could wreak untold disaster on a ...

»  Postponement of burial ceremonies for Kojo mass graves victims
»  Consultations begin to choose new Iraqi PM