Thorn also recommends more research into how young people use blocking and reporting to combat threats online.
Stroebel told Ars that she wants to better understand why kids choose to block versus report in certain circumstances. She’s worried that kids overwhelmingly choose to block users sharing fake nudes, rather than reporting the attack and prompting a broader platform response that could help minimize harm to more users. It’s also concerning, she said, to think that particularly young users who feel imminently in danger may wrongly expect that reporting a fake nude triggers immediate action.
Other changes tech companies could consider include prioritizing stronger deepfake detection, Duffield recommended.
Perhaps most urgently, Stroebel told Ars that institutional support is needed to stop fake nudes from becoming a normal part of the teenage experience.
Right now, she said that kids want to know how to protect themselves and their friends, and, in this moment, there just isn’t a clear message that making fake nudes carries serious consequences. Even schools where scandals have hit don’t seem to publicly share their plans to support kids once the backlash dissipates, potentially leaving open a debate over what’s right and wrong that Stroebel said leaves young people in harm’s way the longer it goes unsettled.
“There is not a uniform institutional response, whether we are talking about at the policy level, at the family unit level, at the school level, and normalization ultimately ends up existing when it is widely accepted and it is anticipated,” Stroebel said. “If there isn’t an institutional response for too long” explicitly clarifying “that this is harmful behavior, then it becomes [about] individual opinion on if this is harmful or not,” Stroebel said.
That risks leaving today’s absent standard as the status quo and keeping “the control in hands of the person making a choice of whether or not they’re going to create non-consensual abuse imagery,” Stroebel said.
Many teens told Thorn that they’re already acknowledging the abuse and already asking for help. As a 15-year-old girl explained to Thorn, “even though it’s not real,” victims currently “have no way to prove that, and they can’t just deny it, because their face is most likely on it.”
