Co-designing Platform Governance Policies: Tackling malicious flagging and de-platforming with impacted social media users

This report provides social media platforms with user-centred and research-informed recommendations to improve the design and effectiveness of their flagging and appeals tools. Research has shown these affordances to be inadequate at tackling online abuse, and to provide malicious actors with opportunities to exploit strict platform governance to de-platform users with whom they disagree. This has disproportionately affected marginalised users like sex workers, LGBTQIA+ and BIPOC users, nude and body-positive content creators, pole dancers, but also journalists and activists.

The idea underpinning this report is that content moderation often fails to take the human experience into account to prioritise speed and platform interests, lacking in the necessary empathy for users who are experiencing abuse, censorship, loss of livelihood and network as well as emotional distress. As a result, this report is a free resource for both users to feel seen in a governance process that often erases them and, crucially, for platform workers to avoid escaping stakeholder engagement in the drafting of their policies.

The workshops were supported by Northumbria University’s Policy Support Fund and by the Engineering and Physical Sciences Research Council, and ran together with Ageless Citizen Innovation Fellow and product designer Dr Henry Collingham. The World Wide Web Foundation’s TPDL Playbook was crucial in developing an accessible and open-ended format to harness the power of a diverse set of participants from different walks of life. Thanks to the structure provided by the Playbook, and to Superbloom’s co-facilitation during the workshops, the TPDLs I ran became a space in which we could harness participants’ experience and expertise to learn directly from those affected by technology, for policymakers, tech companies and researchers to take their stories into account.