*This is an AI-powered machine translation of the original text in Portuguese

Content moderation, understood as the set of governance mechanisms that structure participation on a platform to facilitate cooperation and prevent abuse [1], has been widely employed by social media and content-sharing application providers on the internet, as well as in search results [2].

In just the first quarter of 2020, Facebook removed or took action on about 1.8 billion inauthentic accounts and content involving nudity and sexual activity, violence, dangerous organizations (terrorism and hate), hate speech, drugs and firearms, child pornography, suicide, and self-harm. The latest Twitter transparency report reveals the suspension of 244,188 accounts for the exploitation of minors (91% using technology) and 115,861 accounts for violent content (87% using proprietary computational tools) [3]. According to the first quarter of 2020 report [4], YouTube moderators manually removed 399,422 videos, while 5,711,586 videos were removed by automated tools [5]. Google's transparency report, on the other hand, indicates that over four billion URLs had their delisting requested by users.

Due to the central position of digital platforms in shaping the current public discourse space, the trend in Europe [6] and the United States of America [7] is to view moderation as crucial not only for promoting the quality of the provided services, reflecting the private interests of the providers, but also for ensuring the individual's freedom of expression, fundamental rights, and the democratic environment. This is why there is a need for convergence of terms of use with the state legal framework in contemporary constitutional democracies. On the other hand, it is increasingly recognized that this content curation implies the platforms' responsibility in adopting appropriate governance procedures and available technical means to prevent abuses and mitigate the impacts caused by harmful content propagation. In other words, it is emphasized that content moderation duties for the platforms have indeed become hosts of the space for public discourse and "curators" of freedom of expression, so their rules cannot be limited to the contractual relationship with users and must incorporate national legislation, being compatible with the rules regarding the legality of content disseminated in the public sphere [8]. This includes not only criminal offenses [9] but also national rules on advertising [10] in communication vehicles, online advertising practices, and content boosting.

Considering the public interest surrounding it, the private governance systems established by providers for content moderation on the internet have been the focus of international debates, expressing concerns about the lack of transparency and arbitrariness by the providers [11]. In this sense, civil society entities have promoted initiatives and promulgated documents to encourage the adoption of transparency mechanisms, including the Santa Clara Principles [12], the Corporate Accountability Index [13], and the Electronic Frontier Foundation's principles on content filtering [14].

These documents list the need for application providers to disclose the numbers of applied measures (flagging, blocking, takedown) and notify the affected user, providing corresponding reasons, as well as the opportunity for their contestation. Legislative initiatives have also incorporated some procedural duties of transparency, such as the German NetzDG law [15] and the European regulation proposal on the prevention of online dissemination of terrorist content [16], and the Recommendation on the Responsibility of Internet Intermediaries of the Council of Europe [17].

Although platforms have started to disclose moderation activity numbers, driven by government pressures [18], they have been timid in providing effective mechanisms for contestation, and decisions by the providers regarding the removal of content and accounts remain arbitrary [19].

On the other hand, academic and digital activist debates [20] have pointed out the insufficiency of these measures. Note that all of them refer to good practices after the decision is made by the application provider regarding measures on content or accounts. However, a series of concerns have been raised about the adequacy and impacts on fundamental rights arising from the decision-making process adopted by the platforms themselves. This decision-making process basically involves moderation teams and computational tools for content detection or automated decision-making.

Regarding moderators, there are three orders of questioning. The first concerns the adequacy of moderation standards, typically modulated for constitutional democracies, assuming, for example, that misinformation and hate are created and tolerated only by a minority when, in fact, they may have the support of the majority of the population and even have government origins [21]. The second concerns the competence of moderators to deal with local issues, including language skills and relevant cultural and political aspects, considering that moderation is not always national and may even be outsourced to foreign companies [22]. Recently, Facebook made commendable efforts to establish and disclose its Oversight Board [23] for content moderation, emphasizing the chosen team, composed of members known for their defense of freedom of expression and human rights. However, the Oversight Board decides on a small number of selected cases, aiming to provide guidelines for moderation teams, which will effectively make decisions about content moderation on the social network. Little is known about the composition, training, and capabilities of moderation teams on any of the currently dominant social networks. The episode in Myanmar [24], in which Facebook was unable to detect how the social network was used as a weapon to incite violence against the Rohingya ethnic minority, illustrates these two concerns well. The third concern relates to the working conditions of the moderator, who is constantly exposed to degrading images and content, which can affect their mental health [25].

In turn, automation in moderation for detecting and deciding on abusive content, both ex ante and ex post, has been encouraged by regulations such as the European Union Copyright Directive, the German NetzDG, and the Australian Abhorrent Violent Material law, which require the adoption of measures in an efficient manner, sometimes within 24 hours. However, critics of this legislation point out the risk of overreliance on these tools [26] and that the platforms' discretion in their use can lead to counterproductive results [27], such as violations of freedom of expression due to excessive blocking [28] and other human rights if there is no transparency and accountability regarding these tools [29], particularly regarding their role in decision-making, their explainability, i.e., relevant classification criteria, often made opaque by the algorithm employed [30], their level of accuracy, and the possible incorporation of discriminatory social traits [31] or distortions in accuracy based on gender or race [32].

Finally, considering that, although they are not responsible for their creation, the propagation of content is enabled by the platform, it has been highlighted the need for application providers to employ technical means available to mitigate impacts eventually caused by moderation actions or omissions. Such action includes not only restoring accounts and mistakenly excluded content but also using available means to correct the information or inform about the abusive nature of the content to those exposed to it or, at the very least, those who interacted with it [33].

Based on recent international debate, it is possible to observe that the treatment given by PL nº 2630/2020 to content moderation is incomplete. This is because, in addition to not stipulating the need for the terms of service of application providers to incorporate parameters of illicit content, in accordance with national legislation on communication and advertising, they only address one aspect of moderation, namely, what occurs after the providers' decision on content, with the obligation to offer an opportunity for defense and appeal, in addition to the disclosure of moderation metrics.

There are lacking rules of transparency and accountability for what happens in the platform's decision-making process, both regarding moderators and regarding the algorithms employed, as well as in providing clear mechanisms for complaint to offended users and the providers' efforts to mitigate the impacts of harmful content propagated on the network.

In this regard, the following points deserve special attention regarding private content moderation by providers:

1. The rules elaborated by the application provider on content moderation must be widely debated, considering their specific impacts on public discourse, involving different representatives of civil society, considering the relevant role of the discourse conveyed on the platforms for the exercise of individual rights, social cohesion, and democracy [34]. Both the rules and the tools for their application, including automatic and human moderation, must be described transparently. Platforms must be accountable for both the rules and their application, ensuring that they adopt appropriate procedures and measures, considering their essential role as forums for public discourse and stages for the exercise of a range of online human rights, including freedom of expression, but also freedom of information, dignity, and privacy;

2. Content moderation against misinformation should cover not only social networking services that allow users to post content online for social networks or the general public (Facebook, Twitter, etc.), but also video posting and sharing services (YouTube), as well as search services that direct the user to disinformative content (Google search). Therefore, the article on moderation should not limit its reference to social network providers but encompass application providers in general;

3. Attention should also be given to advertising moderation. The provider has the opportunity and duty to filter the content prior to its boosting, considering that they will obtain financial returns for dissemination. This duty needs to be aligned with transparency rules for identifying the sources of funding for the dissemination of illicit content, subjecting itself to existing advertising regulations in Brazil;

4. Human moderation activity must have a reviewing team appropriate to the volume of communications, with linguistic competence and adequate knowledge of the local culture for content examination;

5. Computational tools that employ artificial intelligence have increasingly proven to be cost-effective and effective in detecting and classifying illicit content. However, they are fallible and must comply with ethical parameters to prevent them from being counterproductive or threatening other fundamental rights, including acceptable standards of accuracy for their use, transparency, explainability, and bias control;

6. Given the fallibility of moderation and the possibility of exclusion or marking of the public expression of lawful and legitimate content, the power of platform moderation cannot be arbitrary, and it must also incorporate online mechanisms for contestation, easily accessible to the user, ensuring effective adversarial proceedings, as well as prompt and justified responses by the providers regarding their final decision;

7. There must be transparency by the providers regarding the team employed for content moderation, contracted companies, and technical means, particularly regarding the artificial intelligence systems used;

8. Application providers must, to the extent possible and technically feasible, seek to inform users who have come into contact with content that violated the terms and conditions of the contract or that was subject to fact-checking by independent sources. This informative measure by the platform aligns with its traffic duties aiming to contribute to the mitigation of the effects of misinformation or abusive content circulation and propagation. This measure should reach, at a minimum, those who interacted explicitly with abusive content during the time it was maintained on the network and, when possible, all those who were exposed to the content.

   [1] GRIMMELMANN, James. The virtues of moderation. Yale Journal of Law & Technology, 2015, v. 17. Available at: https://digitalcommons.law.yale.edu/cgi/viewcontent.cgi?article=1110&context=yjolt. Access: on Aug 27, 2020.

   [2] Barrett, Paul M. Who moderates the social media giants? In: New York University Stern Center for Business and Human Rights, 2020. Available at: https://bhr.stern.nyu.edu/blogs/2020/6/4/who-moderates-the-social-media-giants. Access on: Aug 27, 2020.

   [3] TWITTER. Transparency. Rules Enforcement. Available at: https://transparency.twitter.com/en/twitter-rules-enforcement.html. Access on: Aug 28, 2020.

   [4] GOOGLE Transparency Report. Compliance with YouTube Community Guidelines. Available at: https://transparencyreport.google.com/youtube-policy/removals. Access on: Aug 28, 2020.

   [5] GOOGLE Transparency Report. Compliance with YouTube Community Guidelines. Available at: https://transparencyreport.google.com/youtube-policy/removals?total_removed_videos=period:Y2020Q1;exclude_automated:human_only&lu=total_removed_videos. Access on: Aug 28, 2020.

   [6] JØRGENSEN, Rikke Frank. When Private Actors Govern Human Rights. Research Handbook on Human Rights and Digital Technology 346, 363. In: Ben Wagner, Matthias C. Kettemann, Kilian Vieth (Eds.) 2019; ROBERTS, Sarah T. Behind the Screen. Content moderation in the shadows of social media, Edward Elgar: London, 2019 p. 33 ss.

   [7] CITRON, Danielle K; FRANKS, Mary Anne. The internet as a speech machine and other myths confounding section 230 Reform. Boston University School of Law Public Law and Legal Theory Paper, 2020, n. 20-8; CITRON, Danielle K; WITTES, Benjamin. The internet will not break: Denying bad Samaritans § 230 immunity. Fordham Law Review, v. 86, pp. 401–423.

   [8] WIELSCH, Dan. Network frameworks: Terms and conditions of use - Code - Communication standards. In: Ricardo Campos, Georges Abboud, Nelson Nery Jr. (Eds.) Fake News and Regulation, 2nd Edition, RT: São Paulo 2020, p. 91 ss.

   [9] In Brazil, at a minimum, the practices outlined in the criminal offenses in articles 138, 139, 140, 147, 282, 283, and 284 of Law 7,209 of July 11, 1984 (Criminal Code), in Law 7,716 of January 5, 1989, and articles 323 and 337 of the Electoral Code, and articles 33, paragraph 4, and 34, paragraph 3, of Law 9,504 of September 30, 1997.

   [10] In Brazil, where applicable, Laws No. 4,680/65 and Law No. 12,232/10 and the provisions of article 36 of Law 8,078/90.

   [11] HAMILTON, Rebecca J. Governing the Global Public Square. Harvard International Law Journal, v. 62, 2021 (forthcoming); JØRGENSEN, Rikke Frank; ZULETA, Lumi. Private Governance of Freedom of Expression on Social Media Platforms. Nordicom Review, 2020, pp. 51-67; CAPLAN. Ronyn; GILLESPIE, Tarleton. Tiered Governance and Demonetization: The Shifting Terms of Labor and Compensation in the Platform Economy. Social Media + Society, 2020; GORWA, Robert; BINNS, Reuben; KATZENBACH, Christian. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society, 2020; KETTEMANN, Matthias. Human Rights and Political Participation in the Digital Age, 2020; TIEDEKE, Anna Sophia; KETTEMANN, Matthias. Back up: can users sue platforms to reinstate deleted content? Internet Policy Review Journal on internet regulation, vol. 9, 2020.

   [12] THE SANTA CLARA PRINCIPLES on Transparency and Accountability in Content Moderation. Available at: https://santaclaraprinciples.org/. Access on: Aug 26, 2020.

   [13] Ranking Digital Rights Header Branding. Available at: https://rankingdigitalrights.org/index2019/. Access on: Aug 26, 2020.

   [14] GEBHART, Gennie. Who has your Back? Censorship Edition 2019. Electronic Frontier Foundation. Available at: https://www.eff.org/wp/who-has-your-back-2019. Access on: Aug 26, 2020.

   [15] Gesetz zur Verbesserung der Rechtsdurchsetzung in sozialen Netzwerken, Section 2, which requires the production of transparency reports. See translation of the German law into Portuguese in: Ricardo Campos, Georges Abboud, Nelson Nery Jr. (Eds.) Fake News and Regulation, 2nd Edition, RT: São Paulo 2020, p. 337 ss.

   [16] EUROPEAN COMMISSION. Regulation of the European Parliament and of the Council on preventing the dissemination of terrorist content online, 2018. Article 8(2) of said regulation requires the publication of content removal requests.

   [17] https://search.coe.int/cm/Pages/result_details.aspx?ObjectID=0900001680790e14

   [18] MASNICK, Mike. How Government Pressure Has Turned Transparency Reports from Free Speech Celebrations to Censorship Celebrations, Techdirt, published on Apr 17, 2018. Available at: https://www.techdirt.com/blog/?d=17&m=4&y=2018. Access on: Aug 26, 2020.

   [19] GILLESPIE, Tarleton. Custodians of the Internet. Platforms, content moderation, and the hidden decisions that shape social media, New Haven 2018, p. 24 ss.; KLONICK, Kate. The New Governors: The People, Rules, and Processes Governing Online Speech. 131. Harvard Law Review, 1598.

   [20] See globalvoices.org and datasociety.net.

   [21] WU, Tim. Is the First Amendment Obsolete? Michigan Law Review, vol. 117, 2018, pp. 547-557; HAMILTON, Rebecca J. Governing the Global Public Square. Harvard International Law Journal, v. 62, 2021 (forthcoming);

   [22] GILLESPIE, Tarleton. Custodians of the Internet. Platforms, content moderation, and the hidden decisions that shape social media, New Haven. Yale University Press, 2018; ROBERTS, Sarah T. Digital detritus: ‘Error’ and the logic of opacity in social media content moderation. First Monday, 23(3); SUZOR, Nicolas P.; WEST, Sarah Myers; QUODLING, Andrew; YORK, Jillian. What do We mean when We talk about transparency? Toward meaningful transparency in commercial content moderation. International Journal of Communication 2019, vol 13. Available at: https://ijoc.org/index.php/ijoc/article/view/9736. Access on: Aug 26, 2020.

   [23] Available at: https://www.oversightboard.com/news/announcing-the-first-members-of-the-oversight-board/?fbclid=IwAR1oZWZjr3eJqa06kwLkWJZTaSG7H8M9bnPz2qNu0RlbBNSXopEjPnZwZuI. Access on: Aug 27, 2020.

   [24] STECKLOW, Steve. Inside Facebook’s Myanmar Operation Hatebook: A Reuter’s Special Report, REUTERS, published on Aug 15, 2018. Available at: <https://www.reuters.com/investigates/special-report/myanmar-facebook-hate/ [https://perma.cc/2QWF-D6L8>. Access on: Aug 26, 2020.

   [25] ROBERTS, Sarah T. Behind the Screen. Content moderation in the shadows of social media, Yale University Press: London, 2019 p. 33 ss.

   [26] LI, Sydney; WILLIAMS, Jamie. Despite What Zuckerberg’s Testimony May Imply, AI Cannot Save Us. Electronic Frontier Foundation, published on Apr 11, 2018. Available at: https://www.eff.org/deeplinks/2018/04/despite-what-zuckerbergstestimony-may-imply-ai-cannot-save-us. Access on: Aug 26, 2020.

   [27] BLOCH-WHEBA, Hannah. Automation in Moderation, Cornell International Law Journal 2020.

   [28] BAR-ZIV, Sharon; ELKIN-KOREN, Niva. Behind the scenes of online copyright enforcement: Empirical evidence on notice & takedown. Connecticut Law Review, 2018; URBAN, Jennifer; KARAGANIS, Joe; SCHOFIELD, Brianna. Notice and takedown in everyday practice. UC Berkeley Public Law Research Paper, Mar 15, 2016.

   [29] GORWA, Robert; BINNS, Reuben; KATZENBACH, Christian. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society 2020.

   [30] ANANNY, Mike.; CRAWFORD, Kate. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society , 2018, 20(3), pp. 973–989. GORWA, Robert; Garton ASH, Timothy Garton. Democratic Transparency in the Platform Society. In: Persily N, Tucker Josh, Social Media and Democracy: The State of the Field and Prospects for Reform. Cambridge, UK: Cambridge University Press, 2020.

   [31] BINNS, Reuben, VEALE, Michael, KLEEK, Max Van; SHADBOLT, Nigel. Like trainer, like bot? Inheritance of bias in algorithmic content moderation. In: International conference on social informatics, pp. 405–415. Berlin: Springer, 2017.

   [32] ANGWIN, Julia; GRASSEGGER, Hannes. Facebook’s Secret Censorship Rules Protect White Men, But Not Black Children. In: ProPublica. Available at: <org/article/facebook-hate-speech-censorship-internaldocuments-algorithms>. Access on: Aug 25, 2020; HOFFMANN, Anna Lauren. Where fairness fails: Data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society 2019, vol. 22, ed. 7, pp. 900–915.

   [33] ECKER, Ullrich K H; SWIRE-THOMPSON, Briony. Misinformation and its Correction: Cognitive Mechanisms and Recommendations for Mass Communication (2018). Available at: https://www.researchgate.net/publication/317603082_Misinformation_and_its_Correction_Cognitive_Mechanisms_and_Recommendations_for_Mass_Communication

   [34] Cf. KETTEMANN/SCHULZ, Setting Rules for 2.7 Billion. A (First) Look into Facebook’s Norm-Making System (Leibniz Institute for Media Research Working Paper #2, 2020).

   *Co-authored with Ricardo Campos, Matthias Kettemann, Juliana Abrusio, and Giovanni Sartor. Originally published on Conjur.

   **Image freepik.