Moderation of illegal content and social media scraping. Privacy and data protection constraints in the processing of publicly available data by law enforcement authorities

Essays – peer-reviewed https://doi.org/10.6092/issn.1825-1927/18870 i-lex – Rivista di Scienze Giuridiche, Scienze Cognitive ed Intelligenza Artificiale. Vol. 16 n. 2 (2023) ISSN 1825-1927 Moderation of illegal content and social media scraping. Privacy and data protection constraints in the processing of publicly available data by law enforcement authorities∗ Flavia Giglio† Abstract. Social media scraping can be used in connection to a wide range of law enforcement tasks, and to investigate various forms of crime. At the same time, these practices create peculiar fundamental rights concerns, linked to the rights to privacy, data protection and freedom of expression. To complicate the matter, the recently adopted EU legal instruments on the moderation of illegal content create new grounds for potential social media monitoring activities by law enforcement authorities, possibly leading to an ehanced use of social media scraping techniques. In light of these legislative novelties, the paper aims to present potential fundamental rights criticalities of the use of social media scraping technologies in this context, when aimed to collect and process publicly available data. Keywords: Social media scraping, content moderation, law enforcement authorities, crime prevention 1 Introduction. Social media scraping as a form of Open Source Intelligence Open Source Intelligence (OSINT) refers to actions aimed to gather and analyse data that is accessible to any individual or entity, within the remits of the fight against crimes. Considering open source data as the raw material that can be extracted by publicly available surces to collecte information on a given subject, OSINT represents the stage where such information is discriminated and distilled in order to respond to a question that guides the screening of such sources.1The benefits of OSINT for the activities of law enforcement authorities (LEAs) in the criminal justice domain has been recognised by the European Union (EU), while also sparking discussions about potential fundamental rights-related concerns on such practices. In this regard, it was observed that the use of OSINT should be put in place with sufficient guarantees to protect publicly available personal data, such as including settings that allow the end-users to adapt the possibility to access personal data according to the facts and circumstances of the use.2The practice to monitoring social media content in order to extract publicly available data from social networks is known as Social Media Intelligence (SOCMINT), at it falls under the broader umbrella of OSINT practices. Potential privacy-related criticalities of the LEAs’ activities aimed to repurpose publicly accessible data from the Internet are connected to the use of web crawling and web scraping technologies. Web crawlers are tools used to navigate the web in order to gather sources and index them, according to certain criteria. Web scraping, instead, refers to the act of extracting data from previously identified online sources.3Therefore, data scraping allows to retrieve unstructured data from the Internet. This data ∗This publication was prepared in the context of the FERMI project, funded by the European Union’s Horizon Europe research and innovation programme under grant agreement No 101073980. †KU Leuven, Centre for IT & IP Law (CiTiP); fla[email protected] 1. Steele, Robert David. “Open source intelligence.” Handbook of intelligence studies 42.5 (2007): 129-147. 2. Shere, Anjuli. “Reading the Investigators their Rights: A review of literature on the General Data Protection Regulation and open-source intelligence gathering and analysis.” The New Collection 3, Oxford, 2020, 3-21. 3. Vanden Broucke, Seppe, Baesens, Bart, “From web scraping to web crawling.” Practical Web Scraping for Data Science. Best practices and examples with Python, Apress, Berkeley, 2018, 155-161. Copyright © 2023 Flavia Giglio This work is licensed under the Creative Commons BY License. https://creativecommons.org/licenses/by/4.0/ 17 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) can, in turn, be stored and analysed in a central database. In the context of SOCMINT, this process can be used for emotion and sentiment analysis purposes, for example in order to predict the outcome of an election based on what users share online. For this type of use, retrieving posts or comments that mention a specific candidate’s name may not be necessary, as sentiment analysis algorithms can infer certain information about individuals’ opinions even when the analysed content is not explicit.4 In the context of LEAs’ activities, the results of an horizon scanning exercise conducted in 2022 on the use of OSINT by investigators of eight different countries leads to the conclusion that LEAs make use of the latter instrument of data scraping, as they analyse data available online. This is especially true in the context of social media analysis, considered as one of the most important capabilities available to the investigators.5The study also highlighted the ability of such instruments to return a massive amount of data, with little capacity to limit the volume of the retrieved information. This characteristic rises doubts about how to assess the proportionality of an investigation with regard to the access to data.6 When it comes to detecting a certain type of content online, social media scraping techniques that retrieve data from a certain source usually work along with natural language processing (NLP) libraries, as a means to organise the gathered data in a structured way. NLP algorithms allow to classify scraped content against a database of previously provided examples, normally sets of words that illegal or harmful content is likely to contain. As it will be further explored in the next sections of this paper, such algorithms suffer some fundamental limitations from a technological point of view. These limitation add up to those that characterised data scraping techniques in general. The present paper aims to analyse the privacy and data protection constrains that should in in place in social media scraping practices put in place by LEAs, with a specific focus on social media monitoring and content moderation activities. It first highlights why social media content can be a source of publicly available data whose data protection guarantees should be kept in high consideration, due to the connection between the rights to privacy and data protection and the enjoyment of online freedom of expression. In turn, it analyses the existing EU content moderation landscape, with special focus on some of the existing provisions that exemplify the enhanced presence of LEAs, and public actors in general, in the context of content moderation. Afterwards, it highlights the fundamental rights concerns arising from the use of NLP algorithms to carry out content moderation and categorise publicly available data scraped from social media. Finally, it explores the legal status of publicly available data as defined in the EU legal framework, in the European Court of Human Rights (ECtHR) and in the EU Court of Justice (CJEU) case law. Using the principles extracted by this normative framework, potential social media scraping practices performed by LEAs are assessed in light of the rights to privacy and data protection. The next section of the paper investigates the privacy and data protection concerns arising from the processing of publicly available data extracted by social media content. Due to the pivotal role played by social media in fostering freedom of expression, the protection of this category of publicly available data from unduly interferences of public authorities is particularly challenging as well as important in ensuring the enjoyment of fundamental rights. 2 The peculiar status of social media as a source of publicly available data LEAs’ practices involving social media scraping are a type of OSINT presenting its own peculiarities in relation to fundamental rights. Social media sources are by nature characterised by broad visibility and accessibility. When it comes to monitoring activities of LEAs, this characteristic creates an asymmetry between the visibility of online social life and police activities. This asymmetry can create concerns, especially with the view of extreme scenarios when the LEAs’ monitoring activities become to overreaching that every subject is treated as a potential suspect, with a consequent “criminalisation of online spaces”.7Moreover, in traditional social media platforms, expressions made through posts or comments can be easily associated with their authors. An example is Facebook, whose community guidelines requiring to use the users’ legal name was subject to debate, due to 4. Khder, Moaiad Ahmad, “Web scraping or Web crawling. State of Art, Techniques, Approaches and Application.” International Journal of Advances in Soft Computing & Its Applications 13.3 (2021), 144-168. 5. Bayerl, Petra Saskia, et al., “Future Challenges and Requirements for Open Source Intelligence in Law Enforcement Investigations: Results from Horizon Scanning Exercise.” European Law Enforcement Research Bulletin 21 (2022), 21-38. 6. Idem. 7. Trottier, Daniel, “Open source intelligence, social media and law enforcement: Visions, constraints and critiques.” European Journal of Cultural Studies,18.4-5 (2015), 530–547. https://doi.org/10.6092/issn.1825-1927/18870 18 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) the alleged disregard for situations where anonymity can benefit to the expression of minorities and human rights activists.8 Therefore, compared to traditional public spaces that could be monitored by LEAs, the expectation of anonymity of individuals expressing themselves on social networks is largely diminished. These privacy-related preoccupations can have a fallout on another fundamental right: freedom of expression. In the era of social media, social networks amount to a new public forum of great importance for exchange of ideas and information, ultimately influencing the public debate.9Given this status of social networks as facilitators to freely express opinions, it has become crucial to consider the implications on freedom of expression inherent to any activity aimed to monitoring or analysing what is shared online.10 This is especially in light of the fact that social media, and the Internet as a whole, have become a powerful instrument for political activism and social movements, facilitating the organisation of demonstrations and resistance acts everywhere in the world, including in less democratic countries. In this context, the creation of content on social media platforms, and the interactions deriving from it, give a powerful chance to enhance collaboration and communication.11 The disproportionate use of social media scraping technologies can result in a state of mass surveillance capable of having a chilling effect on freedom of expression. This risk is enhanced when considering that, in line with the OSINT logic, information about the content posted online can be combined with other data freely available in multiple Internet sources, returning a very comprehensive picture of individuals to LEAs. The dangers of aggregating data from multiple sources was underlined by the UK Information Commissioner’ Office and other data protection authorities in a joint statement on data scraping and data protection, published in 2023. The statement pointed out that this aspect of data scraping could lead to individuals losing power over their personal data, as information that they decide to delete could still be processed and circulated by entities which scraped it. It also acknowledged the risk of “monitoring, profiling and surveilling individuals” among those to be considered when using data scraping techniques.12 Conversely, the expansion of platforms where to express opinions has profoundly changed the landscape of traditional media, affording an unprecedented freedom to individuals to share any type of content online. This enhanced freedom came with the emergence of distortions in the use of social media, such as the sharing of malicious content or the perpetration of harmful actions through online platforms. The misuse of social media can have severe effects to the detriment of persons or society. In such cases, the intervention of LEAs and the enactment of criminal laws protecting the public values at stake is called for.13 The search for a fair balance between the need of LEAs to monitor social media and the protection of fundamental rights is therefore particularly challenging. To complicate this scenario, content moderation of online illegal content emerged as a relatively new area where LEAs can play a role, adding up to the grounds at least for social media crawling, and potentially for social media scraping. The next section of this paper analyses the existing EU framework on content moderation, and highlights how this framework can impact on the role of LEAs in content moderation practices. 3 The EU legislative landscape on content moderation and the role of LEAs The growing attention of the policy debate on the dissemination of forms of illegal or harmful content is demonstrated by the recent legislative initiatives at the EU level to establish duties of care of digital platforms with regard to tackling said content. . The involvement of LEAs in tackling certain types of content can derive from the fact that their dissemination is a criminal offense per 8. Sander, Barrie, “Freedom of expression in the age of online platforms: the promise and pitfalls of human rights-based approach to content moderation.” Fordham International Law Journal, 43.4 (2020), 939-1006. 9. Vladimir Kharitonov v. Russia, App no 10795/14 (ECtHR 23 June 2020), par. 33. Melike v. Turkey, App no 35786/19 (15 June 2021), par. 44. Magyar Helsinki Bizottság v. Hungary [GC] App no 18030/11 (ECtHR 8 November 2016), par. 168. 10. Scott, Jeramie D. “Social media and government surveillance: The case for better privacy protections for our newest public space.” Journal of Business and Technology Law, 12.2 (2017), 151-164. 11. Sandoval-Almazan, Rodrigo, Gil-Garcia, G. Ramon, “Towards cyberactivism 2.0? Understanding the use of social media and other information technologies for political activism and social movements.” Government Information Quarterly, 31.3 (2014), 365-378. 12. UK Information Commissioner’s Office and others, “Joint statement on data scraping and the protection of privacy”, August 2023. https://ico.org.uk/media/ about-the-ico/documents/4026232/joint-statement-data-scraping-202308.pdf 13. Coe, Peter. “The social media paradox: an intersection with freedom of expression and the criminal law.” Information & Communications Technology Law 24.1 (2015), 16-40. https://doi.org/10.6092/issn.1825-1927/18870 19 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) se, or it is made otherwise illegal. This is the case for terrorist content, as defined in the Terrorist-Content Regulation (TERREG)14 through a reference to the Counter-Terrorism Directive15, or of the dissemination of child sexual abuse material online,16 for which the EU is currently acting in order to enshrine a specific set of rules in a regulation.17 Moreover, the concerns about the effects of disinformation have led the EU Member States to take legislative initiatives to tackle the phenomenon, which sometimes resulted in the criminalisation of certain forms of false information with a potential to cause societal harms.18 Additionally, LEAs can be appointed with tasks related to the monitoring of content shared online when non-legislative initiatives provide for such measures in order to safeguard public order, or other relevant interests such as the integrity of election processes, that can be undermined by the spread of harmful content itself.19 The provisions enshrined in the TERREG and DSA that are hereby analysed provide an example of how the EU legislator tried to address this necessity. The TERREG, adopted in 2021, represented an important step in the direction of regulating the spread of a type of content that has been made uniformly illegal across the EU.20 More recently, the Digital Services Act (DSA) was adopted with the aim to harmonise rules imposed on digital services providers concerning the moderation of illegal content.21 While the TERREG only focused on terrorist content, defining it by reference to the definitions of terrorist offenses as delineated in the Counter-Terrorism Directive22, the DSA defined “illegal content” as any content which has been made illegal either by EU law or by domestic laws of Member States.23 In other words, the provisions enshrined in the legal instruments can apply differently across the EU, depending on which type of content has been made illegal in the national jurisdictions. However, it is noteworthy that the DSA shows a willingness of the EU legislator to tackle the worrying phenomenon of online disinformation. Various recitals of the DSA highlight that the instrument was conceived to address societal risks from dissemination campaigns.24 Moreover, very large online platforms and very large search engines25 are obliged to perform a risk assessment of a number of systemic risks for their platforms26, in order to take adequate mitigating measures.27 One of the systemic risks triggering these obligations concerns “any actual or foreseeable negative effects on civic discourse and electoral processes, and public security”.28 While this formulation is very broad, the definition of this risk resonates with the content of former EU policy documents addressing the matter of disinformation. In particular, the communication of the European Commission adopted in 2018 to define a common EU approach to disinformation identifies the hampering of democratic political and policy-making processes as one of the threats posed by this phenomenon.29 The next two subsections provide an in-depth analysis of how the TERREG and the DSA encourage LEAs to monitor social media spaces in order to tackle illegal content, but also with a broader view to prevent and control crimes. 14. Regulation (EU) 2021/784 of the European Parliament and of the Council of 29 April 2021 on addressing the dissemination of terrorist content online, [2021] OJ C 110 (TERREG). 15. Directive (EU) 2017/541 of the European Parliament and of the Council of 15 March 2017 on combating terrorism and replacing Council Framework Decision 2002/475/JHA and amending Council Decision 2005/671/JHA, [2017] OJ C 177 (Counter-Terrorism Directive). 16. Directive 2011/93/EU of the European Parliament and ofthe Council of 13 December 2011 on combating the sexual abuse and sexual exploitation of children and child pornography, and replacing Council Framework Decision 2004/68/JHA, [2011] L 194/1. 17. Proposal for a Regulation of the European Parliament and of the Council laying down rules to prevent and combat child sexual abuse, [2022] COM(22)209. 18. Ó Fathaigh, Ronan, Natali Helberger, Naomi Appelman. “The perils of legally defining disinformation.” Internet policy review 10.4 (2021), 2022-40. 19. van Hoboken, Joris, Ronan Ó. Fathaigh. “Regulating Disinformation in Europe: Implications for Speech and Privacy.” UC Irvine Journal of International, Transnational and Comparative Law 6 (2021), 9-36. 20. TERREG (n. 13). 21. Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) [2022] OJ L 277 (DSA). 22. TERREG (n. 13), art. 2(7). 23. DSA (n. 18), art. 3(h). 24. DSA (n. 18), recitals 2-9-69-83-84-88-95-104-106-108. 25. DSA (n. 18), art. 33. 26. DSA (n. 18), art. 34. 27. DSA (n. 18), art. 35. 28. DSA (n. 18), art. 34.1(c). 29. Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, Tackling online disinformation: a European Approach [2018] COM/2018/236. https://doi.org/10.6092/issn.1825-1927/18870 20 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) 3.1 The role of LEAs in identifying and removing illegal content Both the TERREG and the DSA are examples of legal instruments that institutionalise the possibility for competent authorities to issue removal orders in their specific scope.30 While the competent authorities empowered by the Member States to issue such orders do not have to be necessarily LEAs, at least the process of national designation pursuant to the TERREG shows how the Member States mostly selected police bodies to carry out this function.31 Moreover, the DSA established rules on the so-called “trusted flaggers”. These entities are designated based on their particular expertise in the area of illegal content, their independency from online platforms’ providers, and their diligence, accuracy and objectivity in carrying out their tasks.32 Such entities have, like any other user of online platforms, the possibility to flag an allegedly illegal content to the providers of online platforms to have it removed, pursuant to the terms and conditions of the service or to a EU or domestic law making that content illegal.33 However, contrary to other individuals entitled to resort to this notice and action mechanism, the trusted flaggers’ notices must be given priority in the processing by online platforms. As confirmed by the recitals of the DSA, the status of trusted flaggers can be recognised, among others entities, also to national LEAs or Europol.34 The concept of “trusted flagger” is not new in the area of content moderation, as it resembles the functioning of the already existing Internet Referral Units (IRUs) in the area of terrorist content. The EU Internet Referral Unit was established within the European Counter-Terrorism Center of the EU Agency for Law Enforcement Cooperation (Europol) in 2015 with the aim to support EU Member States in their counter-terrorism efforts by tackling terrorist content.35 Its establishment was functional to coordinate the activities of the national IRUs across the EU. The mechanism of referrals, which implies a notice made by LEAs based on the terms and conditions of online platforms, is referred to in the recitals of the TERREG, as a means alternative to removal orders that can be used to tackle terrorist content online.36 This tool was criticised by the Fundamental Rights Agency (FRA) in its comments to an early proposal of the TERREG, which aimed to institutionalise it through a dedicated provision. The FRA expressed its concerns with regard to the fact that the proposal did not clearly delineate the circumstances where the referral mechanism was to be preferred to a removal order, with a risk of the former being used because of less stringent requirements – and diminished safeguards – at the national level.37 While the provision was not included in the final text of the TERREG, the mechanism is still in place at both the EU and national level. Moreover, the competences of the EU IRU were recently enlarged beyond the scope of tackling terrorist content, to other types of dangerous information. In general, the EU IRU has shown a certain degree of flexibility in adapting its range of actions to different categories of content depending on national LEAs’ needs.38 The IRUs have been identified as one of the areas where online platforms and public authorities’ cooperation results in a coproduction of security.39 In fact, while LEAs are entitled to use referrals, the final assessment about a flagged content is left to the discretion of online platforms, which decide based on their terms and conditions. This aspect differentiates the IRUs from the trusted flaggers, as the latter are entitled, pursuant to the DSA, to flag content that is either against the terms and conditions of online platforms or against EU or Member States’ law.40 However, the characterisation of trusted flaggers significantly resembles that of the IRUs, and renewed the question of opportunity of allowing LEAs to act on this ground, instead of making use of the 30. TERREG (n. 13), art. 3, DSA (n 18), art. 9. 31. European Commission, List of national competent authority (authorities) and contact points, https://home-affairs.ec.europa.eu/policies/internalsecurity/counter-terrorism-and-radicalisation/prevention-radicalisation/terrorist-content-online/list-national-competent-authority-authorities-andcontact-points_en 32. DSA (n. 18), art. 22. 33. DSA (n. 18), art. 16. 34. DSA (n. 18), recital 61. 35. Council of the European Union, “Justice and Home Affairs Council, 12-13 March 2015”, 2015, https://www.consilium.europa.eu/en/meetings/jha/2015/03/ 12-13/. 36. TERREG (n. 13), recital 40. 37. Opinion of the European Union Agency for Fundamental Rights, Proposal for a Regulation on preventing the dissemination of terrorist content online and its fundamental rights implications, 2019. 38. Kilpatrick, Jane, Jones, Chris, “Empowering the police, removing protections: the new Europol Regulation”, Statewatch, 2022. https://www.statewatch. org/media/3615/empowering-the-police-removing-protections-new-europol-regulation.pdf 39. Bellanova, Rocco, De Goede, Marieke, “Co‐Producing Security: Platform Content Moderation and European Security Integration.” JCMS: journal of common market studies 60.5 (2022), 1316-1334. 40. DSA (n. 18), art. 22. https://doi.org/10.6092/issn.1825-1927/18870 21 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) removal order mechanisms. Due to transparency and rule of law concerns, the possibility enshrined in the DSA to designate LEAs and Europol as trusted flaggers has been criticised.41 Due to the importance of the dissemination of content on the Internet to exercise freedom of expression, initiatives trying to tackle illegal content are subject to a close scrutiny in the academic community. As noted in a report of 2023 about legislative initiatives taken across the world to combat disinformation, such initiatives can have a chilling effect on freedom of expression, due to the vagueness of the definitions of illegal content and the sometimes disproportionate sanctions associated with its spread.42 Similar concerns were expressed in occasion of the adoption of the TERREG, due to the divergence of interpretations across the EU about what amounts to terrorist content.43 The TERREG itself demonstrates the awareness of the EU legislator about the risk for content moderation practices to become an instrument to silence minorities or dissenting opinions in the political realm. In fact, one of its provisions specifies that material which represents an expression of polemic or controversial views in the course of public debate, shall not be considered to be terrorist content.44 In this regard, it has also been noted how important it is to distinguish between terrorist content and content raising awareness about terrorism, which should not be banned. In general, a common difficulty in content moderation activities is that certain types of content, such as terrorist and extremist content, are more complicated to uniformly define compared to other, such as child sexual abuse material.45 In reason of its recent adoption, and the broad scope of the DSA with regard to what constitutes illegal content, its impact on social media monitoring practices by LEAs in the quality of trusted flaggers remains to be seen. However, the functioning of IRUs already provides an example of how LEAs are involved in patrolling social media. Given the amount of information available online, IRUs resort to OSINT capabilities to detect illegal content online.46 Besides the huge number of pieces of content and online platforms analysed47, the 2021 Consolidated Annual Report of Europol states that, once identified, the content is manually assessed by a Unit’s expert to verify whether it has wrongfully been categorised as terrorist.48 This activity adds up to that of the national IRUs, which provide Europol with the open source information they collected.49 In turn, all the information obtained thanks to this activity is then included in a centralised database. In fact, one of the main interests of the EU IRU is not only to detect illegal content, but also to retrieve it, as it falls within the competence of Europol, providing a legal basis for such retention.50 Moreover, while the EU IRU was originally create to tackle terrorist content, its reach was broadened in relation to its cooperation with national IRUs. In 2021, its tasks included tackling content related to right-wing violent extremism, online terrorist propaganda and violent jihadist ideology, migrant smuggling during the “Belarus Crisis”. Moreover, it also acted in the context of an Internet Archive platform aimed to strengthen public-private cooperation in content moderation.51 The analysis of the EU IRU’s work shows how the connection between social media activities and offline crimes can justify a policy-making approach enlarging the grounds for content monitoring. The potential for LEAs’ monitoring activities to go beyond the mere detecting of illegal content is discussed in the next subsection. 41. Komaitis, Konstantinos, Rodriguez, Katitza, Schmon Christoph, “Enforcement Overreach Could Turn Out To Be A Real Problem in the EU’s Digital Services Act”, Electronic Frontiers Foundation, 2022. https://www.eff.org/deeplinks/2022/02/enforcement-overreach-could-turn-out-be-real-problemeus-digital-services-act 42. Lim, Gabrielle, Bradshaw, Samantha, “Chilling Legislation: Tracking the Impact of”Fake News” Laws on Press Freedom Internationally.” Center for International Media Assistance, 2023. https://www.cima.ned.org/wp-content/uploads/2023/06/CIMA-Chilling-Legislation_web_150ppi.pdf 43. Berthélémy, Chloé, “EU Terrorist Content Online Regulation Could Curtail Freedom of Expression across Europe.” EDRi, 2021. https://edri.org/ourwork/eu-terrorist-content-online-regulation-could-curtail-freedom-of-expression-across-europe/ 44. TERREG (n. 13), art. 1.3. 45. Chang, Brian. “From Internet Referral Units to International Agreements; Censorship of the Internet by the UK and EU.” Columbia Human Rights Law Review 49 (2017), 114-212. 46. Chang (n. 43), p. 135. 47. Europol Consolidated Annual Activity Report, 2021. https://www.europol.europa.eu/cms/sites/default/files/documents/Consolidated%20Annual% 20Activity%20Report%202021.PDF. 48. Europol EU Internet Referral Unit Transparency Report, 2021. https://www.europol.europa.eu/cms/sites/default/files/documents/EU_IRU_Transparency_ Report_2021.pdf-. 49. Chang (n. 43), p. 135. 50. Bellanova, De Goede (n. 37). 51. Kilpatrick, Jane, Jones, Chris (n. 39). https://doi.org/10.6092/issn.1825-1927/18870 22 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) 3.2 The role of LEAs beyond the identification and removal of illegal content While not all of the content that is made illegal under EU law leads to its dissemination being a criminal offense under EU or Member States’ law, the connection between the spread of illegal content online and the risk of commission of certain forms of crimes was highlighted in the EU policy debate on content moderation. Prior to the adoption of the TERREG and the DSA, the Commission recognised, in a more general communication on the moderation of illegal content, that LEAs have a duty to prosecute crimes, and online platforms are responsible for preventing that their services are misused in order to commit them.52 Therefore, the intersection between the dissemination of certain content and the area of criminal justice can result in a potential overlapping between LEAs’ tasks and the monitoring of social media beyond the tackling of illegal content. In particular, LEAs can be involved in monitoring content published on social networks as a response to the strict connection between the dissemination of said content and the commission of criminal offenses in the offline world. In other words, social media monitoring activities can serve the purpose of act in a crime prevention perspective, by analysing social media content to detect and anticipate potential criminal offense. The connection between the dissemination of certain content online and the possibility of online activities resulting in a broad range of threats is confirmed by the DSA In its recitals, the regulation specifies that the concept of illegal content should be defined in relation to existing rules in the offline environment. Thus, it should encompass any information that not only is made illegal by a specific law, but is also connected to illegal activities in general. As an example of online content that can signal criminal offenses, the DSA mentions the non-consensual sharing of private images, online staking, the sale of non-compliant or counterfeit products, copyrights infringements.53 The monitoring of social media provides LEAs with an affordable source of information to observe individual behaviours and social relations. Due to the sociological premise that the observation of individuals in social environments can help gain knowledge about future crimes, monitoring activities deriving from the necessity to moderate illegal content can result in further actions by LEAs.54 The analysis and social media data can be use not only to take down harmful content, but to direct LEAs’ efforts in their crime prevention activities. Therefore, data extracted for content moderation purposes can be also used in other, and broader, contexts. An example is the possibility of using social media data inferred by the content shared online to make behavioural predictions about individuals. AI-driven predictive models are versatile, as they can be used with any amount of publicly available personal data retreived from social media in order to make a broad range of predictions. In combination with certain assumptions on the way human beliefs and choices are formed, such predictions can result in conclusions on motivations and inclinations that are not always accurate. In the case of predictions of LEAs based on publicly available data, there is risk of social control and stigmatisation deriving from these practices, which can in turn harm the autonomy of individuals in expressing themselves online.55 In light of the possibility for social media data to be used for purposes that go beyond what is traditionally understood as content moderation, it is all the more important to carefully assess the impact of data scraping techniques on the protection of personal data that are publicly available. Such an evaluation should be carried out by considering the technological features of data scraping practices, and in particular, in the context of social media, the premises and pitfalls of NLP algorithms in detecting illegal content, or suspicious activities in general. The next section will explore the fundamental rights’ concerns arising from these technological means. 4 The application of social media scraping technologies in the law enforcement domain. Natural Language Processing techniques to classify online content As already observed, NLP technologies are a precious tool in order to detect and categorise certain types of online content. However, it has been noted that such technologies, while widely used, are not yet at a developing stage allowing their use without 52. Communication from the Commission to the European Parliament, the council, the European Economic and Social committee and the Committee of the Regions, Tackling Illegal Content Online Towards an enhanced responsibility of online platforms [2017] COM/2017/0555. 53. DSA (n. 18), recital 12. 54. Susser, Daniel, “Predictive policing and the ethics of preemption.” The ethics of policing: New perspectives on law enforcement (2021), 268-292. 55. Ploug, Thomas. “The right not to be subjected to AI profiling based on publicly available data—privacy and the exceptionalism of ai profiling.” Philosophy & Technology 36.1 (2023), 14. https://doi.org/10.6092/issn.1825-1927/18870 23 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) any human supervision.56 First, NLP algorithms are not able to take into account contextual clues and circumstances that, for certain types of content, is essential in order to assess its nature.57 This is, for example, the case for terrorist content, for which the importance of context in the identification is made explicit in the recitals of the TERREG58, and disinformation, as falsehood of a piece of information is an highly contextual element.59 This point is connected to the inherent difficulty in categorising certain types of content. In some cases, such as with regard to violent extremist content, defining what falls within this category can be challenging due to the lack of an uniformly accepted definition, as opposed to the case of other illegal content, such as child sexual abuse material, which is more straightforwardly identifiable.60 Secondly, the margin of error of such technologies may result in wrongful labelling of content, due to the difficulties of identifying harmful content online by matching the content with a pre-defined set of words. In other words, false positive and false negatives are a risk to be considered.61 Thirdly, as a recent report of the Fundamental Rights Agency outlined through an empirical research how algorithms used to identify hate speech are susceptible to be biased, and their use can lead to discriminatory effects for persons, based, for example, on their religious or political affiliation.62 Notwithstanding these limitations, the range of possible applications of NLP technologies in the context of LEAs’ activities seems wide. A brief literature review of research projects investigating the potential of NLP for the criminal justice area reveals that this technology is proposed in the context of social media monitoring as a potential solution to investigate illicit COVID-19 product sales63, human trafficking64, religious and political extremism.65 Moreover, NLP’s potential uses for LEAs have also been identified in situations where the spread of illegal content itself represents the source of societal harm, as opposed to scenarios where it is connected to ulterior crimes perpetrated offline. This is the case for studies investigating the use of NLP and its limitations to tackle hate speech66 and disinformation.67 The ongoing discussion about the use of data scraping technologies on social media by using NLP requires a legal analysis of the privacy and data protection implications of using such techniques to monitor online environments. The aim of this analysis is to draw more general conclusions about the possible impact of such technologies to fundamental rights and freedoms in the area of law enforcement. The next section explores the legal status of publicly available data collected from social media and their protection at the EU level. 5 The legal status of publicly available data collected from social media The present section discusses the status of publicly available data from a privacy and data protection standpoint, then applying the outcomes of the analysis to social media scraping practices performed by LEAs. Firstly, it analyses what protection is granted to publicly available data pursuant to the EU data protection framework. Secondly, it analyses privacy and data protection guarantees afforded in relevant judgements of the ECtHR. Finally, it focuses on the CJEU case law, by extracting some useful principles to guide the lawful processing of publicly available data from the EU Court’s decisions. 56. Krotov, Vlad, Leiser Silva. “Legality and ethics of web scraping.” Twenty-fourth Americas Conference on Information Systems, New Orleans (2018). 57. Gorwa, Robert, Bonns, Reuben, Katzenback, Christian “Algorithmic content moderation: Technical and political challenges in the automation of platform governance.” Big Data & Society 7.1 (2020), 2053951719897945. 58. TERREG (n. 13), recital 11. 59. Pielemeier, Jason. “Disentangling disinformation: What makes regulating disinformation so difficult?.” Utah Law Review 4 (2020), 917-940. 60. Chang (n. 43), p. 137. 61. Gorwa, Binns, Katzenback (n. 53). 62. Fundamental Rights Agency, Bias in algorithms. Artificial Intelligence and discrimination, 2022. 63. Mackey, Tim Ken, et al. “Big data, natural language processing, and deep learning to detect and characterize illicit COVID-19 product sales: infoveillance study on Twitter and Instagram.” JMIR Public Health and Surveillance 6.3 (2020), 360-376. 64. Granizo, Sergio L., et al. “Detection of possible illicit messages using natural language processing and computer vision on twitter and linked websites.” IEEE Access 8 (2020), 44534-44546. 65. Torregrosa, Javier, et al. “A survey on extremism analysis using natural language processing: definitions, literature review, trends and challenges.” Journal of Ambient Intelligence and Humanized Computing 14.8 (2023), 9869-9905. 66. Poletto, Fabio, et al. “Resources and benchmark corpora for hate speech detection: a systematic review.” Language Resources and Evaluation 55 (2021), 477-523. 67. de Oliveira, Nicollas R., et al. “Identifying fake news on social networks based on natural language processing: trends and challenges.” Information 12.38 (2021). https://doi.org/10.6092/issn.1825-1927/18870 24 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) 5.1 The EU legal framework From a data protection point of view, the activities involving the scraping and classification of data from content shared on social media amount to processing of personal data when the detection of illegal content implies that such content can be associated to the civil identity of a user. In other words, such technologies make a certain data subject identifiable, thus falling within the scope of the EU data protection framework.68 The processing attains in this case to personal data that are publicly available online. Personal data is considered as publicly available when its access is not limited to a specific group of persons, with different nuances about what should amount to such a free access. However, regardless of the divergencies over the definitory approach to publicly available data, the EU legal system extends data protection guarantees to publicly available data, even in the law enforcement context, as evident from the Europol legal framework.69 The 2016 Europol Regulation mentions data deriving from publicly available sources among those that the EU agency can process in the course of its activities.70 Nontheless, the restrictions enshrined in the regulation on the possibility to share publicly available data with private parties71, as well as the rights of erasure, rectification or access restriction to which data subjects are entitled with regard to this type of data72, confirms that the public availability does not entail a release from any obligation to respect data protection principles. Personal data that can be retrieved from content shared online represents a peculiar type of publicly available content, for two orders of reasons. First of all, the data mining on social media can allow to extract implicit and potentially useful information that regards sensitive data of the users, such as their political affiliation and religious beliefs. This aspect can be especially problematic in the context of profiling to be used in predictive policing activities.73 Secondly, while other publicly available data is made available online by third parties, the content that is shared online is made available by the publishers themselves, triggering the application of a specific regulatory framework when LEAs process, in particular, sensitive data of the data subjects. Under the Law Enforcement Directive (LED), governing the processing of personal data by competent authorities in relation to the prevention, detection or prosecution of crimes74, special categories of personal data, including data pertaining to political or philosophical beliefs, should only be processed when strictly necessary and based on specific legal grounds, subject to appropriate safeguards for fundamental rights.75 One of the grounds justifying the processing is the fact that sensitive data have been manifestly made public by the data subject.76 From the letter of the law it can be drawn that, while the processing of publicly available sensitive data by LEAs is possible in this case, the processing still needs to comply with the substantial requirement of strict necessity, and the procedural requirements of providing appropriate safeguards to data subjects. Both the Working Party 29 (WP 29) and the European Data Protection Board (EDPB) contributed to delineating the meaning of “manifestly public”. While a clear intention of the data subject to make personal data available should be present to consider the processing lawful, the WP 29 provided some specifications with regard to publicly available data in social media. In this case, most of the individuals publishing content online do not expect their data to be accessible by police authorities. A combination of elements should play a role in the assessment about this expectation of privacy, such as the accessibility of the source where 68. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L 119, art. 4(1). 69. Gottschalk, Thilo. “The Data-Laundromat? Public-Private-Partnerships and Publicly Available Data in the Area of Law Enforcement.” European Data Protection Law Review 6 (2020), 21-40. 70. Regulation (EU) 2016/794 of the European Parliament and of the Council of 11 May 2016 on the European Union Agency for Law Enforcement Cooperation (Europol) and replacing and repealing Council Decisions 2009/371/JHA, 2009/934/JHA, 2009/935/JHA, 2009/936/JHA and 2009/968/JHA [2016] OJ L 135 (2016 Europol Regulation), art. 17.2. 71. 2016 Europol Regulation (n. 66), art. 37. 72. Regulation (EU) 2022/991 of the European Parliament and of the Council of 8 June 2022 amending Regulation (EU) 2016/794, as regards Europol’s cooperation with private parties, the processing of personal data by Europol in support of criminal investigations, and Europol’s role in research and innovation [2022] OJ L 169. 73. Mitrou, Lilian, et al., “Social media profiling: A Panopticon or Omniopticon tool?.” Proceedings of the 6th Conference of the Surveillance Studies Network, 2014. 74. Directive (EU) 2016/680 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, and on the free movement of such data, and repealing Council Framework Decision 2008/977/JHA [2016] OJ L 119 (LED). 75. LED (n. 70), art. 10. 76. LED (n. 70), art. 10(c). https://doi.org/10.6092/issn.1825-1927/18870 25 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) C. Jasserand, Article 10. Processing of Special Categories of Personal Data, in LED Commentary, OUP (forthcoming Fall 2023). M. A. Khder, Web scraping or Web crawling. State of Art, Techniques, Approaches and Application, in “International Journal of Advances in Soft Computing & Its Applications”, 13.3 (2021). J. Kilpatrick, C. Jones, Empowering the police, removing protections: the new Europol Regulation,Statewatch, 2022. K. Komaitis, et al., Enforcement Overreach Could Turn Out To Be A Real Problem in the EU’s Digital Services Act, Electronic Frontiers Foundation, 2022. V. Krotov, S. Leiser, Legality and ethics of web scraping,Twenty-fourth Americas Conference on Information Systems, New Orleans (2018). G. Lim, S. Bradshaw, Chilling Legislation: Tracking the Impact of “Fake News” Laws on Press Freedom Internationally, Center for International Media Assistance, 2023. T. K. Mackey, et al., Big data, natural language processing, and deep learning to detect and characterize illicit COVID-19 product sales: infoveillance study on Twitter and Instagram, in “JMIR Public Health and Surveillance”, 6.3 (2020). L. Mitrou, et al., Social media profiling: A Panopticon or Omniopticon tool? Proceedings of the 6th Conference of the Surveillance Studies Network, 2014. R. Ó Fathaigh, et al., The perils of legally defining disinformation, in “Internet policy review”, 10.4 (2021): 2022-40. J. Pielemeier, Disentangling disinformation: What makes regulating disinformation so difficult? In “Utah Law Review”, 4 (2020). T. Ploug, The right not to be subjected to AI profiling based on publicly available data—privacy and the exceptionalism of ai profiling, in “Philosophy & Technology”, 36.1 (2023). F. Poletto, et al., Resources and benchmark corpora for hate speech detection: a systematic review, in “Language Resources and Evaluation”55 (2021). S. Quattrocolo, Hacking by Law-Enforcement: Investigating with the Help of Computational Models and AI Methods, in Artificial Intelligence, Computational Modelling and Criminal Proceedings: A Framework for A European Legal Discussion, Springer, 2020. P. M. Sabella, Il fenomeno del cybercrime nello spazio giuridico contemporaneo. Prevenzione e repressione degli illeciti penali connessi all’utilizzo di internet per fini di terrorismo, tra esigenze di sicurezza e rispetto dei diritti fondamentali, in “Informatica e diritto”, 26.1-2 (2017). B. Sander, Freedom of expression in the age of online platforms: the promise and pitfalls of human rights-based approach to content moderation, in “Fordham International Law Journal”, 43.4 (2020). R. Sandoval-Almazan, G. R. Gil-Garcia, Towards cyberactivism 2.0? Understanding the use of social media and other information technologies for political activism and social movements, in “Government Information Quarterly”, 31.3 (2014). J. D. Scott, Social media and government surveillance: The case for better privacy protections for our newest public space, in “Journal of Business and Technology Law”, 12.2 (2017). A. Shere, Reading the Investigators their Rights: A review of literature on the General Data Protection Regulation and open-source intelligence gathering and analysis, in “The New Collection”, 3, Oxford, 2020. R. D. Steele, Open source intelligence,Handbook of intelligence studies, 42.5, London, Routledge, 2007. D. Susser, Predictive policing and the ethics of pre-emption, in B. Jones (Ed.) The ethics of policing: New perspectives on law enforcement, NYU Press,2021. J. Torregrosa, et al., A survey on extremism analysis using natural language processing: definitions, literature review, trends and challenges, in “Journal of Ambient Intelligence and Humanized Computing”, 14.8 (2023). D. Trottier, Open source intelligence, social media and law enforcement: Visions, constraints and critiques, in “European Journal of Cultural Studies”, 18.4-5 (2015). https://doi.org/10.6092/issn.1825-1927/18870 32 Moderation of illegal content and social media scraping. i-lex. Vol. 16 n. 2 (2023) M. Tzanou, and K. Spyridoula K., Privacy international and quadrature du Net: One step forward two steps back in the data retention saga? In “European Public Law”, 28.1 (2022). S. Vanden Broucke, B. Baesens, From web scraping to web crawling, in Practical Web Scraping for Data Science. Best practices and examples with Python, Apress, Berkeley, 2018. J. van Hoboken, and R. Ó. Fathaigh, Regulating Disinformation in Europe: Implications for Speech and Privacy, in “UC Irvine Journal of International, Transnational and Comparative Law”, 6 (2021). https://doi.org/10.6092/issn.1825-1927/18870 33