ARTIFICIAL INTELLIGENCE FOR CHILD ONLINE PROTECTION IN INDIA: NAVIGATING THE POLICY, PRIVACY, AND TECHNOLOGICAL TIGHTROPE

305 CHAPTER-27 ARTIFICIAL INTELLIGENCE FOR CHILD ONLINE PROTECTION IN INDIA: NAVIGATING THE POLICY, PRIVACY, AND TECHNOLOGICAL TIGHTROPE Dr. Amitesh Anand Associate Professor, College of Law, IIMT University Ganganagar Meerut Dr. Ashutesh Anand Assistant Professor, College of Law, IIMT University Ganganagar Meerut ABSTRACT Over the past few decades in India the internet tools and their easy access has expanded rapidly and regardless of age significantly impacted every person. Therefore the children are also not exception of it. In addition these tools have not only transformed children’s digital interaction but also have increased their exposure to a number of online dangers such as cyber bullying, online grooming, and Child Sexual Abuse Material (CSAM). Likewise, technologies like Artificial Intelligence (AI) have emerged have developed into essential tools for identifying and preventing these kinds of crimes. On one hand it keeps them safe, on the other hand in India integrating AI for child online protection raises significant legal, ethical, and privacy concerns such as present status and scope of AI-enabled tools used for online child protection in India, how far they are adequate and effective, the principal ethical and privacy-related challenges associated with the deployment of AI and how India can develop a balanced regulatory framework for child protection, their privacy rights, and other issues within the Indian legal and social framework. This paper examines the rules, regulations, and scope surrounding the use of AI for child protection, assesses privacy and ethical issues and makes suggestions for a well-balanced regulatory framework that upholds both i.e. the individual rights and the child safety. INTRODUCTION Over the past few decades in India the internet tools and their easy access has expanded rapidly and significantly impacted every individual without any age difference. In accordance with a UNICEF (2021) report, in India under the age of 18 approximately over 400 million children are using internet. Somewhere, this electronic revolution is making the children beneficiary and making somewhere the vulnerable targets of digital evolvement. The nature and scale of online threats faced by children have grown in both scale and complexity, particularly Child Sexual Abuse Material (hereinafter CSAM), cyber-bullying, and online grooming. Artificial Intelligence (hereinafter AI) technologies, such as machine learning, image recognition and analysis, and predictive analytics, have shown considerable potential in combating these challenges worldwide. However, integrating such technologies in the Indian context raises significant issues involving privacy, consent, data protection, and the risk of misuse. Additionally, 306 India’s legal framework on AI ethics and children’s data rights remains fragmented, highlighting the need for thorough analysis. So the study efforts • To find out the present status and scope of AI-enabled tools used for online child protection in India? • How adequate and effective are the current policies and legal provisions in regulating AI-driven interventions aimed at ensuring child online safety? • What are the principal ethical and privacy-related challenges associated with the deployment of AI in detecting CSAM and other online risks faced by children and also • How can India develop a regulatory framework that balances child protection, privacy rights, and technological innovation? To achieve this, the study will examine legal frameworks, policies, ethical considerations, judicial decisions, AI applications in India, and relevant ethical and privacy guidelines. In this way, the study seeks to provide a comprehensive understanding of the intersection of Artificial Intelligence, child protection, and privacy rights in India. It further aims to highlight significant gaps in the existing legal and policy framework regulating AI applications for child online protection. Furthermore, the work offers policy recommendations to make sure that AI tools strengthen children's digital safety and safeguard them without endangering their constitutionally provided rights. THE AI-BASED TOOLS FOR CHILD ONLINE PROTECTION IN INDIA AND THEIR SCOPE AI-based tools for child online protection in India are evolving through a combination of international collaborations, government portals, AI-driven moderation by tech companies, and NGO initiatives. India still requires a focused, regional, AI-based the child protection system that adjusts to its diverse digital environment. A number of AI-assisted methods and platforms are being used in India to combat online child sex exploitation and associated concerns. Some of them areThe CyberTipline India, integrated with the National Cyber Crime Reporting Portal (NCRP) - This system in operation is supported by INTERPOL and the National Center for Missing & Exploited Children (NCMEC. This AI-assisted systems acts and analyze upon the received complaints related with child sexual abuse material (CSAM). Through collaborations with INTERPOL, Indian law enforcement agencies (LEAs) utilize Microsoft’s PhotoDNA technology, which uses AI-powered hash-matching methods to identify known child sexual abuse material (CSAM) images and videos circulating online. Likewise, Google’s AI Content Safety API offers AI-based tools to online service providers in India for detecting and reporting harmful content. 307 Non-governmental organizations such as Anyay Rahit Zindagi (ARZ) and the Childline India Foundation are also similar agencies which have started AIbased data analysis tools to monitor suspicious online activity and report potential cases of CSAM. Additionally, India’s Computer Emergency Response Team (CERT-IN) enabled AI-driven threat detection systems to identify and evaluate cyber threats, including those which targets the children, and issues timely alerts to concerned law enforcement bodies. Besides major social media platforms such as Meta (operating Facebook and Instagram), X (formerly Twitter), and YouTube India also have integrated AIdriven content moderation technologies that scan images, videos, and text to detect CSAM and other child protection violations, flagging them for swift actions. Additionally, AI-focused initiatives at Indian research institutes and start-ups such as the AI Safety Research Lab at the Indian Institute of Science (IISC) are also some of the emerged actively predictive AI models which have been designed to recognize high-risk online behavior patterns and potential grooming activities. Despite these efforts, most AI-based child online protection tools in India are adaptations of international technologies or remain confined to limited pilot projects. At present, India is still in the nascent stage in the process of integrating AIdriven child protection tools within its domestic digital infrastructure and policy frameworks. The country lacks a fully indigenous, large-scale AI platform dedicated solely to the detection of CSAM and comprehensive child online safety mechanisms. However the key applications of AI in this domain include AI-driven tools and systems like CSAM Detection Algorithms, 1 Chat Monitoring Systems, 2 Image and Video Analysis, 3 Behavior Prediction Models, 4 the international technology companies like Meta, Google, and Microsoft etc. No doubt today Artificial Intelligence (AI) technologies have become vital instruments in child online protection, primarily through algorithms designed to detect explicit content, monitor suspicious online activities, and identify potential cyber threats targeting children. 1 AI-powered technologies such as Microsoft’s PhotoDNA and Google’s Content Safety API are extensively utilized to identify and remove Child Sexual Abuse Material (CSAM) from digital platforms by using hash-matching and content analysis methods. 2 These tools are capable of analyzing interactions to identify patterns suggestive of grooming, predatory conduct, or other harmful communications targeting children. 3 These can scrutinize online conversations to identify patterns indicative of grooming, predatory conduct, or harmful interactions aimed at children. 4 Used to monitor and assess online activity patterns, enabling the early identification of at-risk children and potentially harmful digital environments. 308 POLICY AND LEGAL FRAMEWORK IN INDIA India’s approach to child online protection is shaped by a patchwork of statutory laws, regulatory guidelines, and policy initiatives, each addressing different aspects of online safety and child rights protection. Among the most significant legal instruments are- ➢ The Protection of Children from Sexual Offences (POCSO) Act, 2012This Act explicitly criminalizes the possession, distribution, and creation of Child Sexual Abuse Material (CSAM) and is a landmark legislation which addresses rapidly expanding digital sexual offences against minors in India’s. According to this law the visual depiction of any sexually explicit acts in which child are involving, includes photographs, videos, and digital images, considered under CSAM. Besides the Information Technology (IT) Act, 2000 (particularly after its 2008 amendment), also plays a key role in regulating harmful online content and serves as a critical safeguard against online child exploitation. Its provisions explicitly criminalize publication, transmission, or viewing of material that depicts children engaged in sexually explicit acts, including in electronic formats i.e. the content hosted on websites, social media platforms, and other digital services within India. 5 ➢ The Juvenile Justice (Care and Protection of Children) Act, 2015This statutory law is also another legislation which provides additional safeguards for children identified as being in need of care and protection. This legislation underscores the duty of law enforcement agencies and child welfare committees to address the cases related to cyber exploitation. However, it does not specifically address or regulate the use of AI-based child protection systems. 6 Nonetheless, its provisions are often invoked in cases involving minors affected by cyber threats. ➢ The National Cyber Crime Reporting Portal (NCRP) - This is an Indian government launched online platform for reporting cyber offences including CSAM-related incidents. While the NCRP primarily depends on traditional reporting mechanisms, it is progressively incorporating AI-powered tools to enhance the prioritization and classification of complaints, particularly in relation of threats against child safety. 7 ➢ The Draft Digital Personal Data Protection Bill, 2023India has also introduced The Draft Digital Personal Data Protection Bill, 2023 to control the growing complexities of digital governance. This bill introduces strong provisions for handling personal data, including data related to children. This bill requires explicit consent for the collection and processing of children's personal data and mandates higher standards for security. However, the bill is 5 Section 67B of the Information Technology (IT) Act, 2000 ( Ministry of Electronics and Information Technology, 2008). 6 Ministry of Women and Child Development, Model Guidelines under Section 39 of the Protection of Children from Sexual Offences (POCSO) Act, 2012, Government of India (2015). 7 Ministry of Home Affairs, Annual Report 2020–21, Government of India (2021). 309 yet to be enacted and does not contain clear guidelines on AI ethics or accountability. 8 Despite recent developments, India still lacks a comprehensive and dedicated regulatory framework to govern the ethical use, operational limits, and accountability of AI technologies in the context of child online protection. The absence of clear guidelines on algorithmic transparency, data governance, and AIbased surveillance oversight has raised significant concerns among child rights activists, privacy specialists, and legal scholars. 9 The situation highlights the urgent need for India to implement a nuanced, rights-oriented regulatory approach that addresses the ethical challenges of AI-based child protection systems. ETHICAL, INTERNATIONAL, AND TECHNOLOGICAL CHALLENGES IN AI-DRIVEN CHILD ONLINE PROTECTION AI-powered detection systems, though instrumental in combating online child exploitation, raise significant ethical challenges that warrant careful scrutiny. One primary concern is the potential infringement of children’s right to privacy, as guaranteed under Article 21 of the Indian Constitution. The Apex Court in a case held that surveillanceoriented AI tools frequently monitor personal digital interactions, raising apprehensions degree to which such monitoring might infringe on privacy rights in the name of protection. 10 Striking an appropriate balance between children safety and preserving their informational autonomy is a delicate one, especially given the far-reaching capabilities of AI surveillance technologies. Another serious issue involves data consent and profiling. AI systems frequently often gather and analyze children’s personal data, sometimes without obtaining clear, informed consent from guardians. This raises the risk of unauthorized profiling and potential misuse of sensitive information. The lack of a strong legal framework in India to govern consent for data processing involving minors further exacerbates this issue according to UNICEF. 11 The ethical obligation to obtain consents transparently and accessible in a manner accessible to guardians or custodians, remains a critical ethical challenge in AI-enabled child protection mechanisms. One more complicating issue involves the algorithmic bias. AI models trained on incomplete or skewed datasets can produce inaccurate outcomes, disproportionately flagging marginalized or vulnerable communities. These biases can reinforce existing social inequalities and potentially misdirect resources away from legitimate cases, thereby compromising the effectiveness 8 Ministry of Electronics and Information Technology, Digital Personal Data Protection Bill, Government of India (2023). 9 UNICEF, Policy Guidance on AI for Children, United Nations Children’s Fund (2021). 10 Justice K.S. Puttaswamy (Retd.) v. Union of India, (2017) 10 SCC 1. 11 UNICEF, Policy Guidance on AI for Children, United Nations Children’s Fund (2021). 310 and credibility of AI-driven intervention. 12 Moreover to mitigate discriminatory outcomes in AI applications for child safety, it is essential to ensure that datasets are representative and that fairness is built into system design and implementation. Additionally, challenges related to transparency and accountability remain prevalent, largely due to the opaque or ‘black box’ nature of AI systems. It is often hard to assign liability in instances of false identification or erroneous surveillance. 13 In the sensitive domain of child protection, where rights are paramount, the inability to explain or contest AI-driven decisions could result in grave injustices In the sensitive domain of child protection, where rights are paramount, the lack of explainability and avenues to challenge AI-generated decisions can lead to serious injustices At the international front, India can learn valuable lessons from prevailing best practices of international frameworks. For example The European Union’s General Data Protection Regulation (GDPR) provides comprehensive protections for children’s data, emphasizing transparency, parental consent, and the minimization of data collection. 14 Similarly, The United Kingdom’s Age-Appropriate Design Code (2021) that online services prioritize children’s privacy, and incorporate features that reflects the best interests of the child. 15 In Australia, the Office of the eSafety Commissioner employs employs AI and machine learning tools to detect CSAM while operating under a strict legal and ethical framework. 16 Furthermore, international initiatives such as the WeProtect Global Alliance promote for AI-enabled child protection strategies that respect human rights, privacy, and due process. 17 From a technological standpoint, the effectiveness of AI in child protection is contingent upon several key elements including availability of large, diverse, and ethically collected datasets is necessary for training reliable AI models. Equally vital to ensure cohesive functioning of AI tools is interoperability between platforms which enables integration of AI systems across social media, messaging apps, and cloud storage services. 18 However, Indian law enforcement 12 West, S.M., Whittaker, M., & Crawford, K., Discriminating Systems: Gender, Race and Power in AI, AI Now Institute (2019). 13 Calo, R., Artificial Intelligence Policy: A Primer and Roadmap, 51 U.C. Davis L. Rev. 399 (2017). 14 European Union, General Data Protection Regulation (2016). 15 UK Information Commissioner’s Office, Age-Appropriate Design Code (2021). 16 eSafety Commissioner, Annual Report 2020–21, Government of Australia (2021). 17 UNICEF, Policy Guidance on AI for Children, United Nations Children’s Fund (2021). 18 Ibid. 311 agencies continue to face notable resource constraints, including restricted access to advanced AI technologies, infrastructure, and skilled personnel capable of deploying and managing such syste ms effectively. Moreover, localization remains a technical significant challenge, as AI models must adapt to India’s linguistically and culturally diverse digital landscape to function accurately. 19 While global platforms have made substantial progress in deploying AI for child protection, Indian efforts remain at an early stage due to these ongoing constraints and limitations. JUDICIAL INTERVENTIONS AND CASE LAWS ON CHILD ONLINE PROTECTION AND AI IN INDIA While India’s legal framework provides for the protection of children in digital environments, the judiciary has played a pivotal role in interpreting and expanding these safeguards, particularly in relation to privacy, digital rights, and child welfare. A key development was the landmark Supreme Court ruling in Justice K.S. Puttaswamy (Retd.) v. Union of India (2017) 20 which established the constitutional right to privacy under Article 21 and declaring informational privacy as a fundamental aspect of personal liberty. This judgment holds significant implications for AI-based child protection technologies that often rely on intrusive data collection and surveillance practices. The Court emphasized that any encroachment on privacy must meet the tests of proportionality, necessary, and backed by a legitimate state interest, a principle that are equally pertinent to AI-driven measures aimed at safeguarding children online. In Re: Prajwala Letter Dated 18.2.2015, the Supreme Court took suo motu cognizance of the circulation of child sexual abuse videos on social media and online platforms. 21 The Court instructed the Union Government to develop a comprehensive mechanism to combat the circulation of Child Sexual Abuse Material (CSAM) and to work in coordination with technology companies for effective content identification and removal. As a result, major tech platforms, such as Google and Microsoft, implemented AIdriven CSAM detection tools like PhotoDNA within Indian jurisdictions. This development highlights the judiciary’s active role in promoting the use of AI-based content moderation technologies and in advocating for corresponding legislative reforms. Furthermore, in the case KN Govindacharya v. Union of India (2019), the Delhi High Court addressed concerns of user data protection and surveillance practices by social media platforms. 22 While not directly focused on child protection, the Court acknowledged the potential risks associated with the misuse of personal data, especially by AI-driven algorithms functioning without transparent 19 West, S.M., Whittaker, M., & Crawford, K., Discriminating Systems: Gender, Race and Power in AI, AI Now Institute (2019). 20 (2017) 10 SCC 1. 21 (2018) 11 SCC 599. 22 2019 SCC OnLine Del 6664. 312 accountability. The judgment emphasized the necessity for stronger legislative protections and regulatory frameworks to govern AI-enabled data processing, including systems that handle children’s information. Additionally, in a case the Bombay High Court highlighted, emphasized the need of rapid digital forensic analysis in cases of online sexual exploitation of minors 23 and the Court directed the Maharashtra Cyber Cell and law enforcement authorities to adopt advanced technologies for real-time detection and removal of CSAM from digital platforms. Though not explicitly mentioning AI, the order’s insistence on technologically advanced tools aligns with the adoption of AI-based systems for child online protection. Similarly the verdict given by Apex Court in Anuradha Bhasin v. Union of India (2020) have also highlighted ethical dilemmas in AI surveillance and it was reaffirmed that internet restrictions must satisfy the principles of proportionality and necessity, thereby reaffirming constitutional protections against arbitrary digital surveillance.⁵ This precedent is relevant for AI-driven child protection mechanisms, which often involve continuous monitoring of online environments. Collectively, Indian courts have consistently advocated for a rights-based approach to digital governance, emphasizing the balance between safeguarding children and preserving constitutional freedoms. Their decisions emphasize the importance of developing AI systems call for robust, transparent, and accountable AI systems and governed by clear statutory provisions and governed by clear statutory provisions and ethical oversight. These judgments highlight the urgent need for India to establish a dedicated regulatory framework governing AI applications in child protection, ensuring that technological solutions align with legal standards and uphold fundamental rights. CONCLUSION AND RECOMMENDATIONS On above bases that AI has arrived and currently these AI technologies present valuable opportunities for strengthening the child online protection in India. And no doubt the rapid expansion of digital technologies in India has opened up unprecedented opportunities for children but has also simultaneously expose them to serious online threats such as Child Sexual Abuse Material (CSAM), cyberbullying, and online grooming. Artificial Intelligence (AI) offers promising technological interventions to address these threats tracking suspicious online activity, and forecasting potential harm .Yet, the ethical, legal, and privacy challenges they introduce necessitate a thoughtful and balanced approach. It is essential for India to develop AI governance structures that prioritize child rights while utilizing technological innovations for their protection. 1. Formulate AI-specific regulations addressing ethical deployment, accountability, and rights protection for children. 2. Enact strengthen the Digital Personal Data Protection Bill with explicit child data safeguards. 23 XYZ v. State of Maharashtra (2020) 3 SCC 637. 313 3. Mandate fairness, transparency, and explain ability in AI systems used for child protection. 4. Invest in training police and cybercrime units in AI-based detection tools. 5. Encourage Public-Private Partnerships between tech companies, NGOs, and the government. 6. Develop AI tools tailored to India’s socio-cultural and linguistic diversity. 7. Establish independent oversight mechanisms and child data protection authorities to audit AI deployments. Achieving this will involve multi-stakeholder collaborations, legal reforms, and robust ethical supervision to ensure AI becomes a protector, not a threat to the rights of children in the country’s digital era. However, as this study emphasizes, integrating AI into child online protection frameworks brings forth complex ethical, legal, and policy dilemmas within the Indian context. While India’s current legal provisions, notably the Protection of Children from Sexual Offences (POCSO) Act and the Information Technology Act are robust in addressing CSAM and related crimes, yet they fall short in regulating the unique challenges posed by unique implications of AI technologies. Furthermore the judiciary, through significant judgments in cases such as Puttaswamy, Anuradha Bhasin, and Prajwala, has acknowledged the constitutional importance of privacy and child rights in the digital space, but a comprehensive legal structure explicitly regulating AI ethics, child data processing, and algorithmic accountability remains absent. Concerns related to ethical children’s right to privacy, the necessity of informed consent for data collection, algorithmic discrimination, and the opaque nature of AI decision-making systems have yet to be meaningfully addressed within Indian policy discourse. Although AI-powered surveillance and content moderation tools have proven, effective, their risk infringing upon children’s informational autonomy and fundamental rights if not accompanied by adequate legal safeguards and transparent oversight mechanisms. In response to these findings, this paper proposes several key recommendations - First, India should introduce a comprehensive legal framework specifically addressing AI ethics and accountability, with specific provisions or child online protection. This framework must include mandatory transparency protocols for AI-driven processes, algorithmic fairness guidelines, and procedures for human oversight in decision-making processes impacting children. Second, the proposed Digital Personal Data Protection Bill should be fast-tracked and revised to include child-specific data protection norms in line with international frameworks such as EU’s GDPR and the UK’s Age-Appropriate Design Code. Third, AI based tools which are designed for child safety must undergo regular ethical audits and impact evaluations to assess their privacy implications, effectiveness and accuracy.