Millions of students are unwittingly surrendering their personal data and emotional well-being to AI chatbots every day, sparking concerns among educators, parents, and kids about how this technology is being used in schools. The use of artificial intelligence has become ubiquitous in classrooms, embedded in many required tools for learning, but its impact on student safety and privacy remains unclear.
School districts are starting to take action. In a bid to address the growing AI backlash, New York City and Los Angeles, the country’s two largest school districts, have announced a one-year moratorium on student use of AI, pending further review of how to incorporate the technology into education responsibly. This move reflects a wider recognition among educators that the benefits of AI in schools must be balanced against its potential risks.
In response to these concerns, Microsoft has made a significant commitment by agreeing to adopt strict guardrails and privacy standards for its AI products used in schools. The agreement, negotiated with the American Federation of Teachers (AFT), the country’s second-largest teachers union, sets out limits on how student data can be collected, stored, and used. For instance, Microsoft has pledged not to use student or educator data to train AI systems, except for narrow exceptions related to student safety.
The agreement also prohibits AI companions and features designed to foster emotional attachment or dependency in students, as well as the use of AI tools that keep students engaged beyond what is necessary for a learning task. Crucially, Microsoft has agreed to regular third-party audits to ensure compliance with these new standards, which will apply to all schools with existing contracts from November 1st.
While this move by Microsoft represents an important step forward in addressing concerns about AI safety and privacy in schools, it remains unclear whether other tech giants will follow suit. Josh Golin, executive director of online safety nonprofit Fairplay, warns that the agreement may be misunderstood by schools, who might assume that using a particular tool is acceptable if it meets these new standards. Golin argues that this could lead to a false sense of security and undermine efforts to protect student data.
The AFT has hailed the agreement as a significant achievement, which sets an industry standard for AI safety in schools. Two other leading AI companies, OpenAI and Anthropic, have expressed interest in discussing similar pacts with the AFT. However, Google, the dominant provider of education technology in American schools, has yet to confirm whether it will offer similar protections.
Ultimately, this development highlights the need for industry-wide agreement on standards for AI safety and privacy in schools. As we navigate an increasingly digital world, our children’s data and well-being must be protected with equal urgency and attention. Parents and educators should remain vigilant and demand that all companies operating in schools prioritize transparency, accountability, and student safety above commercial interests.
In practical terms, parents can take steps to protect their children’s online presence by:
* Reviewing school technology contracts and demanding greater transparency from educational institutions
* Encouraging schools to adopt strict AI safety standards for all tools used in classrooms
* Staying informed about emerging technologies and how they may impact student data and well-being
By working together, we can create a safer digital environment for our children and ensure that the benefits of AI are realized without compromising their fundamental rights.
Source: SecurityWeek — 2026-09-15