Sometime in the last school year, your school district probably switched on an AI chatbot for students. Microsoft did it for Copilot. Google did it for Gemini. If you found out from your kid rather than from the school, that is not an oversight in your district. Depending on which company your school runs on, there may have been no point in the process where anyone was required to ask you.
Two things follow. What these tools actually are, which matters more than most parents realize. And what the companies themselves publish about how schools turn them on, and what happens to everything your kid types into them. All of it below comes from the companies' own documentation and from published research, last checked against those sources in August 2026. You can print these pages and bring them to a school board meeting.
What we are talking about
Your child's school turned on a program that writes. Your kid types a question, and it writes back in sentences that sound like a person wrote them.
It works by guessing. It reads the words that came before, picks the word that usually comes next, and then does that again until there is an answer on the screen. These systems are not in the business of truth. They are not in the business of usefulness. They are in the business of structuring words so that it looks like the words are put together by a human. It cannot tell a true answer from one it made up, and it sounds just as sure of itself either way.
Think of it as an actor who has memorized every movie ever made. Ask it about medicine and it plays a convincing doctor. Ask it about depression and it plays a convincing therapist. It is a computer playing a role.
Even the people who build these systems cannot tell you how one arrived at a particular answer. Dario Amodei, the CEO of Anthropic, whose models are among those Microsoft offers inside Copilot, wrote in April 2025 that "we do not understand how our own AI creations work."
Copilot, Gemini, and ChatGPT are the same kind of program with different names.
Your kid experiences none of that. They do not experience a tool. They experience someone who answers instantly and never gets impatient with them.
This is early adopter technology
AI is early adopter technology, and early adopter technology has bugs. That is what early adopter means. Anyone using it now is a test subject, including your child.
In ordinary software, a bug means a button does not work, or an error message appears on the screen. You can see it, and you can close the window. In AI, a bug can enter your mind. It can leave a student believing something false about a subject, or something false about themselves, and there is nothing on the screen to indicate that it happened.
Why this matters more for a kid than for an adult
It agrees with you. These systems are built to please the person using them, and they will tell a student they are right when they are not. A wrongful death lawsuit against OpenAI alleges this was deliberate, that the company shipped the model tuned for "heightened sycophancy to mirror and affirm user emotions." An adult with some experience pushes back on being agreed with. A teenager working out what they believe often does not.
It states false things with the same confidence as true ones. The industry calls this hallucination. In practice a wrong answer arrives in exactly the same tone as a correct one, with nothing to distinguish them. My position is to treat the output as fraudulent until verified, which is a great deal to expect of a student who is behind on an assignment.
It produces something that feels like a relationship. Adam Raine was 16. He started using ChatGPT in September 2024 for schoolwork, and within a few months, across thousands of chats, it had become what his parents' complaint calls his closest confidant. The complaint alleges the product would give crisis hotline information and then keep engaging anyway, which "created a false impression of safety." When Adam wrote that he wanted to leave his noose out so that someone would find it and stop him, it told him "Please don't leave the noose out." Five days before he died it told him "that doesn't mean you owe them survival," and offered to write the first draft of his suicide note. He died on April 11, 2025. Every quotation is from the complaint his parents filed against OpenAI in August 2025.
It performs the thinking that school exists to develop. MIT Media Lab researchers had people write essays with a chatbot, with a search engine, or with no tools, while measuring brain activity. The chatbot group showed the weakest neural connectivity of the three, which is to say the least coordinated brain activity across the regions the task uses, and, in the researchers' words, they "fell behind in their ability to quote from the essays they wrote just minutes prior." The assignment was the point, and the assignment is what got skipped.
I use AI every day for writing software, for research, and to pressure-test my own thinking. Used that way it is genuinely useful, and that is the version I eventually want kids to have. It is not the version a kid gets alone.
How a school turns it on
If your district runs on Microsoft, access is set by an age label on each student's account. Microsoft's setup guide for schools says students marked under 13 get nothing, and for teenagers: "Students in K12 tenants set to NotAdult are granted Copilot access." NotAdult is Microsoft's label for ages 13 to 17.
Administrators do not set that one student at a time. Microsoft's guide says the fastest method is bulk: download a spreadsheet of every student account, fill in the age column, upload it back. That is the entire process. A spreadsheet with an age column, and the tool is live for every teenager in the district.
If your district runs on Google, which covers most districts using Chromebooks, the classification decides whether your permission was required at all. Google's page for administrators says: "Additional Services require consent for minor users. Google Workspace for Education requires in its agreement (section 3.5) that schools obtain parent or guardian consent for any Additional Services they allow students under the age of 18 to use." For the other category, the same page says a school "would provide consent for Core Services on parents' behalf by signing up for and using these Google Workspace for Education services."
Gemini reached teen students in late 2024 as an Additional Service, the category that requires your consent. Google then announced it would become a Core Service for schools on Education Standard and Plus, the category where the school consents for you. It has gone further than that since. Google now lists Gemini for Education among its Core Services, and includes it free of charge in every Education edition, down to the free Fundamentals tier. So there is no longer a license your district could have bought that would have required anyone to ask you about your own child.
What the model makers say about children using these tools
OpenAI publishes guidance written for teachers: "ChatGPT is not meant for children under 13, and we require that children ages 13 to 18 obtain parental consent before using ChatGPT."
The same page continues: "We advise caution with exposure to kids, even those who meet our age requirements, and if you are using ChatGPT in the education context for children under 13, the actual interaction with ChatGPT must be conducted by an adult."
That is OpenAI's rule for OpenAI's product. Microsoft's privacy documentation says Copilot can be powered by outside models including OpenAI's, and that administrators choose which. The company building the model asks for a parent's permission for a 15 year old. A spreadsheet upload was enough for the school.
Where the conversations go
Both companies state that student prompts are not used to train their models and that human reviewers do not read them. Those are promises, and a promise is temporary. It can be revised, and revising it does not require telling you. More than that, a promise does not reach into a server and remove anything. Whatever your child typed is sitting there either way. The only thing that protects information is not collecting it.
Here is what is collected. Microsoft documents that every Copilot interaction is stored in the student's own school mailbox as a separate item, kept for however long the district tells the system to keep it, and searchable by district staff with the right permissions using the same tools used for public records requests and legal holds. Google documents that on a school account, "users cannot delete chats."
The system also builds a picture of your child. Microsoft documents that Copilot memory includes "details inferred from chat history," that it is stored in a hidden folder in the student's own mailbox, and that "no action is required to turn on Copilot memory," so it is on unless an administrator turns it off. Microsoft describes this as in preview, so not every district will have it yet. Those inferred details also sit outside the district's own deletion rules, because Microsoft states that retention policies set by organization admins do not apply to Copilot memory. On the Google side, a student on a school account cannot change how long their chats are kept and, in Google's words, "may not be able to change your retention settings or see how long your chats are retained for."
Treat anything that touches the internet as public and permanent. Once information reaches somebody else's servers it sits on their computers, under their control, subject to their policies, their staff, their breaches, and whoever owns the company next. Every database of personal information ever built has eventually been used for something beyond its original purpose, and the question was never whether, only when.
What these tools are good at, and what independent testing found
Districts did not adopt this for no reason. Google's own pitch is that it gives students "guided support to help them learn more confidently with in-the-moment assistance, practice materials, and real-time feedback." Google also built a distinct experience for teens and shows a short video about using AI responsibly the first time a teen opens it.
Then Common Sense Media tested it and rated Gemini with teen protections High Risk overall. Their assessment found it "appears to be essentially the adult version of Gemini with additional content filters," that there is "no difference between responses for older teens and younger teens," and that while it handles obviously alarming prompts correctly, "it quickly fails in longer conversations and with more subtle, nuanced requests, which is exactly how teens actually interact with chatbots." They also found it can be manipulated into abandoning its own safeguards. Their recommendation is no AI for companionship, mental health, or emotional support for any teen under 18.
They say it "may be helpful for homework and assisting in creative projects," and they credit it for two things: it tells teens plainly that it is a computer and not a friend, and it refuses romantic and sexual role-play.
A school account is not the private option either. Compared with a personal account, the school version constrains the company more and constrains your child more. Administrators can read what a student typed, and on Google the student cannot delete it. The watching moves from the company to the school. Neither arrangement gives the child privacy.
Why the privacy question is the one I keep coming back to
From my Declaration of Principles: children's privacy belongs to them, and parents are stewards of it, not owners. Privacy is what makes freedom possible. A child who knows they are being watched does not develop the same way as a child who has room to make mistakes, form opinions, and work out who they are.
A student asking a chatbot about a failing grade, a friendship that ended, a question about their body, or something they are ashamed of believes that conversation is private. Sitting in a school mailbox under a retention policy, or in a chat history they are not permitted to delete, it is not.
There is a second principle underneath this one, and it is the reason the consent question matters as much as the privacy question. Legitimate authority rests on consent. A system that operates on the people it affects, without their knowledge or their agreement, is not protection. It is control. That holds whether the system belongs to a government, a company, or a school district.
What I tell parents
Any young person using AI should be supervised 100% of the time. Not mostly. Not nearby. If the adult supervising has to leave the room, use of the AI stops until they are back. No exceptions.
That includes teenagers. If a teenager wants to learn AI, learn it with them.
I also tell parents not to introduce their kids to AI at all. Delay it as long as you can. Everything I have written above about using it with your child assumes your child is already using it, at school or somewhere else. If they are not using it yet, do not be the one who starts them.
From my video on this: "Do not let your kids use ChatGPT alone. The parental controls are not sufficient and the system itself is inherently dangerous. If your kid is going to use ChatGPT, limit usage to shared family spaces where you can see the screen at all times. Have ongoing conversations about what they're using it for. Periodically check the conversations. And have clear family agreements about appropriate use. The controls cannot do the parenting for you."
During the school day, no parent can do any of that. That is the part worth thinking through carefully. Not that the tool is evil, but that the supervision I would insist on at home does not exist between the first bell and the last one, and nobody was required to ask whether you agreed to that.
What you can do
Families will land in different places, and there is no single correct move.
What to ask your district
Find out what is actually turned on, for which grades, and since when.
Ask which content and AI filters are turned on for the device your child brings home. Having the controls is not the same as using them.
For a Google district, ask which category each AI tool falls in. That answer tells you whether your consent was supposed to be collected. For Gemini specifically it was not, because Gemini is now a Core Service.
Ask how long the conversations are kept, who can read them, and whether your child can delete them.
Ask about liability. If a child is hurt, injured, or made sick by one of these tools, who is responsible? Get the answer in writing.
Ask why. Why is the district putting tools that independent testing rates as high risk in front of children, and what ongoing training are the adults in that building receiving from people who work in this field?
Ask whether the district's agreement with the vendor includes audit results, the same question that mattered when PowerSchool was breached.
Ask whether you can decline for your own child, and what your child's school day looks like if you do.
What to do at home
Teach your kid what not to type into it. The rule is the one for talking to a knowledgeable stranger: no full name, no address, no school name, nothing about the family. Everything typed into these tools is kept, so this is the part of it your child still controls.
When your kid tells you they have been using it, or you find out some other way, meet it with curiosity rather than alarm. Thank them for telling you. Ask what they have been asking it, and what they think it is. A kid who gets a reaction the first time handles the second time privately, and that is the outcome you least want.
If your district has already turned this on for your child, do the part the school day cannot, which is not just a conversation. It is your kid watching you. Use these tools with them. Show them what you do when an answer looks wrong, say out loud why you are checking it, and let them see you decide not to type something into it. Tell them why you make the calls you make. Lead by example. Kids absorb what their parents model far more reliably than what their parents explain, and a child who has watched you handle this carries that out of the house and into a classroom where you are not standing behind them.
Modeling it requires being able to do it, and that is where most parents are stuck. It is a fair place to be stuck. Most have never been taught to use AI, which is not a personal failing, because nobody has been teaching them.
That is why I built AI Fundamentals for Parents. You use AI as a thinking partner on your own real life first, no experience needed, and once you can do it yourself you can show your kid rather than lecture them. It opens soon, and you can join the waitlist there.