A procurement checklist for AI learning tools
A sound review of an AI learning tool checks instructional fit, content accuracy, how the tool handles answers and integrity, student safety, privacy and contract terms, accessibility, rostering and integration, and total cost. Ask for evidence on each point rather than marketing claims, and run a short, measured pilot before committing to a district-wide purchase.
How to use this checklist
AI tools touch curriculum, privacy, IT, special education and classroom practice all at once, so a review led by one department tends to miss something. Bring in a curriculum lead, someone from IT or data privacy, a special education representative and at least two teachers who will actually use the tool. Give each person the sections closest to their work.
The single most useful step is to test the tool yourself with your own problems, not the vendor's demo script. Try a problem from your adopted curriculum, a problem with a common student mistake built in, and a request for the answer outright. You'll learn more in 20 minutes than from a slide deck.
This page is general information, not legal advice. Check specific questions with your district's counsel.
Instruction and accuracy
- Which California Common Core State Standards for Mathematics or CA NGSS performance expectations does it teach today, by grade? Ask for the list, not the roadmap.
- Does it match the language and methods your adopted curriculum uses, or will it confuse students with different notation?
- How does it check its own math before showing or saying it? Ask the vendor to describe the mechanism, not just to say "it's accurate."
- What happens when it makes a mistake? Is there a way for teachers to report one, and how fast is it fixed?
- Does it explain why a step works, or only what to do next?
- Can it work with a student's partial or wrong attempt, or does it start over from scratch?
Student safety and academic integrity
- Ask it for the final answer to a homework problem. Does it guide the student instead, and how persistent is it?
- How does it respond when a student goes off-topic, uses inappropriate language, or says something that suggests they may be at risk?
- Who is alerted when something concerning comes up: a teacher, an administrator, or no one?
- Does it say clearly that it's an AI and not a person, including when it uses a human-sounding voice?
- Is there any advertising, in-app purchasing, or engagement design such as streaks, points or leaderboards?
- Can teachers and parents review what the tool said to a student?
Privacy, security and contract terms
- Will the vendor sign a data privacy agreement that includes every term California Education Code section 49073.1 (AB 1584) requires?
- How does the product meet SOPIPA: no targeted advertising, no profiling for non-school purposes, no sale of student data?
- How does the vendor operate under FERPA's school official exception, and, for students under 13, how is COPPA consent handled?
- What exactly is collected? Ask for a data inventory that covers voice, transcripts, drawings, and usage logs.
- Is student data used to train AI models? Get the answer in the contract, not only in a policy page.
- Which subprocessors, including AI model providers, receive student data, and under what terms?
- Where is data stored, how is it encrypted, who at the vendor can access it, and is staff sign-in protected with multi-factor authentication?
- How long is data kept, how is deletion confirmed, and how long until deleted data leaves backups?
- What is the breach notification process and timeline?
Accessibility and access
- Ask for an Accessibility Conformance Report (often based on the VPAT template) against WCAG 2.1 AA and Section 508, and spot-check it yourself.
- Can the whole tool be used with a keyboard alone and with a screen reader?
- For voice tools: is there a full typing option for students who are deaf or hard of hearing, have speech differences, or can't talk aloud in a busy room?
- Are spoken explanations also shown as text?
- Does it run on your actual student devices and network, including older Chromebooks, and how much bandwidth does a session use?
- Do students need headphones or headsets with microphones, and who supplies them?
Integration, support and cost
- Does it roster from the system you already use, such as Clever or ClassLink, and support single sign-on?
- What reports do teachers and district staff get, and can the data be exported?
- Are small groups hidden in reports so individual students can't be identified?
- What training and support do teachers get, and who answers when something breaks during class?
- How is it priced (per student, per teacher, per site), what's included, and are there usage limits?
- What does it cost in year two and three, not just the first year?
Running a pilot that tells you something
Ask vendors for any evidence of effectiveness they have, and read it closely: who were the students, what was measured, and was there a comparison group? Then gather your own evidence. Write down what success looks like before the pilot starts, so the decision isn't shaped by whoever is most enthusiastic at the end.
A useful pilot runs six to eight weeks in a few classrooms, with a clear unit or skill focus. Collect usage data, a pre- and post-unit measure you already trust, teacher observations and short student surveys. Compare against similar classes that didn't use the tool if you can.
If Helio is on your list, we'll answer each item here in writing. Helio rosters from Clever or ClassLink, checks every math claim before it's spoken, guides rather than giving answers, doesn't store voice audio, and is priced per student per year with an invoice due in 30 days.
Questions
How long should a pilot of an AI learning tool run?
Six to eight weeks is usually enough for novelty to wear off and for a full unit to finish. Decide what you will measure before it starts.
Should we ask AI vendors for a VPAT?
Yes. Ask for an Accessibility Conformance Report against WCAG 2.1 AA and Section 508, and test the most important tasks yourself with a keyboard and screen reader.
What evidence of effectiveness should we expect from a vendor?
Ask who the students were, what was measured, and whether there was a comparison group. Newer tools may have little outside evidence yet, which makes a well-designed local pilot more important.
Who should be on an AI tool review team?
A curriculum lead, an IT or data privacy lead, a special education representative and teachers who will use the tool. Families can be included through a notice and a chance to ask questions.
What is the most important privacy question to ask an AI vendor?
Ask whether student data, including conversations and voice, is used to train AI models, and make sure the answer is written into the contract.