Are you leaking your questions to AI
You paste a question into ChatGPT. You refine it. It's exam-ready. And it's no longer confidential.
Are You Leaking Exam Questions to AI?
You paste a draft question into ChatGPT. You ask it to improve the distractors. You refine the stem. You iterate until it's exam-ready.
The question is better. And it's no longer confidential.
How Memorisation Works
Most consumer AI services retain your conversations, not just for your history, for model improvement. Content you submit may be analysed, stored, and incorporated into future training cycles.
Once your question enters that pipeline, it becomes retrievable. Not by you. By anyone.
Research on large language models shows that targeted prompts can surface specific training examples with surprising accuracy. A candidate who knows — or guesses — that questions were developed using a particular chatbot can craft queries designed to elicit similar content.
This doesn't require technical sophistication. Publicly available prompt engineering guides include strategies specifically designed to extract memorised content. Online communities share methods openly. A motivated student with a subscription and half an hour of practice could attempt it.
The Scale of What's Happening
Across universities and professional bodies, academics are already using public chatbots to draft questions, generate distractors, rewrite stems, and review existing items for technical flaws.
In most cases, this work is informal and undocumented. Neither sanctioned nor prohibited. Assessment committees may not know AI was involved. The author may not realise the security implications of a free web chat.
The result: confidential assessment content distributed across commercial AI training pipelines. No audit trail. No retention policy review. No risk mitigation.
What a Leaked Question Costs
A single high-quality MCQ for professional assessment represents one to two days of combined professional effort. Author time. Subject matter expert review. Psychometric review. Revision cycles. Standard setting. Metadata tagging. Maintenance as guidelines evolve.
When a question is compromised, it must be retired immediately. Blueprint coverage becomes incomplete. Candidates who sat earlier exams with the leaked content gained an unfair advantage. Score validity across multiple cycles may be questioned.
The financial cost is direct. The reputational cost, particularly for professional certification, can be severe.
But the real risk is that you don't know it's happened.
What You Can Do Now
Use enterprise configurations. Major AI providers offer tiers where your data is excluded from model training. Consumer accounts do not provide this by default. Check your tier.
Use API access where possible. Content submitted via API can be processed ephemerally — analysed and discarded without storage.
Keep finished questions out of public chat. Use chatbots for early ideation if you must. Explore structures, generate ideas, develop testing points. Never paste complete exam-ready questions into a consumer interface.
Document AI use in your item metadata. If a question was co-developed with AI, record when, how, and which service was used. If a security incident occurs, you need that trail.
This is part two of a series on AI in assessment. Previously: Can AI Write My Exam Questions?. Next: Your Biggest Exam Security Risk Is an Open Browser Tab — the vulnerability almost nobody is talking about.
CrtQ — Sharper questions. Smarter exams. crtq.ai