Should our university run AI detection at all, or write a policy that works without it?

Secondary school, so different rules, but here is what happened when our district switched the indicator on. Referrals went up. Findings did not. What went up was the number of meetings in which I explained to a parent, with a shaken sixteen-year-old in the room, that the number on the screen was not evidence of anything. Twice I was explaining it to a colleague.

If I had a vote on your committee I would take option (b). But I teach fifteen-year-olds, and the people who got hurt were the careful kids, the ones who revise. Your undergraduates may be different. The dynamic I would bet is not.

TA perspective, and I have lived under something close to your option (c). Our department keeps the indicator on, but the written policy says a score may not be cited in an allegation and may not be mentioned to the student as a reason. It is a prompt for the instructor to look more closely: ask for drafts, look at version history, have a conversation about the argument.

Under that rule it has mostly worked. What has not worked is onboarding. Every term a new instructor sees a high percentage, has not read the policy, and writes the student up directly. So if you choose (c), the policy text matters less than how you make sure every person who can see the score has read the paragraph that constrains it. I tried a one-page summary attached to the course shell. It helped, a bit.

A policy can work without a detector, and several universities have chosen that route. If you keep one, vendor guidance and the research point the same way: a score cannot be the sole evidence, a human reviews it, and the student has a right to reply. Decide the process first, then whether a score has any place in it.

The clearest public example of option (b) is Vanderbilt's August 2023 notice, "Guidance on AI Detection and Why We're Disabling Turnitin's AI Detector". Their reasons: the feature arrived with less than 24 hours' notice and no way to switch it off, the vendor gave no detailed account of how it reached a determination, false accusations elsewhere were being reported, and the non-native English research had come out. They also raised a point your committee should not skip: sending student work to a third-party detector is a privacy question in its own right. Their alternative is your option (b): set expectations early, require disclosure and citation where AI use is allowed, redesign assessments toward in-class writing and course-specific topics, compare suspect work to the student's other writing, check for fabricated sources, and talk to the student.

OpenAI's educator FAQ, "How can educators respond to students presenting AI-generated content as their own?", is worth putting in front of anyone arguing for (a). Asked whether AI detectors work, it answers "not in our experience," reports that their own classifier labelled Shakespeare and the Declaration of Independence as AI-generated, notes indications of disproportionate impact on students learning English and on formulaic writers, and observes that small edits defeat detection anyway. Its alternatives are process-based: shared conversation logs and cited AI use.

If the committee lands on (c), write the restriction into the evidential standard rather than a usage note, since usage notes are what Priya's new instructors do not read. See the thread on which detectors colleges use and the false flag thread for how it looks from the student's chair.

Option (b). Turn it off.

A tool that cannot be explained to the person it accuses has no place in a disciplinary process. Full stop. If a client accused me of plagiarism on the word of a black box, I would walk and take my invoices with me. Your students cannot walk. That is the difference, and it is the whole argument.

@Maya Lindqvist thank you. The distinction between deciding the process first and the tool second is the framing the committee needed, and the privacy point about third-party processing of student work had not appeared in our papers at all. I will circulate both documents before the next meeting.

One caution on the Vanderbilt arithmetic, since it gets repeated as if it were a finding. Taking a vendor's stated false positive rate and multiplying it by annual submissions gives an illustration, not a measurement. The number of wrongly flagged students depends on the share of submissions that actually contain AI text, the threshold, document length, the writing population, and how the rate was measured in the first place. The underlying point is sound: errors scale with volume, and nobody outside the vendor can check the rate on your students. I would just quote it as "we do not know how many, and neither does the vendor" rather than as a count. That version is also harder for a sceptical colleague to dismiss.

Please, whatever option you choose, write down clearly that the student can see the full report and respond before anything is recorded in a file. When my friend was referred last year, she was told a percentage and nothing else, and she did not know what it meant or what she was allowed to say. She was cleared, but it took five weeks, and for five weeks she thought she would lose her visa. A sentence in the policy about the right to see and reply would have made that different.

@Hana Sato fair, and I will stop quoting the count. "We do not know how many, and neither does the vendor for our students" is more honest and, you are right, harder to wave away in a faculty meeting.

Formally, Tobias, the question in front of your committee is the evidential standard, not the tool. Most codes decide on the balance of probabilities. A detector score is at best an unexplained expert opinion with no witness available to question, no methodology disclosed, and a known error rate in a population that is not yours. On its own it should not meet the standard under any of your three options.

So if (c) is chosen, I would define corroboration in the text: what counts (drafts, version history, a viva on the content, inconsistency with the student's other work) and what does not (a second detector agreeing with the first). I would also state in terms that a score does not shift the burden onto the student. Without that sentence, "respond to this percentage" becomes "prove you wrote it," which is the inversion Aisha's friend experienced.

@Kwame A. the corroboration list is exactly the sort of text I was looking for. Our code already uses the balance of probabilities. I will propose a clause stating that an indicator score alone does not meet it and does not shift the burden, and we will see whether the committee accepts (c) with that attached or decides the simpler course is (b).