In the last few weeks of the spring semester this year, Turnitin dropped an AI detector into its plagiarism reports, and I, along with writing teachers everywhere, got slammed with questions of what to do with this information. Was it reliable? Had the students really cheated? Did the students know they were cheating? What could we even do about it if we had not taught anything about AI and had no policies related to AI. I was scrambling, and it was not pretty. It was stressful for me and for the students.
Turnitin claims its AI detection yields only 1% false positives. Birth control pills companies claim the same thing, and we all know someone who falls into the 1%. Turnitin has admitted that the accuracy of its AI detection is a little lower for high school students as opposed to higher ed students. Furthermore, Stanford researchers say AI detectors in general have a bias against ESL students. This makes me wonder how that bias might affect students who speak a non-standard dialect of English at home.
I’ve only gone through two essay cycles with students so far with the AI checker, and I did have a handful of students flagged both times. I communicated with those students directly before ultimately deciding to have them redo the parts marked as AI in the reports. I believe in using tools like Turnitin as teaching devices. I would never use the plagiarism detector just to catch students either intentionally or unintentionally plagiarizing without giving them a chance to see their own reports and make corrections accordingly. Often what the plagiarism detector finds is just sloppy paraphrases. I encourage students to use the reports to help themselves figure out where their paraphrases need more work before reaching the stage of the final draft. One major problem with the AI detector is that students can’t see their own reports. To have students redo their work based on these reports, I had to take screen shots and email them to the students. I really hope Turnitin reconsiders this in future.
Based on all of the recent hype, I expect ChatGPT to be the biggest issue with AI I had to deal with, but it wasn’t. Instead, my nemesis was a little program called Quillbot. We had a lovely cycle of unintentional plagiarism to AI going on. The students were getting flagged by the Turnitin report for small instances of plagiarism in their sloppy paraphrases, so they were loading their essays into online grammar and plagiarism checkers that promised to help them avoid plagiarism. That was a good little game while it lasted. Then the AI detector dropped, and suddenly their paraphrases went from being marked as plagiarized in one draft to being marked as AI in the next draft.
What this tells me is that there was a teaching void that I was unaware of until this tech vs tech tool dropped into my classroom uninvited. Now that I am aware, it is my job to prepare to teach my students not to fall into this trap. Right now I’m just reading and thinking. I plan to write a series of blog posts here as I organize my thoughts. Ultimately, that should lead to a series of classroom materials that I can use and share.
