Teaching Responsibly with AI, Office Hours: GPTZero’s New AI Detection Tools

This was the second office hours in our Teaching Responsibly with AI webinar series. We sat down with Robert Nicolaides (Head of Design at GPTZero) and Joe Zakielarz (founding software engineer at GPTZero) for a closer look at the platform’s latest AI detection features, including a brand-new Bibliography Checker.

The core message from the session was that educators don’t need to become AI police. Instead, the tools are evolving to give teachers better data points, clearer conversations, and stronger support. Here are some key takeaways. 

Six Most Obvious Signs of AI-Generated Text

Robert launched the session by laying out six of the most obvious warning signs when it comes to spotting text that hasn't been written by a human.

Repetitive Patterns

“Writing as humans means we tend to mix and match our language with a rhythm that isn’t identical to someone else. In AI-generated writing, the text patterns can get overwhelmingly repetitive.”

Monotonous Tone

“Even if the assignment is formal or highly academic, human writing tends to have an inherent sense of voice. On the other hand, AI-generated writing can feel quite flat.”

Atypical Voice or Stylistic Choices.

If a student’s writing suddenly reads like a professional columnist, that’s worth a second look. Robert said, “If your student suddenly starts turning in work that sounds like a New Yorker journalist or a Nobel laureate, you may want to compare it to their in-class assignments.”

Zero Typos

“Work that is absolutely flawless - flawless grammar, zero typos - can actually be suspicious. This is especially true if that particular student’s work isn’t typically perfect.”

Generic Examples vs. Real-World Detail.

“The keyword here is cookie-cutter. AI content tends to be vague and lacking in specificity, especially for recent events.”

Lack of subject matter expertise

“AI can spit out loads of words very quickly, but it tends to lack genuine subject matter expertise.”

How AI Detection Tools Work

Unlike plagiarism detectors that simply compare text to known documents, GPTZero analyzes writing patterns using multiple models layered together. Robert explained, “Our tool uses - I believe it’s seven different components - to process text and identify if it was written by AI. This works together to produce predictions that are as accurate as possible.”

When teachers scan a document, they don’t just get a blunt ‘AI or human’ label. Instead, GPTZero color-codes sentences from light green (strongly human) to dark yellow (likely AI), along with detailed explanations.

For example:

• A flagged sentence might exhibit “robotic formality”:

“The sentence is formal and clear but structured phrasing (for example: key traits of a good citizen) but the repetitive tone contributes to a robotic feel.”

• Or “rigid guidance”:

“The sentence focuses on practical advice by suggesting specific actions parents can take to instill values in the children.”

Meanwhile, human sentences often show "varied sentence structure”, “subjective stance” and “qualitative focus.” As Robert put it, “This allows educators to have a much more sophisticated conversation with students. It’s not simply about catching them.”

What to Do If You Suspect AI Usage

What happens if you suspect a student has used AI-generated text? AI isn’t going anywhere. These are some ways you as educators can set your students up for success.

First of all, don’t punish; discuss. Suspecting AI or getting a positive result should not automatically result in academic punishment. If you receive a positive AI detection, think about taking some steps to work with students constructively.

Ask them to demonstrate their understanding in a controlled environment, like an in-person assessment or an editor with an edit history feature like Google Docs. You can also ask to see the drafts, revision notes, or brainstorming documents.

Crucially, see if there’s a history of AI-generated text in the student’s work, as it’s better to be on the lookout and look for that long-term pattern of AI use and not just rely on a single instance to gauge whether the student has used AI.

The second is training students on academic integrity. You can start an open dialogue around the ethics of AI that can help students see these tools with a more clear-eyed perspective as opposed to seeing them as shortcuts to simply ticking a box and getting their work done.

The third is creating clear guidelines for assignments. Clarity has never been more important when it comes to using AI and assignments, though this can be understandably a challenging area for educators, as it’s still such a recent development that’s always changing.

Articulating your expectations for acceptable AI use means that students have boundaries about what is and isn’t okay when it comes to using those AI tools. Guidelines can help students explore AI responsibly without veering into plagiarism territory.

Introducing the New Bibliography Tool

Robert introduced Joe to present the next phase of AI detection, highlighting the growing issue of hallucinated citations, with over 50% of GPT-4’s citations estimated to be fabricated.

Joe, one of GPTZero’s engineers, built and launched the bibliography tool on May 15th. Although still in its early stages, the tool has already received a lot of positive feedback from educators.

Joe explained that the purpose of the bibliography scan is simple: to check whether cited information in research papers is valid, and whether it actually supports the points being made.

The idea for the tool came directly from what teachers were reporting. Back in February, GPTZero started hearing from educators that students were trying to humanise AI-generated text, but bibliographies were often where things fell apart. Citations were being made up. As Joe put it: “Some of you might have seen this, maybe you’re nodding your heads, like, this is a pretty big problem.”

As they spoke to more teachers, it became clear how common this was: “Pretty much all of them were like: ‘Yeah, I’ve seen this before, this is a pretty big deal.’”

Joe walked through a few of the biggest challenges teachers described. First, teachers are overwhelmed, especially those teaching online, grading lots of papers with limited time. Many weren’t able to check bibliographies at all, or could only verify whether the bibliography existed, not whether the sources were legitimate. The manual process - searching source by source, copying links (if they exist), or manually hunting for citations - just isn’t feasible at scale.

But even when teachers knew citations were fake, they often lacked leverage. As Joe described, “Teachers would say: ‘I know.’ They’d show us papers and say, ‘This is fake.’ And we’d confirm that. But the problem was they didn’t have any leverage to negotiate with students.”

That’s where the idea for this tool came in: to serve as a third-party verification tool, something educators can point to when discussing suspected cases with students. The same principle that applies to GPTZero’s original AI scan applies here: it’s a tool to support the teacher.

How the Bibliography Tool Works

Joe described this as part two of AI detection. While GPTZero’s AI scan evaluates the writing itself (looking at phrasing, patterns, and writing style to assess authorship), the bibliography tool focuses on the information inside the document.

“Teachers, your current workflow right now is: if you see citations, you Google search it, Bing search it, check if it exists. Maybe you’ll dig a little deeper and check Semantic Scholar or academic databases, but that’s pretty much it.”

Now, GPTZero automates all of that, as the tool scans the essay, extracts the bibliography, and searches multiple databases and sources simultaneously: search engines, academic journals, and publications. It replicates what teachers would do manually, but a lot faster.

Once the tool finds a source, it doesn’t stop there. The AI reads both the source and the student’s paper to assess whether the citation is actually relevant and being used correctly. Joe explained: “AI will often grab a quote or fact from a real source, and on the surface, it might look fine... but when you read the source, you realise the essay kind of just got lucky pulling something that sounds good but isn’t actually related.”

The tool also compares what the student says about the source to what the source actually says. If, for example, a student writes “X is Y,” but the source actually says “X is Z” (the opposite), the system flags that. Joe added: “AI gets the general gist right but will often say something completely wrong.”

As an added feature, the tool also includes a kind of “uncited claims detection.” If GPTZero spots sentences that look like they should be backed up with a citation but aren’t, it highlights them in grey. Teachers can then review and decide whether additional sourcing is needed.

How to Interpret the Results

Joe then walked through how teachers can interpret the tool’s output. The system flags citations with simple colour codes:

• Green means GPTZero is very confident there’s no issue.
• Red means there’s a strong likelihood of fabrication or misuse.
• Yellow is for ambiguous cases, where GPTZero isn’t certain and recommends a closer look.

“As you use the tool more, you’ll build up more intuition around when to check yellow flags. Every teacher has different policies and philosophies around what counts as academic dishonesty, and we wanted to leave space for you to apply your judgment.”

Joe wrapped up by noting that the bibliography tool is still very new. Released on May 15th, it’s already been used by over 20,000 people but remains a young product compared to the rest of GPTZero’s suite. He said, “We’re improving accuracy, reliability, and speed, and we’re listening to feedback. If you have ideas or technical questions, we’d love to hear from you.”

Watch the entire replay here.