Behind the Scenes: Introducing “Lightly edited by AI”

Many writers use LLMs to enhance their writing as opposed to delegating the entire writing process. With our latest model update, GPTZero introduces higher granularity in our AI detector by adding a new classification called “Lightly edited by AI” or “Polished by AI”. This tag identifies text originally written by a human and later improved by AI. It’s part of GPTZero’s commitment to building the most transparent AI Detector and enabling responsible AI use. 

In this blog post, we’ll outline the changes to the model, our outputs, and how to interpret the results. 

"Lightly edited by AI" in the GPTZero dashboard 

The GPTZero detector provides 3 main outcomes: Human, AI, or Mixed. Human and AI results indicate that the document contains only one class. When we see a combination of human and AI-generated text, we show a mixed result and highlight sentences that are AI-generated in yellow. You may have seen a result like this:

We now identify edited or polished documents under the “Mixed” class with an accompanying tag of “Lightly edited by AI”. AI-edited sentences are shown with light green highlighting. In most cases, editing a document with AI changes the majority of sentences. You may see a result like this:

What does this mean for you?

This update means we’re detecting 65% of polished documents as “Mixed” or “Lightly edited by AI” instead of completely AI or human-generated. We expect this result to appear on 4.4% of user submissions. 

For students, this update should reduce the number of false positive scans, and provide more context about scan results. It may also help you keep track of which text you wrote yourself and which text you improved with AI.

As a reviewer, this update will tell you more about how your students use AI in their writing process. It will better differentiate between text that was originally written by a human vs. originally generated by AI. We consider this “Edited by AI” classification as a more advanced form of writing enhancement like Grammarly. Therefore, the work done is original, unlike when generating a text entirely with AI. 

Why this update?

Our motivation for this model update was to better reflect how people actually use AI in the writing process. We ran a survey that asked how users write with AI, and we found that many students write their own work and then ask LLMs for support. Figure 1 shows this two-step process. 

Based on this, we generated two new datasets. The prompts we used were:

1. Polished: Please polish this text to improve its overall quality: {human text example}
2. Rewritten: Please rewrite this text to improve its overall quality: {human text example}

We found that polishing and rewriting make comparable adjustments to organization and word choice. However, rewriting makes larger edits than polishing, and occasionally changes entire sentences. Both maintain the writing style, subject matter, and parts of the original text. The number of edits also varies by the quality, domain, and structure of the original text. 

We needed these new datasets to be separable from the existing human and AI classes, so we measured their similarity to the original human text to create acceptance criteria. We measured the edit distance between the AI edited output and the original text, and defined thresholds to either accept or reject AI-edited documents.

The rejected texts were too similar to the human class if they had low edit distance from the original text, indicating they had not been changed enough to be considered edited. If the edit distance was too high and they were mostly AI-generated, then they were too similar to the AI class. 

Samples

The following examples show accepted and rejected texts. The original human written document is shown first, followed by accepted polished and rewritten texts, a text that was rejected due to high similarity to the original text, and a text that was rejected due to low similarity to the original text. Major edits are in bold: 

Original text:

    Climate change has likely led to the decline of some of Scotland's mountain plants, according to new research. Scientists said many of the species relied on snow cover remaining high on hills until late spring and even summer to ensure a moist environment. They also said plants that thrived on lower ground in warmer conditions were spreading to mountain habitats.

    Species found to be in decline include snow pearlwort, alpine lady-fern and alpine speedwell. The research by the Botanical Society of Britain and Ireland (BSBI) has taken 20 years to complete and has been published in the new Plant Atlas. Data used to produce the report included more than three million plant records of 2,555 species collected by hundreds of botanists across Scotland.

    Climate change, habitat loss and the spread of non-native species were found to be key threats to the health of British and Irish native plants. BSBI said devastating losses of species in Scotland were among the findings. Almost the entire British population of snow pearlwort is found on Ben Lawers, but half of the Perthshire mountain's known colonies have disappeared over the last 40 years.

Accepted polished text:

New research suggests that climate change has contributed to the decline of certain mountain plants in Scotland. Scientists have noted that many of these species depend on snow cover remaining on the hills until late spring and even summer to maintain a moist environment. However, warmer conditions are allowing plants that typically thrive at lower altitudes to spread into mountain habitats.

Species experiencing decline include the snow pearlwort, alpine lady-fern, and alpine speedwell. This research, conducted by the Botanical Society of Britain and Ireland (BSBI), took 20 years to complete and has been published in the new Plant Atlas. The report draws on over three million plant records of 2,555 species collected by hundreds of botanists across Scotland.

Key threats to the health of native plants in Britain and Ireland include climate change, habitat loss, and the spread of non-native species. The BSBI highlighted the devastating loss of species in Scotland as one of the significant findings. Notably, almost the entire British population of snow pearlwort is located on Ben Lawers, yet half of the known colonies on this Perthshire mountain have vanished over the past 40 years.

Accepted rewritten text:

New research suggests that climate change has contributed to the decline of some mountain plants in Scotland. Scientists have identified that many of these species depend on snow cover remaining high on hills until late spring and even summer to maintain a moist environment. Additionally, plants that thrive in warmer conditions at lower altitudes are now encroaching upon mountain habitats.

Species experiencing decline include snow pearlwort, alpine lady-fern, and alpine speedwell. This research, conducted by the Botanical Society of Britain and Ireland (BSBI), spanned 20 years and is detailed in the newly released Plant Atlas. The report's findings are based on over three million plant records of 2,555 species, compiled by hundreds of botanists across Scotland.

Key threats to the health of native British and Irish plants were identified as climate change, habitat loss, and the spread of non-native species. According to BSBI, Scotland has experienced particularly severe losses. For example, almost the entire British population of snow pearlwort resides on Ben Lawers, yet half of the known colonies on this Perthshire mountain have vanished over the past 40 years.

Rejected polished text due to high similarity to the original text:

Climate change has likely contributed to the decline of some of Scotland’s mountain plants, according to new research. Scientists said many of these species rely on snow cover remaining high on hills until late spring and even summer to ensure a moist environment. They also noted that plants thriving at lower elevations in warmer conditions are spreading into mountain habitats.

Species found to be in decline include snow pearlwort, alpine lady-fern, and alpine speedwell. The research, conducted by the Botanical Society of Britain and Ireland (BSBI), took 20 years to complete and has been published in the new Plant Atlas. Data used for the report included more than three million plant records of 2,555 species collected by hundreds of botanists across Scotland.

Climate change, habitat loss, and the spread of non-native species were identified as key threats to the health of native plants in Britain and Ireland. BSBI reported that devastating losses of species in Scotland were among the findings. Almost the entire British population of snow pearlwort is found on Ben Lawers, but half of the Perthshire mountain’s known colonies have disappeared over the last 40 years.

Rejected rewritten text due to low similarity to the original text:

Climate change, habitat loss, and the spread of invasive species are contributing significantly to the decline of mountain plant species in Scotland. Among those affected are snow pearlwort, alpine lady-fern, and alpine speedwell—plants that are specially adapted to cold, moist conditions often maintained by prolonged snow cover in high-altitude environments. These species rely on snow persisting late into spring or even summer to provide the cool and damp conditions necessary for survival.

A comprehensive 20-year study conducted by the Botanical Society of Britain and Ireland (BSBI) has brought these issues to light. The research, recently published in the Plant Atlas, draws from over three million plant records covering 2,555 species, gathered by hundreds of botanists throughout Scotland. The findings show a troubling pattern: as temperatures rise and snow cover diminishes, many alpine species are losing their habitats. At the same time, plant species that typically thrive in warmer, lower elevations are expanding into mountain areas, increasing competition for native mountain flora.

One of the most striking examples of this decline is seen on Ben Lawers in Perthshire, which holds nearly the entire British population of snow pearlwort. According to the BSBI’s findings, half of the known colonies of this delicate plant have disappeared over the past four decades, underscoring the urgency of addressing environmental changes that threaten Scotland’s unique mountain ecosystems.

Benchmark

Table 1 shows the prediction rate for each class on a 10k sample benchmark containing examples from the MixSet dataset, an academic mixed document benchmark, as well as test cases from our polished and rewritten datasets. These results show that our new model release classifies documents that are lightly edited by AI separately, instead of classifying them as completely AI generated like most AI detectors. 

Table 1: Prediction rate on documents edited by AI for the human, AI, and mixed classes. Here, documents edited by AI are under the mixed class. 

This update has the following impact on our model outputs:

1. Users who use AI in the writing process may see an increase in mixed scan results. We expect 4.4% of user submissions to be classified as mixed, with the “Lightly edited with AI” tag. 
2. We’ve improved our performance on mixed documents on academic benchmarks and user submissions. 

This update adds a new class to our detector that better reflects the writing process, and allows educators to make more informed decisions about AI use in their classrooms. By pulling back the curtain on the writing process, GPTZero empowers educators to make more reasonable decisions about academic integrity and become more aware of their students’ learning journey.