Meet Your New Study Sidekick: GPTZero Essay Grader

We’re thrilled to release Essay Grader, which helps you get instant and accurate feedback on your writing. Think of it as your new study sidekick to improve your writing in standardized exams: 

• Best-in-class performance: GPTZero’s essay grader delivers state-of-the-art performance in accuracy and feedback, for standardized writing exams - surpassing ChatGPT by 119.1% and closely matching expert human graders.
• Proven writing improvement: Essays revised using GPTZero’s feedback show a 13% greater score increase.
• Free to try out: Get started for free with built-in presets for major English-language exams, including IELTS, TOEFL, GRE, and more.

Why We Built Essay Grader

Standardized tests are an inevitable part of various application processes – whether it’s for college, grad school, or immigration. Many of these exams are handwritten (and naturally resistant to AI-generated cheating). At GPTZero, our mission goes beyond providing transparency regarding the origin of a text – we aim also to increase the quality of our user’s writing and thus the quality of writing found on the web.

Practicing for standardized tests is a golden opportunity to refine writing skills that will last a lifetime – but getting personalized, accurate feedback has always been the hard part. Having a professional evaluate every essay is costly, time-consuming, and often inconsistent due to human subjectivity, let alone the additional effort required to provide meaningful feedback.

With GPTZero’s AI-powered essay grader, instant feedback and accurate scoring becomes available - you can score your essay based on rubrics from a selection of standard tests (TOEFL, GRE, ACT, IELTS, etc.) and get accurate grades for each criteria with actionable feedback. You can also check out our writing feedback tool, where you can get grammar suggestions and sentence improvement tips on various categories, to improve your everyday writing beyond standardized tests.

Getting Started

• Open the tool: From the main dashboard, select Essay Grader from the scan menu– or or click it from the hidden toolbar on the right at any time.

• Choose your test: Use the dropdown to pick the exam you’re prepping for. The grading rubric will adjust automatically.

• Sign up and get going: Log in or sign up – and once you’ve done that, you can start grading in seconds.

How It Works

Our essay grader analyzes each piece of writing with their writing prompt and assesses each criterion according to the official rubrics. Our model carefully compares the writing against all scoring bands, and calibrates with historically graded essays to ensure prediction accuracy.

We finetune the model with human annotated ground truth samples and adopt ongoing calibration when needed. We are continuously developing and improving our model to enhance its ability to align with grading standards from official institutions, better understand diverse writing styles, and provide more personalized and actionable feedback to refine your writing.

How Accurate Is Our Grader?

We prioritize accuracy and reliability in our essay grading model to make sure it aligns closely with human evaluators. We track performance with metrics like:

0.5 Band Match Rate: this measures the percentage of essays where our prediction falls within the ±0.5 of the actual score. A high match rate indicates higher precision in closely replicating human experts grading.

1/1.5 band match rate: expands the range to ±1/1.5, accounting for the intrinsic subjectivity in human grading - even official standardized test graders often disagree, resulting in final scores determined by averaging multiple scores. A strong performance of this metric supports our model’s capability to remain well-calibrated to human scoring variability.

Mean Absolute Error (MAE): this represents the average absolute deviation between the predicted score and actual score, providing an overall measure of how much the predictions differ from ground truth. A lower MAE score indicates more accurate estimation. 

Mean Squared Error (MSE): this measures the squared difference between prediction and ground truth, with a greater penalty on larger errors. Combined with the MAE score, this indicates how bad the worse cases are, and how well the model avoids extreme mispredictions.

We evaluate our model on public IELTS scoring datasets, which contains 402 unique writing questions and 1274 unique writing essays. Each essay is scored based on the standard IELTS evaluation criteria (Task Response, Coherence and Cohesion, Lexical Resource, and Range Accuracy) and an overall score is given as ground truth. The score ranges from 0 - 9. The results on a held-out validation set of 200 essays are listed below:

[1,2] are from the kaggle competition

Note that grading inherently involves a degree of subjectivity, which sets an upper limit on achievable accuracy. As a quantitative reference, we analyzed the public ASAP dataset (score range: 0 - 6), by treating one rater’s score as the ground truth and evaluating the other rater’s score as the prediction. The results are as follows:

Following the same protocols, we also evaluate on the Ellipse dataset with 200 graded samples, exclusively for research and evaluation purposes to assess our model’s performance. The results are as follows:

Compared with the baseline, our model improves the exact matching by 53% and 157% accordingly. 

Additionally, we provide an analysis of the error distribution on IELTS validation set:

The maximum error remains within 2 points (on a 12 point scale) indicating strong consistency of our model.

How the Feedback Helps

GPTZero Essay Grader goes beyond simply providing a score - we personalize actionable feedback tailored to the rubric to help you understand what exactly you can improve. Instead of generic comments, our essay grader highlights specific strengths and weaknesses for each criteria. For example, for the following essay:

Nowadays, due to the development in science and technology, expolring the remote natural places has become available. Exploring is what scientists and tourists are always like and eager to do because, invariably, unknown keeps an attraction of mystery. However, there are pros and cons in travelling to remote natural environment.

As far as we know, the improvement in science and progress in technology always comes from exploring the unknown destinations. Travelling to a remote natural environment, take the South Pole for example, can conduct many research which cannot be done in common places. These kinds of research is helpful in Biology and in finding the origin of human being. The second biggest advantage is that it can improve the tourism in its local area, and tourism means a better condition for local people. In the past, in some remote places of China without torism, the native people there had a poor sanitory condition and after the arrivals of travellers, people there can earn a living and improve their living conditions.

However, everything in the world has its donwside. The activities in scientific research and commerce will cause contalmination to the environment in remote places. For instance, the constructions of the scientific stations have occupied the places where many wild animals there used to call home. There is no doubt that such activities will lead to a decrease in the number of these animals which will also influent the local ecosystem.

To sum up, the travel to remote natural environment will not only help to improve the development of science but also help with the local tourism, but the disadvantage is that these activities will cause damages to the local environment.

We get the following feedback:

Task Response (score: 7) To improve your essay, focus on providing more precise and detailed examples to support your points. Avoid over-generalizations and ensure that each argument is clearly linked to the main position. Additionally, work on refining your language to enhance clarity and coherence.

Coherence and Cohesion (score: 6) To improve your writing, focus on enhancing the logical flow between sentences and paragraphs. Ensure that cohesive devices are used accurately and effectively to link ideas smoothly. Additionally, work on refining your paragraph structure to clearly delineate different points or arguments.

Lexical Resources (score: 6) To improve, focus on expanding your vocabulary range and ensuring precision in word choice. Pay attention to spelling and word formation to enhance clarity and avoid errors that could distract the reader.

Grammatical Range and Accuracy (score: 5) To improve your writing, focus on enhancing the accuracy of complex sentence structures and ensuring subject-verb agreement. Pay attention to punctuation, particularly in complex sentences, to improve clarity. Additionally, review spelling and word choice to avoid errors that can distract the reader.

Overall: Your essay effectively addresses the main parts of the prompt and presents a clear position, which is commendable. However, there is room for improvement in providing precise examples, enhancing logical flow, and refining language to improve clarity and coherence. Keep working on these areas, and with practice, your writing will become more polished and impactful

To demonstrate the effectiveness of the feedback, we fed both the original essay and our generated feedback to Gemini-1.5-pro, instructing it to revise the content (up to 30%) solely based on the feedback - without access to the grading criteria. The LLM has improved the essay’s quality by 12.6%, demonstrating how our feedback gives feasible direction enhancing writing clarity, structure and coherence. 

The graph below shows the average score improvement on 20 essay samples.

This personalized guidance makes GPTZero’s essay grader an effective learning tool, empowering you to refine your writing and achieve higher scores. What used to be an expensive, and time-consuming process (waiting for tutor availability), is now inexpensive and accessible, available on demand. By providing this feature to our users, GPTZero contributes to an increase in text quality online as an indirect result of helping their learning process.