Authenticity in the Application Process — Case Study with Coding it Forward

At GPTZero, we strive to empower organizations with the tools to respond to AI. In this case-study, we explore how the GPTZero detection model aided Coding it Forward in ensuring the integrity of their 2023 application process. Let's dive into the details.

Several components of our application process require a personal touch — elaborations on past experiences, future goals, and detailed thought processes. AI-generated responses to these questions lacked nuance and prevented our reviewers from learning more about a candidate. - Coding it Forward

Identifying a Challenge: Coding it Forward's team anticipated new challenges when launching their assessment process. However, they encountered an unexpected hurdle: the emergence of ChatGPT. Upon discovering that some essays contained generic phrases and sentence structures lacking personal anecdotes, they suspected that applicants were using generative AI.

Validating Intuition: To validate their suspicions, Coding it Forward employed an open-source GPT-2 detector by HuggingFace, which flagged several essays as "99.99% Fake." This confirmation prompted the team to address the issue promptly. They recognized the importance of ensuring that the essays represented candidates honestly and showcased their personal qualities, which are vital for success in public interest technology.

Seeking Enhanced Accuracy: Coding it Forward initially attempted to use the HuggingFace detector, but its accuracy fell short since it was trained on the previous GPT-2 corpus rather than the more advanced GPT-3.5 used in ChatGPT. After evaluating the GPTZero, they were able .

The GPTZero Advantage: Coding it Forward became one of the first users of the GPTZero API service, which proved instrumental in their quest for accuracy. GPTZero utilizes a proprietary algorithm that identifies words and phrases written in a "random" manner. By assessing the "perplexity" score, GPTZero distinguishes between genuine human responses and those generated by AI. Initial testing yielded promising results, making GPTZero an invaluable tool for Coding it Forward's application review process.

Findings and Implications: Upon systematic investigation of all submissions, Coding it Forward uncovered a notable number of essays with low perplexity scores, indicating the possibly of AI-generated responses. Particularly concerning were the essays addressing personal experiences, future goals, and detailed thought processes. AI-generated content lacked the nuance and personal touch required to assess candidates effectively.

A Commitment to Integrity: Coding it Forward's case study highlights their dedication to upholding the integrity of their application process. By leveraging GPTZero's advanced detection capabilities, they were able to maintain authenticity, provide fair evaluations, and ensure that their fellows were genuinely passionate and equipped with the necessary skills for public interest technology.

Conclusion: The partnership between Coding it Forward and GPTZero exemplifies the power of innovative solutions in addressing emerging challenges. By deploying GPTZero's detection model, Coding it Forward successfully tackled the issue of AI-generated content in their application process, fostering an authentic environment for candidate evaluation. We are proud to have supported Coding it Forward in their mission to identify technical innovators and shape the future of public interest technology.

If your organization is facing similar challenges, contact us to explore how GPTZero can assist you in maintaining integrity, authenticity, and understand the impacts of Artificial Intelligence in your application or evaluation processes.

Remember, authenticity matters, and together, we can build a future where that preserves the value in human writing.