[ad_1]
A MEDIOCRE EFFORT
OpenAI – the creator of ChatGPT – launched a “classifier for indicating AI-written text” in late January.
The classifier was trained on external AIs as well as the company’s own text-generating engines. In theory, this means it should be able to flag essays generated by BLOOM AI or similar, not just those created by ChatGPT.
We give this classifier a C- grade at best. OpenAI admits it accurately identifies only 26 per cent of AI-generated text (true positive) while incorrectly labelling human prose as AI-generated 9 per cent of the time (false positive).
OpenAI has not shared its research on the rate at which AI-generated text is incorrectly labelled as human-generated text (false negative).
A PROMISING CONTENDER
A more promising contender is a classifier created by a Princeton University student during his Christmas break.
Edward Tian, a computer science major minoring in journalism, released the first version of GPTZero in January.
This app identifies AI authorship based on two factors: Perplexity and burstiness. Perplexity measures how complex a text is, while burstiness compares the variation between sentences. The lower the values for these two factors, the more likely it is that a text was produced by an AI.
We pitted this modest David against the goliath of ChatGPT.
First, we prompted ChatGPT to generate a short essay about justice. Next, we copied the article – unchanged – into GPTZero. Tian’s tool correctly determined that the text was likely to have been written entirely by an AI because its average perplexity and burstiness scores were very low.
[ad_2]
Source link