PRINT AS PDF

With the advent of ChatGPT, there is growing demand for a resource to differentiate between human- and AI-generated written text. One new product is GPTZero. So, I tried it out.

My sample:

The day dawned cold,
Thick gray clouds quickly shrouded the sun,
As time passed, they lowered,
A biting northeast wind added to the chill.

Suddenly, the air was filled,
Countless swirling snowflakes danced toward the frozen earth,
They hid the familiar landmarks from view,
They knitted an ever thicker blanket of white that covered everything,
The blizzard had arrived.

From ChatGPT:

The blizzard comes with frosty swirls,
A blanket of white upon the world,
It muffles the sounds of day and night,
And transforms the landscapes in its sight.

With every flake that falls and flies,
A tale of nature’s grand disguise,
It brings us peace and wondrous dreams,
And ushers in a world of gleams.

GPTZero’s verdict: Both texts were likely to “written entirely by a human.”

My sample:

January was unseasonably warm. Day after day, the Arctic blasts did not come. Day after day, the mercury rose to balmy levels. The thermometer rarely reached freezing. On one day, the mercury crested at a springlike 66°. Flowers began bursting into bloom–crocuses, witch-hazel, even Japanese Apricot. Never had there been a warmer January in the New York City area.

For my paragraph, GPTZero concluded, “Your text is most likely human written but there are some sentences with low perplexities.” The offending text? “January was unseasonably warm.”

ChatGPT’s paragraph:

The city of New York experienced a record warm January this year, with temperatures reaching well above average for this time of year. This unexpected weather pattern brought relief to many New Yorkers, who were able to enjoy outdoor activities such as picnics, walks in the park, and other forms of recreation. The warm weather also brought an influx of tourists to the city, taking advantage of the rare opportunity to experience mild temperatures in the heart of winter. Despite the pleasant weather, experts warn that the warm spell is likely due to the larger trend of climate change and that we should expect to see more frequent and intense fluctuations in temperature in the future.

GPTZero correctly recognized the above paragraph as having been generated by an AI.

My sample:

As long as history has been recorded, the story of ever advancing technology has been told. From printing to automobiles to airplanes to rockets to artificial intelligence, technology has moved into areas that previous generations might never have imagined, much less dared to utilize. In many cases, elements of aroused by a world moving beyond a familiar even comfortable status quo were alarmed. Today, the emergence of AI has sparked great alarm among some college campuses. A hasty rush to ban ChatGPT and other text engines is underway. Yet, imagine if the world had barred calculators upon their appearance, computers when they reached the mass market, or the Internet when its reach proliferated globally. AI, like its technological ancestors, should be embraced where it adds value with limits placed where it poses genuine risks. If past advances of technology are representative–and there is no reason to believe things will be different this time–its benefits will greatly outweigh its risks. As a result, the world will become better for its arrival.

ChatGPT’s paragraph:

The advancement of technology has brought about tremendous changes in the way we live and work. From the invention of the wheel to the development of artificial intelligence, technology has continually evolved to make our lives easier and more efficient. In the past few decades, the rise of the internet and mobile devices has connected people from all over the world, allowing for instant communication and access to vast amounts of information. With technological innovations such as self-driving cars, virtual reality, and 3D printing, the possibilities for what technology can do are almost endless. While there are also concerns about the impact of technology on society, its advancements have undoubtedly had a profound effect on the world, and will continue to shape the future in countless ways.

This time, GPTZero correctly distinguished between the human and AI samples.

Based on the above, GPTZero has demonstrated some success. However, its false negative for one of ChatGPT’s paragraphs and its flagging a generic sentence suggests more work is required to build the kind of reliability that is necessary for its use in the mass market.

This entry is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.