The Problem with AI Detectors
Writers write. From journalers to journalists, it’s an innate passion for many of us, some of whom may not even call themselves writers — but if you write on the reg, then you are. Some of us are lucky enough to spin that skill into a career. And these days, that can often look like pouring your love of language into writing content. Ideally, content you care about.
Over the last few years, we’ve all witnessed a massive shift as AI completely changed the landscape of, well… everything. But for those working in communications specifically, suddenly a research deep dive, fresh story angles, and content could all be produced in seconds. With the time-consuming tasks managed by our robot assistant, we could have all the fun refining, tweaking, and adding those creative flairs that make it all our own. Thanks, robot!
Not only was AI a game-changer for writers, but it also levelled the playing field for those who don’t normally touch communications. For business owners and self-promoters, managing their own marketing suddenly became a million times easier and more affordable. With the right prompts, you could quickly churn out everything you need according to best practice and automate your content strategy tout suite.
AI Slop vs AI Slip
Of course, new frontiers always attract those who want to enjoy the benefit of a job done now, not a job done well. These individuals don’t bother providing AI with specific questions or meaningful prompts. They forget to fact-check (and we all know how AI likes to flex its imagination!) And most importantly, they don’t refine the work to make it their own, ensuring it has an authentic voice with something relevant to say.
As a result, generic content floods our devices, as AI feasts on a feedback loop of content written for algorithms and not for humans. Garbage in, garbage out. Or what we now lovingly know as AI slop. (Speaking of which, if you’ve been leaning too heavily on generic slop and start losing your uniquely authentic edge, AI might begin skipping over your business in search results. More on that in the Invisible Brand Problem.)
Meanwhile, on the flip side, AI has also evolved in the other direction, becoming virtually indistinguishable from human prose. The models have been trained to write like humans, so we shouldn’t be surprised when they do produce natural and beautifully written content. In a flash, a page of thoughts and jotting can be transformed into a gripping draft, in the style and voice of your choosing. You might want to call this AI slip, as this kind of AI magic can slip by unnoticed.
Unlike our disdain for AI Slop, AI Slip brews a different breed of concern. When it comes to companies, academia and much of the public, we can feel duped when we’ve spent our time reading something purely written by AI. Naturally, we’d all prefer to read the experiences and musings of a human vs. the recited repertoire of a robot. So, the consensus seems to be that if the words and thoughts are not your own, then, much like plagiarism, you shouldn’t be taking credit for them.
How Fear Fuels a False Economy
This fear of being duped has created a burgeoning industry of AI detection software. These programs spit out a percentage based on how much of that article is likely to have been written by AI. The rush to call out AI has become so intense that LinkedIn is even introducing a new button to report content users believe is AI Slop. An interesting initiative indeed.
But how can you be so sure?
How can any of us?
Andy Betts had the same questions around the accuracy of AI detectors in his article, “The AI Detection False Economy: Fueling FOW (Fear of Writing)”
Witnessing the rising demand for AI detection software, Betts ran a few experiments with a variety of AI detection programs, which included scanning articles he had written before AI was a thing, just to see how they would measure up.
The results? The scores were all over the place. None of the software platforms could agree on whether something was written by a human or not.
The only thing they did prove with absolute certainty is that you can’t trust the software.
As Bett points out, “People panic-buy products based on emotion – because they suddenly feel essential.
“The cost is jobs. Writers are losing work because nervous clients run scans and treat the numbers as gospel, turning an unreliable technology into a confidence crisis for the content industry….
Fear creates demand. Demand creates products. There’s the false economy….”
Am I writing like AI or is AI writing like me?
Curious about what types of content and formatting might trigger a detection software, I asked ChatGPT for some telltale signs that something has been written by AI.
Even though I’m a human writer, I found that I’m guilty of its top three stylistic faux pas:
- The em-dash — which I’ve used for years and now actually fear using because it’s become synonymous with AI.
- Not only X, but Y – I didn’t realize “not only, but also” was an AI issue! Yes, I do this too. Not only am I learning something new here, but I’m also owning it.
- Three-part lists – Easy three-to-five-part lists have always made sense to the marketer in me, but I guess it’s too clean, so it must be AI.
So, the question is, am I writing like AI, or has AI learned to write like me?
This is where the writers’ fear comes in…
On one hand, you’re competing with a flood of fresh content written at super speeds, which creates a sense of job scarcity. Godspeed to those junior writers just getting into the game.
On the other hand, even if you invest hours writing something meaningful and worthwhile, or heaven forbid, give your work a pass-through AI for a polish, you still run the chance of being called out as a robot and facing potential repercussions.
May you live in interesting times….
Can’t we all just get along for the common good – namely, good writing?
Like any technological advancement, there should still be a place for AI in any marketer or writer’s toolbox. Using the right prompts is invaluable for creating efficiency, inspiring new food for thought, and unlocking your potential.
Instead of a war against AI-written content, perhaps we should pivot into a battle against poorly written content – machine and human alike — because at this point, with all the tools at our disposal, there really is no excuse!
Afterall, if a piece of writing offers something of value, a voice, or a virtue that can be appreciated by its readers, who cares if a machine gave it a nudge?
How an AI Detector Rated this Human-Written Blog
I wrote the balance of this blog from start to finish without the use of AI, and then ran it through an AI detection software for science. The software scanned it for AI-generated text, checked for unnatural patterns, analyzed my writing style, and reviewed the blog for consistency.
The results for my human written blog:
70% written by AI
30% human.I then asked ChatGPT to polish it up to sound more human and tried the software again.
Results for Chat GPT polished blog:
67% written by AI
33% human.
Nice try, robot.
BTW, even though I usually like to give a little AI polish to my posts, this time I’m leaving the 100% human version. Could you tell the difference? The detector can’t.