Defeating AI Detectors

A few popular services claim to be able to identify AI-generated text. I tried a few approaches to defeat these systems, with the goal of obtaining AI-written text that is classified as human. For the most part, I succeeded.

I will begin by describing some attempts at paraphrasing, where we automatically rewrite AI-authored text so that it classifies as human-written. Then, I will describe how I finetuned a language model, Qwen3-4B, to directly produce human-classified responses. A demo for this project can be found on Github, and for some time I will host the model here as well.

Paraphrasing

Ideally, we could use frontier models to produce whatever text we want, and then apply some simple paraphrasing procedure to make it classify as human. To test some approaches, I generated a corpus of a few hundred paragraphs written by popular frontier language models. These paragraphs covered various randomly selected subjects, although they are all the same style of informational paragraphs.

I initially tried to rewrite a few of these paragraphs by hand, but found that my paraphrases were still usually classified as AI-written. I was more successful when I automated the paraphrasing process. I handed Codex an API key for an AI detection service, and told it to move around and rephrase information until it was classified as human. Since AI detectors give granular predictions—sometimes telling you that one or two sentences look human-written while the rest was written by AI—Codex was able to iterate on many of the input paragraphs until the whole thing was classified as human-written. This typically required a few dozen API calls to the AI detector. On the service I was targeting, this API usage costed around $5 per paragraph.

Explore Codex paraphrases (7 samples)

Both versions are AI-written. These are successful rewrites selected by searching against the AI detector's feedback. Colors show its classifications of individual spans; percentages are its reported human fraction.

Classified as: AI Human

Original Classified as AI

Human fraction: 0%

The Universal Software Radio Peripheral (USRP) is a family of software-defined radio devices originally developed by Ettus Research. Rather than dedicating all signal processing to fixed circuitry, a USRP combines radio-frequency hardware, digital converters, and programmable logic with software running on a connected computer or an embedded processor. This allows the same equipment to support different waveforms and communication experiments through changes in code. Models vary in frequency coverage, bandwidth, channel count, and connection type, with options including USB and Ethernet. Many work with GNU Radio, an open-source signal-processing toolkit, through the USRP Hardware Driver (UHD). Researchers, educators, and engineers use USRPs for wireless prototyping, spectrum monitoring, radar experiments, and testing cellular or satellite communication techniques. Transmitting still requires compliance with local spectrum regulations; programmability does not grant permission to use arbitrary frequencies.

Codex rewrite Classified as Human

Human fraction: >99.99%

USRP (Universal Software Radio Peripheral), originally an Ettus Research development, is a family of software-defined radios used in research, engineering and teaching. Uses include monitoring the spectrum, prototyping wireless equipment, experimenting with radar, and testing cellular and satellite communication techniques. Transmitting requires compliance with local spectrum regulations. The hardware combines RF circuitry, digital converters and programmable logic with software, which runs on a host computer or an embedded processor and can be changed to try different waveforms and communication experiments on the same equipment. Frequency coverage, bandwidth, channel count and interface are model-dependent. Interfaces include USB and Ethernet. Many models support GNU Radio through UHD (USRP Hardware Driver).

Universal Software Radio Peripheral · 20 detector checks during the rewrite search.

As you might have noticed, AI detection APIs are incredibly expensive. I actually wonder if this is intentional, since it makes distillation and direct optimization less practical. Either way, API prices are so high that I would need to spend hundreds of dollars to get even a modest dataset of (AI, human) paraphrase pairs. As it is, I spent about $600 on AI detection services during the course of this project.

Since direct optimization against the detection service is expensive, I set out to train a small paraphrasing model that rewrites text without any detection API calls. First, I gathered thousands of documents from human-written sources, and then used both GPT and Claude models to rewrite them. I confirmed that the AI rewrites were often classified as AI, but not always. I then finetuned Qwen3-4B on these (AI, human) pairs to get a small, local paraphrasing model. Note that there are two kinds of "paraphrasing" here: the kind that turns human text into AI text, and the reverse paraphrasing function we are trying to learn.

The results initially seemed promising, but ended up being surprisingly negative. When tested on held-out pairs from my (human, AI) dataset, my new Qwen model did a good job at producing text that the AI detector classified as human. In other words, it was good at "reconstructing" human text from an AI paraphrase of that text.

Explore Qwen reconstructions (7 samples)

Seven randomly selected pairs from the original 16-input evaluation. Inputs are Claude/GPT paraphrases of human text; outputs are from the first Qwen3-4B LoRA. Long passages scroll. Both shown versions are AI-written. Colors show the span classifications; percentages show the reported human fraction.

Classified as: AI Human

The Young Lord, and Other Tales; to which is added Victorine Durocher — paragraphs 371–374

Input paraphrase Classified as AI

Human fraction: 47.19%

Before long, the twelve young girls who were to accompany the Rosiere had gathered in the cottage. All wore white, with blue ribbon sashes fastened beneath the left shoulder, their two ends left to flutter as each wearer pleased. They were among the prettiest maidens in the village, though none could rival Durocher’s daughters. The same number of young men now arrived, wearing the blue ribbon sash that marked the Rosiere’s attendants, and a band of musicians came with them. Soon the villagers crowded the street. After saying a few words to her attendants, the Rosiere withdrew until she should be summoned to lead the procession. Victorine, however, stayed with Caliste, who had seated herself in a corner of the room and watched everything that happened with an expression of haughty disdain. All at once, the band began to play loudly and merrily, the young men opened their banners, the maidens gathered together, and Lisette emerged from the innermost room to welcome the Seigneur de Salency. A moment later, he entered the cottage and approached her, addressing her as the Rosiere and asserting his right to escort her to the church. Lisette bowed as she received his compliments. Before offering him her hand, however, she went to her parents, knelt before them, and asked for their blessing, lowering her head to receive it. “And now I claim the privilege of escorting the loveliest of Rosieres!” declared the Seigneur, helping her rise from her knees and leading her by the hand to the cottage door.

Qwen rewrite Classified as Human

Human fraction: 100%

Soon the twelve young girls to accompany the Rosiere had assembled in the cottage. They were all in white, with blue ribbon sashes tied under the left shoulder, their two ends hanging as they liked. These were among the most beautiful maidens in the village, but none could compare with the daughters of Durocher. So, too, the same number of young men came, with the blue ribbon sash of the Rosiere's attendants, and a band of musicians followed them. Soon the villagers were crowding the street. After saying a few words to her attendants, the Rosiere retired until she might be called to lead the procession. Victorine, however, remained with Caliste, who had seated herself in a corner of the room and watched everything that occurred with an air of lofty contempt. At once the band began to play vigorously and joyfully, the young men opened their banners, the maidens gathered, and Lisette came forth from the innermost room to greet the Seigneur de Salency. Presently he entered the cottage and came to her, calling her the Rosiere and asserting his right to conduct her to the church. Lisette bowed as she received his compliments. But before giving him her hand, she went to her parents, knelt before them, and asked their blessing, bowing her head to receive it. “And now I take the privilege of conducting the most lovely of Rosieres!” cried the Seigneur, helping her up from her knees, and taking her by the hand to the door of the cottage.

Input: GPT-6.1 Sol · Output: first Qwen3-4B LoRA.

These examples were excluded from gradient updates; six were used to monitor validation loss.

However, when I fed this model the encyclopedic AI-written paragraphs on which I had previously tested direct optimization, the outputs were almost always still classified as AI-written.

Explore Qwen encyclopedic rewrites (7 samples)

Seven randomly selected pairs from the 40-paragraph encyclopedic evaluation. Inputs were generated by Claude/GPT; outputs are from the first Qwen3-4B LoRA. Both shown versions are AI-written. Colors show the span classifications; percentages show the reported human fraction.

Classified as: AI Human

Markus Rill

AI-generated input Classified as AI

Human fraction: 0%

Markus Rill, born in 1970, is a German singer-songwriter whose music draws on American folk, country, blues, and roots rock. Singing primarily in English, he pairs a distinctive, gravelly voice with narrative songwriting, placing his work within the broad tradition often called Americana. His recordings include The Devil and the Open Road and The Price of Sin, titles that reflect his affinity for the language and imagery of American roots music. He has performed both as a solo artist and with his band, the Troublemakers, allowing his songs to range from spare acoustic arrangements to a fuller electric sound. Rill’s career illustrates how Americana extends beyond the United States: its storytelling conventions and musical vocabulary also provide a creative home for European musicians writing original material.

Qwen rewrite Classified as AI

Human fraction: 54.79%

Markus Rill (b. 1970) is a German singer-songwriter whose music is influenced by American folk, country, blues, and roots rock. Singing mostly in English, Rill has a distinctive, gravelly voice, and his songs are narrative in nature, placing his work in the broader tradition known as Americana. Rill's recordings include The Devil and the Open Road and The Price of Sin, which reflect his fondness for the language and imagery of American roots music. Rill has performed as a solo artist and with his band, the Troublemakers, with his songs ranging from spare acoustic versions to a fuller electric sound. Rill's career demonstrates how Americana transcends the United States: its storytelling and musical conventions also serve as a source of creative inspiration for European musicians writing their own original material.

Input: GPT-6.1 Sol · Output: first Qwen3-4B LoRA.

After doing a few follow-up experiments, I've concluded that there are actually two levels of "AI-ism" that modern AI detectors look for: style and content. My human→AI→human pipeline keeps the human-arranged content, even in the AI-rewritten version, so my Qwen paraphraser never learned to de-AI-ify content that was directly curated by AI. I tried a few follow-up fixes, such as doing multiple rounds of AI rewriting rather than just one, gathering human-written encyclopedic paragraphs as inputs, and even using frontier LLMs to shuffle information inside human-written paragraphs. None of these approaches led to a reliable AI-to-human paraphraser.

Writing novel answers

My conclusion so far was as follows: if an off-the-shelf language model writes a document, then the AI-isms are baked very deeply into that document. You can directly optimize paraphrases against a live AI-detector to artificially bypass it, but doing so is expensive. To get at the root of the issue, we might actually need text to come directly from a model that doesn't have AI-isms baked in.

To approach this, I trained a language model that answers directly in human-like text. I started by turning my existing corpus of human-written text into an instruction-following dataset. In particular, I used frontier models to write prompts that could plausibly have produced a given human-written text. For example, if a document is a paragraph about butterflies, the imagined prompt might be something like "Write a paragraph (100-150 words) about the evolutionary history of butterflies". I like to call this process "jeopardy". Once I had the dataset, I finetuned Qwen3-4B on it.

This result was much more promising. I used Codex to generate 40 test prompts, and ran samples for each prompt through the AI detector. The resulting text was typically classified as human, with about a 75% success rate. However, the quality of the resulting text from this model was quite bad. Sometimes, the model would repeat a sentence again and again, and other times it would veer completely off topic. While this is a small model, I could tell that it was way worse than the base model.

Refining our answer model with RL

At this point, I had a model which could produce human-classified responses, but its answer quality was quite low. I knew the base model was more capable than this, but of course the base model itself produces highly AI-like text. To improve our human-like model, I started by trying reinforcement learning.

First, I used frontier models to produce a dataset of 256 diverse, imagined user prompts. Then I did RL, starting with the human-text finetuned checkpoint.

In this RL loop, we take one update per prompt in the dataset. Given the prompt, we produce 16 model responses, and rank them using a frontier language model. Somewhat arbitrarily, I decided to assign 0 reward to the bottom 14 responses, while the top 2 responses are run directly through an AI detector to receive a reward proportional to the human-written probability. The resulting rewards are normalized, and a policy gradient estimator is used to take an optimizer step. I also apply a KL penalty between the current policy and the initial model that we started RL with (i.e. the model that was finetuned on human text). In this way, we reward high-quality answers while preventing the model from reverting to non-human answers.

The RL process makes very few AI detector calls, which brings the cost down to $10-$20 for a run, while also ensuring we don't overfit too much to the specific AI detector. In my experiments, the human score tends to stay around the same place, so the model doesn't seem to be overly optimizing against it.

How do we know if the RL process actually improves responses? Well, at the risk of Goodharting, I evaluated the resulting model against the same response ranking procedure that I used during training, but on held-out prompts. To do this, I generated five samples for each model in the comparison and for each of my 40 evaluation prompts, and had a frontier model rank them. Then, I computed the pairwise win-rate across models to arrive at an Elo-style score. The Qwen base model still wins 98% of the time against our RL checkpoint, but the RL checkpoint wins 80% of the time against the pre-RL checkpoint. This suggests that RL is improving response quality, although not yet recovering the performance of the base model.

Mixing the base and human models

We currently have two ends of a spectrum: a smart model that produces AI-classified responses, and a dumb model that produces human-classified responses. It would be great if we could get the best of both models, or at least interpolate between the two.

I tried two ways to interpolate: interpolation in parameter space, and interpolation in token probability space. The latter seems to perform better on the Pareto frontier of Elo score and AI detection score.

Quality versus mean human score for parameter interpolation and probability interpolation of the finetuned and RL models with the base model. Probability interpolation achieves better quality at similar human scores; adding 10% base to the RL model reaches about 1044 Elo and 55% human score.
Joint comparison on 40 prompts: five answers per model for quality rankings, one for the AI detector. Human score is the mean human fraction. Probability interpolation uses weighted logits with weights summing to one. Higher and farther right is better; bars show 95% prompt-bootstrap intervals, and the dotted line is the observed Pareto frontier. Click to enlarge.

I tested a 10% interpolation fraction with the finetuned model on some of my own original test prompts. The success rates reported above seem to match my experience pretty closely. I often get human-classified responses, but the response quality varies a lot and isn't very great. Here are some examples:

Explore model responses (5 samples)

Here are a few responses I sampled from the model. Colors show the AI detector's classifications of the output; percentages show its reported human fraction.

Classified as: AI Human

Prompt

Write a paragraph explaining how squirrels survive during winter.

Model response Classified as Human

Human fraction: 100%

Squirrels survive the harsh cold of winter by curling up within the warmth of underbrush or in tree cavities, or building nests out of utility poles. They do not successfully "hunker down" for the cold, rather they sleep through it peacefully and are unaware of its full extent. Most squirrels need a warm place to sleep in the winter, and most of them choose underneath brush or in tree holes. Many are cold-blooded, and do not grow extra fat to keep warm, but it can help to maintain a body temperature against the cold winds, if it is not possible to find an adequate place to hide. In fact, some of the most popular types of squirrels are the Northern Flying Squirrels, which are easy to recognize because of the tufted ears they have. These squirrels roll into a ball to keep warm in the winter months.

Conclusion

I have presented two effective approaches for bypassing AI detectors. If you want one-time results and can spare a few dollars, your best bet is to use a frontier language model to optimize paraphrases directly against an AI detector. If you want to be able to produce a lot of human-like text over time, you can directly finetune a model to respond with human-style responses. However, I found that maintaining response quality can be difficult with this approach.

There are a few potential follow-up directions that could lead to better results. For a better AI-to-human paraphraser, maybe we could hire human writers to produce a dataset of (AI, human) pairs that actually change the information content of the AI paragraphs. Alternatively, if we want a better human-style question-answering model, we could work on a better human-written dataset of responses to diverse questions. The latter approach seems likely to immediately boost the sample quality of my Qwen3-4B model, with minimal extra effort.

I also suspect that pure sampling-based approaches could push us further. For example, perhaps some kind of MCMC procedure could allow us to better direct the base model using the human-style finetune.

Overall, this was a fun little weekend side project. The goal was to see what the frontier of AI detection "feels like", and how well it seems to work. I'm actually pretty impressed: I didn't expect these systems to be so difficult to defeat. I also didn't realize how expensive these AI detectors would be to use; my budget for this project ended up being an order of magnitude higher than I expected.

Finally, I'd like to a give shoutout to the tools I used for this project. I didn't write any code; Codex handled it all. For all sampling and training, I exclusively used my Mac Studio. I was really impressed that it was possible to do all of this in a few days on a single personal computer.