Is Originality.ai (or any AI detector) good enough to screen freelance writers before paying them?
No, not as a payment gate. The vendor itself says detection scores alone should not be the basis for disciplinary decisions, and independent testing finds accuracy falls sharply once text is edited. Use it, if at all, as Ingrid describes: a prompt for an editor to look sooner. Put the real quality bar in the brief and the review.
Cite
Global 100 Forum, "Is Originality.ai (or any AI detector) good enough to screen freelance writers before paying them?", https://global100.org/forum/q/is-originalityai-or-any-ai-detector-good-enough-to-screen-freelance/, accessed 2026-11-10.Side project question rather than a day job one. A friend and I run a small niche content site. We commission around forty articles a month from a pool of freelance writers, most of whom we found through a marketplace and have never met.
My co-founder wants to add a gate: every submission goes through Originality.ai, and we only approve payment if the score comes back under a threshold he has picked. His argument is that the vendor publishes accuracy figures, the false positive rate is low, and it costs us nothing to check.
I am the ML person in this partnership and I am uneasy, but I want to be fair to the proposal rather than wave it away. My concerns, in order: a low false positive rate on the vendor's own test set is not the rate on our writers, several of whom are not native English speakers; even a genuinely low rate applied to every article every month will eventually flag someone who did nothing wrong, and that someone will have no way to argue with a number; and our brief explicitly allows writers to use AI for outlining and research, which the tool cannot distinguish from drafting.
What do people who actually run content teams do here? Is there a version of this that is defensible, or is the premise wrong?
Write something first.
Give people something to work with: at least 30 words on what happened and what you tried.
That is too long. Keep it under 6,000 characters.
Write the question as the title, 15 to 140 characters, no links.
Pick a category.
Add a name (2 to 40 characters, no links).
That email address does not look right.
That was quick. Read the thread, then try again.
The form expired. Reload the page and post again.
Something went wrong with the form. Reload and try again.
Please complete the check and post again.
Limit reached for now. Try again later.
This thread is closed to new replies.
This thread no longer accepts replies.
Something went wrong with the form. Reload and try again.
Post a reply or question first (name and email), then this browser can vote, edit and accept answers.
You cannot vote on your own post.
Only the author (within 30 days) or the forum team can do that.
That email already has an account. Use your sign-in link instead.
New accounts are paused for the moment. Try again later.
That was already posted.
9 answers
- Accepted answerMaya LindqvistStaffEdits the AI text detection and academic integrity sections ·
No, not as a payment gate. The vendor itself says detection scores alone should not be the basis for disciplinary decisions, and independent testing finds accuracy falls sharply once text is edited. Use it, if at all, as Ingrid describes: a prompt for an editor to look sooner. Put the real quality bar in the brief and the review.
Start with the vendor's own words, because they end the argument with your co-founder faster than anything independent. Originality.ai's accuracy page lists its model history with the false positive rate it reports for each release, from well under one percent for some models to around three percent for others, and then states the company's position: "We do not believe that AI detection scores alone should be used for academic honesty purposes and disciplinary action. The rate of false positives (even if low) is still too high to be relied upon for disciplinary action." Withholding a writer's pay is a disciplinary action.
Your second worry, that a low vendor rate is not your rate, is the one the research supports most clearly. Perkins and colleagues (2024) tested six major detectors on 805 samples and report an average accuracy of 39.5% on AI text, with a further fall of 17.4 points once the text had been altered using simple evasion techniques. So a writer who drafted with a model and edited the result looks, to the detector, much like one who did not, while the honest non-native writer in your pool is the one most likely to score high, as the bias thread sets out.
Your third worry settles the design. Your brief permits AI for outlining and research. No detector can tell permitted use from forbidden use; it sees text. What you actually want to enforce, "the prose is yours and the facts are checked", can be measured by an editor reading the piece, checking two sources, and, for new writers, a fifteen-minute conversation about the draft. The detector comparison thread has what each tool claims against what independent tests found.
- Lena B.Member ·
The premise is wrong. I am the writer on the other end of this gate. I have walked from two clients who installed one, and I was the writer they most wanted to keep. Both times.
Here is what a payment gate tells a freelancer: your word counts for less than a tool you cannot see, and your pay depends on an argument you cannot win. Good writers have options. They leave. The ones who stay are the ones who have learned to write for the detector, which is exactly the writing you did not want to pay for.
- Ingrid V.Member ·
What we do at scale, since you asked for practice rather than opinion. A detector score is allowed to exist in our workflow in exactly one role: as a triage signal that routes a piece to a human editor sooner. It is never a decision, never visible to the writer as a verdict, and never attached to payment. The editor's review is the decision, and it is the same review every piece would get eventually; the signal only changes the order.
Two other things that matter more than the tool. The brief states what AI use is permitted and asks for a one-line disclosure, so the writer is never guessing at the rule. And the policy is published to writers before they accept the first job, including the sentence that payment for delivered work is not conditional on any automated score. That sentence is what keeps the good ones.
- Priya N.Member ·
The teaching parallel is close. Our department moved, over two years, from "score above a line means a meeting" to "every submission gets the same five-minute conversation about the argument", and the second filter caught more than the first while flagging nobody wrongly. A writer who drafted the piece can talk about why paragraph three comes before paragraph four. A writer who pasted it in cannot. The conversation is also the point at which you find out who is good, which the score never tells you.
- Tobias K.Member ·
If your co-founder prevails and the tool is used in any capacity, write the standard down before the first case rather than after it. What score triggers what action; who reviews; what the writer may submit in response, such as drafts or version history; the time within which the review is completed; and the sentence Ingrid quoted, that payment for delivered work is not withheld pending a score. An unwritten standard is applied most strictly to whichever writer happens to be disliked that week, and most leniently to whichever one is needed for Friday's deadline. The written version is a protection for you as much as for them.
- Hana SatoMember ·
@Maya Lindqvist thank you. The vendor's own sentence is the one I will put in front of my co-founder, since it is their product and their words. We will do editor triage with the score hidden from writers, the disclosure field Jules suggests, and a brief that says the quiet part out loud about payment. The threshold idea is withdrawn.
- JulesMember ·
honestly? the writers who use the tools well are the ones i want anyway. ask for disclosure, not detection. "did you use ai, where, for what" as a required field on the submission form. most people answer it straight, and the ones who lie about that will lie about their sources too, which you were going to check anyway. the detector catches neither group reliably and annoys both.
- Kwame A.Member ·
A contract point, since payment is the mechanism here. If a score is going to condition payment, it has to be in the engagement terms, in plain language, with the threshold and the tool named. A term that lets you withhold pay for accepted work on the basis of an unexplained third-party number is the kind of clause that reads badly in a small claims hearing, and the writer will have the drafts while you have a screenshot. Separately, for writers in the EU, uploading their submitted work to a third-party service for scoring is a processing activity you would need to be able to explain. Neither point is fatal. Both are reasons the editor's judgement is the cheaper instrument.