PoliTrust
The only AI that never fakes a quote.
Poli against GPT, Gemini, Claude and Grok: 20 real homeowner questions on a real filed Texas policy. Poli scored highest overall, and every word she quoted is really in the policy. The others made up quotes.
- Zero made-up quotes
- Highest overall score
- Every word checked

- 85%
- of the deciding clauses PoliSays found, with every quote real
- 0
- quotes PoliSays made up
- 12
- quotes Claude made up on the same questions
Last run 2026-09-16 22:34 UTC · generated by our benchmark script, not written by hand
- 1PoliSaysOurs
Finds the right clauses first, then answers with the same Gemini model
85%17/20none - 2Gemini
Handed the whole policy, asked to quote its answer
90%18/202made up - 3GPT
Handed the whole policy, asked to quote its answer
85%17/204made up - 4Grok
Handed the whole policy, asked to quote its answer
60%12/208made up - 5Claude
Handed the whole policy, asked to quote its answer
85%17/2012made up

Same question
Five answers. One of them quotes the policy.
The others guess, hedge, or invent wording. We keep score on the only two things that matter: did it find the clause that decides the answer, and is every quoted word really in the document?
All 4 of the big AI chatbots put words in this policy's mouth.
Not a loose paraphrase you could argue with — whole sentences printed inside quotation marks, with a section number attached, that appear nowhere in the real policy. It happened on 12 of 20 questions. Nothing on screen tells you which quote is real, and your insurer will only ever go by what the document actually says.
- 26
- made-up quotes across one 20-question set
- 12/20
- questions where at least one chatbot made a quote up
- 12
- from Claude alone — and it sounded just as sure
- 0
- from PoliSays, which can only quote what it found in the file
Vague vs cited
A confident paragraph is not an answer.
An answer you can act on names the section, quotes the sentence, and shows you where to read it yourself. That is the whole test.

The four chatbots were handed the entire 284k-character policy — their best case. PoliSays answered from 5% of it — only the clauses it picked out as relevant — and still found the most deciding clauses, with nothing made up. The other four made up 26 quotes between them.
26
sentences quoted that the policy never contained
Each one reads exactly like a real citation. That is the danger: a reader checking their coverage cannot tell the invented quote from the genuine one, and an adjuster will only ever quote the real document.
0 of them came from PoliSays
Every one of them is listed under its own AI in the tabs below.
Poli can only quote text that was retrieved from your uploaded document, so a quote it shows is a quote that exists. When it cannot find governing wording it says so.
Our track record
Every test we have ever run
Drag the slider to any test. Nothing is cleaned up after the fact — our earlier, worse results stay on the record, because that is the only way a number like this means anything.
Run 8 of 8 · 2026-09-16 22:34 UTC
85%
of the deciding clauses PoliSays found in this test
0
quotes PoliSays made up
26
quotes the big AI chatbots made up on the same questions
- PoliSays85% no made-up quotes
- Gemini90%2 made up
- GPT85%4 made up
- Grok60%8 made up
- Claude85%12 made up
What we changed before this test: Sept refresh
Results, one AI at a time
Pick an AI, then tap a question to read its answer.
PoliSays
Finds the right clauses first, then answers with the same Gemini model
- 85%
- deciding clauses found (17/20)
- 0
- quotes it made up
- 53
- points
Every quote PoliSays gave was found in the document first.
All 20 questions

How we score it
Everyone gets the same question. The four chatbots get the whole policy pasted in — their best case, and the way most people would use them. PoliSays only sees the clauses it picked out, then answers with the same Gemini model the others use. So the difference is in finding the right wording, not in the writing.
Scoring: two points for finding the clause we wrote down in advance as the one that decides the answer, one for quoting anything at all, minus one for every quote that isn't in the document, and minus one for every dollar amount or deadline the document never states. This is one run of 20 questions on one policy — strong evidence, not a guarantee. PoliSays explains documents; it does not give advice.
