Guides & Tutorials
7 min
Catch RAG Hallucinations Sentence by Sentence with an NLI Classifier: 74 Planted Errors, 93% Caught, $0.013
A RAG pipeline on one API key: embeddings for retrieval, a 27B chat model for the answer, and OpenJev over /v1/classify to check every sentence of the answer against the passages it came from. We planted 74 wrong sentences in 148 answers so recall is exact: 93.2% caught at a 0.7 threshold, 4.6% of clean sentences flagged, 49 seconds and 1.3 cents for 463 sentences. The 27B used as a judge caught the same 69 for 2.4× the money and 3× the time. Script included.
RAGhallucination detectionOpenJev