Tag

Openjev

4 articles tagged with “Openjev”

The Paralon capybara reading a glowing document through a magnifying glass, one line red among green ones
Guides & Tutorials

7 min read

Catch RAG Hallucinations Sentence by Sentence with an NLI Classifier: 74 Planted Errors, 93% Caught, $0.013

A RAG pipeline on one API key: embeddings for retrieval, a 27B chat model for the answer, and OpenJev over /v1/classify to check every sentence of the answer against the passages it came from. We planted 74 wrong sentences in 148 answers so recall is exact: 93.2% caught at a 0.7 threshold, 4.6% of clean sentences flagged, 49 seconds and 1.3 cents for 463 sentences. The 27B used as a judge caught the same 69 for 2.4× the money and 3× the time. Script included.

Read
The Paralon capybara measuring three holographic bar columns with an emerald caliper, red cubes on the bench as the planted errors
Benchmarks & Comparisons

8 min read

OpenJev vs an LLM Judge on 990 Planted Errors: Recall, Calibration, Cost

The same 990 support tickets with 495 deliberately wrong labels, judged two ways: OpenJev over /v1/classify and Qwen 3.8 27B as an LLM judge. Asked one pair at a time OpenJev catches 82.6% of the planted errors; asked to pick among all twelve intents it catches 98.2%, rejects the same 33 mislabelled 'complaint' rows the 27B judge rejected, and its probabilities are calibrated to within 3 points. Every number is exact because we planted the errors.

Read