MEMON SYSTEMS
Back to Journal

# r a g

Journal entries tagged rag.

Oct 19, 2026

In-Flight Rollback on SGLang: A Sentence-Level Check Cut Wrong Legal Answers From Memory by 83 to 86%

legal-rag-verifier checks each sentence while the model writes it, cuts a rejected one and rewrites it with the statutory text added. 240 UK statute questions answered from memory, 14 October 2026: answers holding a wrong sentence fell from 132 to 23 for Qwen 27B and from 198 to 28 for Qwen 2.5 7B. After a real error, the default retry was wrong again 71 times in 95, and a retry with the provision added at the cut 16 times. With the source already in the prompt, Qwen 27B had 4 such answers with the loop and 4 without, against 3 for Gemini 3.8 Flash and 8 for Claude Opus 4.6.

legal-techraghallucinationsglangkv-cachebenchmarkopen-source
Read Article
Oct 17, 2026

What Failed in My Own Controls: Seven Results From October 2026

Seven failures in my own legal AI controls, five measured against a rule set before the run and two found in review, each with its date, configuration and evidence: an auditor whose rejections were mostly wrong, a model-hiding scheme recovered 100%, routing that failed its pre-registered rule twice, two missed latency targets, a calculator fallback and a wrong statutory instrument on my own case study.

legal-techragevaluationfailure-analysisopen-source
Read Article