This study pairs black-box probes of a live legal-RAG interface with a control and its reference implementation. §3 gives the three failed probes verbatim. §5 specifies a router that binds the statute before vector search. §5.5 records the implementation built for UK legislation and what it measured. §8 gives the argument that a boolean filter cannot return a chunk from an unbound statute.

---

1. Summary

In Spanish law one article number exists in many statutes. Artículo 42 is corporate group control in the Código de Comercio, solidary tax liability in the Ley General Tributaria, and subcontracting in the Estatuto de los Trabajadores.

A legal-RAG platform that markets a sub-1% hallucination rate returned the wrong statute on 3 of 3 article-collision probes, each time quoting the wrong text with full confidence. (Measured, audited system.)

The model was never given the right evidence. The two retrieval steps, dense (embedding) search and a ColBERT reranker, both score the colliding articles almost equally, so the retrieved set mixes statutes. The agent then loops for 30 seconds to 5 minutes and answers from the wrong one.

Under SRA Principles 2 and 7, the solicitor who relies on the answer carries the liability, wherever the error began. Under the EU AI Act, Articles 14 and 15, retrieval accuracy is part of the system's compliance case.

The control is a router that binds the statute before vector search and a boolean filter that restricts retrieval to it. A reference implementation exists for UK legislation (§5.5).

---

2. Why Vector Search Fails on Shared Article Numbers

2.1 Collision Space

Civil-law statutes number their articles independently, so numbers repeat across codes:

A query such as "¿Qué establece el Artículo 42?" ("What does Article 42 establish?") has to be resolved to one statute before the vector database is queried. A standard RAG pipeline does not do this.

2.2 Three Points of Failure

The audited pipeline is a standard retrieve-then-generate chain:

flowchart LR
    A["User Query: 'Art. 42 CdC'"] --> B["Embed Query"]
    B --> C["HNSW Dense Search"]
    C --> D["ColBERT Rerank"]
    D --> E["LLM Agent"]
    E --> F["Response"]

The failures in §3 are measured. The account below of why they happen is an argument.

Failure Point 1: Dense Search Returns All Three Statutes

A dense embedding model turns the whole query into one vector. For a query containing "Artículo 42", that vector sits close to every chunk that mentions "Artículo 42":

$$\mathbf{q} = \text{Embed}(\text{``Artículo 42 del Código de Comercio''})$$

$$\mathbf{d}_{\text{CdC}} = \text{Embed}(\text{``Art. 42 CdC: control de grupo societario...''})$$

$$\mathbf{d}_{\text{LGT}} = \text{Embed}(\text{``Art. 42 LGT: responsabilidad tributaria solidaria...''})$$

$$\mathbf{d}_{\text{ET}} = \text{Embed}(\text{``Art. 42 ET: subcontratación de obras...''})$$

The shared anchor "Artículo 42" dominates the similarity, so the nearest-neighbour search returns chunks from all three statutes.

Failure Point 2: The Reranker Scores Them Equally

ColBERT scores each query token against its best-matching document token:

$$\text{Score}(q, d) = \sum_{i=1}^{|q|} \max_{j \in \{1, \dots, |d|\}} \cos(\mathbf{q}_i, \mathbf{d}_j)$$

The tokens ["Artículo", "42"] appear verbatim in all three statutes and score the same against each. The tokens that name the code are too few to break the tie:

$$\text{Score}(q, d_{\text{CdC}}) \approx \text{Score}(q, d_{\text{LGT}}) \approx \text{Score}(q, d_{\text{ET}})$$

The reranker returns clauses from unrelated laws with near-equal scores.

Failure Point 3: The Agent Loops

The agent receives conflicting chunks and cannot tell which statute the user meant, so it keeps searching:

graph TD
    A([Agent Receives Conflicting Chunks]) --> B{Can Agent Resolve<br/>Statute Identity?}
    B -->|No| C[Fetch More Context<br/>from Vector DB]
    C --> D{Still Ambiguous?}
    D -->|Yes| E[Execute Web Scraping<br/>Fallback]
    E --> F{Resolved?}
    F -->|No| G[Thrash: 29-32<br/>Reasoning Steps]
    G --> H[Budget Exhaustion<br/>or Timeout]
    H --> I[Generate Confident<br/>Hallucination]
    B -->|Yes| J([Generate Correct Response])
    D -->|No| J
    F -->|Yes| J

    style I fill:#000000,stroke:#dc2626,stroke-width:2px
    style J fill:#000000,stroke:#16a34a,stroke-width:2px
    style G fill:#000000,stroke:#f57c00,stroke-width:2px

Latency rises to 30 seconds to 5 minutes per query (Measured, audited system: wall-clock, client-side, across the test battery). The agent then answers from the wrong statute, with no disclaimer.

The model summarises the evidence it was given. The evidence was wrong before the model saw it. Prompting, a larger model or a check on the output does not change what was retrieved. The fix has to sit before the vector database query.

---

3. The Probes

Three probes were run against the live product, with Código de Comercio Artículo 42 as the target. Queries and answers are verbatim.

Test 1 (T3): Verbatim Sub-Clause, letra b) of Art. 42.1 CdC

The probe asked: "Reproduce the exact text of Art. 42.1, letra b) of the Código de Comercio."

The correct text (ground truth):

"Tenga la facultad de nombrar o destituir a la mayoría de los miembros del órgano de administración." (English: "Has the power to appoint or remove the majority of the members of the administrative body.") Art. 42.1 b) CdC (corporate group control: power to appoint or remove board members)

The system returned:

"Las que sean causantes o colaboren en la ocultación o transmisión de bienes... de la Administración tributaria." (English: "Those who cause or collaborate in the concealment or transfer of assets... of the Tax Administration.") Art. 42 of the Ley General Tributaria (solidary tax liability for concealment of assets)

The result is the wrong statute: tax-liability text quoted as commercial code, with no disclaimer.

---

Test 2 (T2): apartado c), the letra/apartado Distinction

The probe asked: "¿Qué establece el apartado c) del Artículo 42 del Código de Comercio?" (English: "What does paragraph c) of Article 42 of the Commercial Code establish?")

The correct answer is the commercial-law sub-clause apartado c) of Art. 42 CdC.

The system returned three failures:

1. It did not use the Código de Comercio.
2. It invented a premise: "según el contexto de los hechos que estamos tratando" ("according to the context of the facts we are discussing"), referring to a prior conversation that did not exist.
3. It cited two wrong statutes: Art. 42.4 c) of the Estatuto de los Trabajadores (subcontracting) and Ley 15/1988 (arbitration).

The result is labour law and arbitration text for a commercial-law query, and an invented premise.

---

Test 3 (T1): Sub-Clause That Does Not Exist, apartado 8

The probe asked: "¿Qué establece el apartado 8 del Artículo 42 del Código de Comercio?" (English: "What does paragraph 8 of Article 42 of the Commercial Code establish?")

The correct answer is that Art. 42 CdC has no apartado 8: "El Artículo 42 del Código de Comercio no contiene un apartado 8." (English: "Article 42 of the Commercial Code does not contain a paragraph 8.")

The system returned:

1. It invented text for apartado 8:

   "Se presumirá que existe control cuando una sociedad, por cualquier otro medio, pueda ejercer una influencia dominante sobre otra." (English: "It shall be presumed that control exists when a company, by any other means, can exercise a dominant influence over another.") No such provision exists in Art. 42 CdC.

2. It invented a source: "De acuerdo con la información proporcionada en el análisis previo" ("According to the information provided in the previous analysis"). There was no previous analysis.
3. It cited repealed law as current: the "unidad de decisión" ("unity of decision") concept from Ley 62/2003, repealed by Ley 16/2007.

The result is three failures in one answer: an invented sub-clause, an invented source and a repealed law.

The French version of this battery, on L233-3 and L233-10, which are near-unique identifiers, passed. That control condition ties the failure to article-number collision, not to the model.

3.1 Sample Size

Three probes establish that the failure exists and reproduces. They do not give its frequency.

• Every collision probe failed, and the French control passed.
• §2 and §8 explain why any collision query meets the same failure, which is why three failures characterise the class.
• The failure rate over the vendor's real traffic is unknown. The $\alpha = 5\%$ collision rate in §4.2 is an assumption.

A larger battery would tighten the frequency estimate. It would not change the argument.

3.2 Corpus and Ground Truth

Each answer was checked against the consolidated text of the Código de Comercio in my own reference corpus. The correct text of Art. 42.1 b) is known; the system returned Art. 42 of the Ley General Tributaria instead.

Two limits:

• The ground truth is my copy of the law. A reader can check the three answers against the BOE's consolidated text of each statute.
• Temporal validity was not tested systematically. The repealed-law failure in T1 appeared inside a collision probe. Whether the system returns the provision in force on a given date is a separate question, and the control in §5 does not address it (§10.3).

The collision finding stands: the wrong statute, checked against the right text, three times out of three.

---

4. Cost of Agent Thrashing

4.1 Per-Query Cost

With mixed chunks, the agent requests more context, falls back to web scraping, and runs 29 to 32 reasoning steps, each an LLM call.

Let $n$ be the number of steps, $c_{\text{step}}$ the cost per step and $t_{\text{step}}$ the latency per step:

$$C_{\text{thrash}} = n \cdot c_{\text{step}}$$

$$T_{\text{thrash}} = n \cdot t_{\text{step}}$$

$$C_{\text{thrash}} = 30 \times \$0.003 = \$0.09 \text{ per failed query}$$

$$T_{\text{thrash}} = 30 \times 6\text{s} = 180\text{s} \text{ (3 minutes)}$$

180 s sits inside the measured 30 s to 5 min range, the only cross-check available.

4.2 Monthly Cost at Scale

Let $\alpha$ be the share of queries that hit a colliding article number (Art. 1, 42, 50, 100 and others):

$$\text{Daily Thrashing Cost} = Q_{\text{daily}} \cdot \alpha \cdot C_{\text{thrash}}$$

With $Q_{\text{daily}} = 50{,}000$ and $\alpha = 5\%$:

$$\text{Daily Cost} = 50{,}000 \times 0.05 \times \$0.09 = \$225/\text{day} = \$6{,}750/\text{month}$$

Warning

$Q_{\text{daily}}$ and $\alpha$ are illustrative. I have no view of the vendor's traffic. The monthly figure is linear in both: $1,350 at $\alpha = 1\%$, $13,500 at $\alpha = 10\%$. The figure to rely on is the per-query cost.

This is compute spent on steps that end in a wrong answer. It excludes the cost of the wrong answers themselves.

4.3 Latency Under Collision

$$T_{\text{query}} = \begin{cases} T_{\text{normal}} \approx 2\text{–}5\text{s} & \text{if article number is unique (no collision)} \\ T_{\text{thrash}} \approx 30\text{–}300\text{s} & \text{if article number collides across statutes} \end{cases}$$

Both branches are Measured, audited system: the fast one from the French control, the slow one from the collision battery.

A mean-latency dashboard hides this split. The slow branch is predictable: it is every query on a colliding article number.

---

5. The Control: A Deterministic Statute-Binding Router

§5.1 to §5.4 are the specification as written in April 2026. §5.5 records the reference implementation, legal-rag-router, and where it departs from the specification. It covers UK legislation.

5.1 Principle

The vector database, embedding model and generation tier stay as they are. One deterministic routing step is added between the query and retrieval.

It runs three operations on local CPU (about 7 ms in total, By design, modeled from published component benchmarks):

1. Entity extraction reads article numbers, statute names and abbreviations from the query.
2. A collision check looks the article number up in a collision index.
3. The router then attaches a metadata filter to the database query, or asks the user which statute they mean.

5.2 Before and After

Before: Unfiltered Retrieval in the Audited System

graph LR
    Q([User Query:<br/>'Art. 42 CdC']) --> Embed[Dense Embedding]
    Embed --> HNSW[HNSW Search<br/>Milvus]
    HNSW --> Mixed[Mixed Chunks:<br/>CdC + LGT + ET]
    Mixed --> ColBERT[ColBERT Rerank<br/>Scores ≈ Equal]
    ColBERT --> Agent[ReAct Agent<br/>30-32 Steps]
    Agent --> Wrong[Wrong Statute<br/>Confident Hallucination]

    style Mixed fill:#000000,stroke:#dc2626,stroke-width:2px
    style Wrong fill:#000000,stroke:#dc2626,stroke-width:2px
    style Agent fill:#000000,stroke:#f57c00,stroke-width:2px

After: Router Inserted, as Specified

graph TD
    Q([User Query]) --> Router

    subgraph SBR ["Statute-Binding Router (~7ms, Local CPU)"]
        Router[spaCy + DistilBERT<br/>Entity Extraction] --> Parse[Extract:<br/>article_num, statute_keyword]
        Parse --> Check{Collision<br/>Detected?}
    end

    Check -->|No Collision| Bind[Bind Metadata Filter<br/>to Vector Query]
    Check -->|Collision + Keyword Found| Bind
    Check -->|Collision + No Keyword| Intercept[Intercept → Ask User:<br/>'By Art. 42, did you mean<br/>CdC, LGT, or ET?']

    Intercept -->|User Selects| Bind
    Bind --> DB[(Vector DB<br/>Hard-Filtered<br/>statute_id = 'CdC')]
    DB --> ColBERT2[ColBERT Rerank<br/>Clean Candidates]
    ColBERT2 --> LLM[LLM Generation<br/>Correct Evidence]
    LLM --> Correct([Correct Response])

    style SBR fill:#000000,stroke:#1a73e8,stroke-width:2px
    style Correct fill:#000000,stroke:#16a34a,stroke-width:2px
    style Intercept fill:#000000,stroke:#f57c00,stroke-width:2px

5.3 The Metadata Filter

Once the statute is bound, from the query or from the user's answer, the database query carries a hard predicate:

$$\text{Filter: } \{\texttt{statute\_id} = \text{``CdC''}, \;\texttt{article\_num} = \text{``42''}\}$$

It is applied before candidate generation, not after retrieval, so the reranker only sees chunks from the bound statute.

5.4 The Collision Index

A pre-computed table from article numbers to the statutes that contain them:

$$\text{CollisionIndex}: \mathbb{N} \to \mathcal{P}(\text{StatuteID})$$

$$\text{CollisionIndex}(42) = \{\text{CdC}, \text{LGT}, \text{ET}, \text{Ley 15/1988}\}$$

If $|\text{CollisionIndex}(n)| > 1$, the number is ambiguous and the router resolves it before retrieval. The index is built by the same extraction that tags the corpus, so query and corpus use the same vocabulary.

5.5 The Reference Implementation: legal-rag-router

azterizm/legal-rag-router implements the control. 0.1.0 was released on PyPI on 2 October 2026 and is listed on /tools. The figures below were measured on 0.1.0. 0.1.1 (17 October 2026) adds "as at" and "as of" dates and changes no binding against 0.1.0 over 102,184 queries.

The scope is UK legislation. It binds UK Public General Acts and UK Statutory Instruments to provision level: 134,219 instruments, index snapshot 28 September 2026. The index is built from legislation.gov.uk, a public source versioned by date, and published with checksums. A third party can download or rebuild it and re-run the sealed batteries without relying on my copy of the law. Scottish, Welsh and Northern Ireland legislation and EU law are reported out of coverage. Spain (BOE) is the next jurisdiction on the roadmap. The three Spanish probes and the acceptance test in §6.3 have not been run against it.

What changed from specification to build:

Measured, reference implementation, UK legislation, sealed run of 29 September 2026: 2,112 labelled queries, hashed before the router ran on them. 1,030 are held-out real citations: taken from legislation and set aside before development; index snapshot 28 September 2026; Python 3.11, macOS arm64.

Measured, reference implementation, routing latency, 30 September 2026, full UK index, 306,336 calls × 3 passes: p50 0.14 ms and p99 2.10 ms on Apple M4; p50 0.50 ms and p99 9.21 ms on an Intel i5-3570 (2012). Both are under Gate 5's 25 ms p99 (§7). The design target of p99 under 2 ms is missed on both. The tail is refusals, which rank suggested titles.

Measured, reference implementation, coverage sweep, full UK index, snapshot 28 September 2026: 100,000 real citations from the statute book, each routed in the clause where a drafter wrote it.

Measured, reference implementation, discovery on queries that cite nothing, sealed test set of 91 drafted queries, 29 September 2026: the right provision is first for 51% and in the top 10 for 88%. Discovery offers candidates and binds none.

Measured, reference implementation against Gemini 3.8 Flash (high), the same 2,112 queries, 1 October 2026: Gemini routing on its own bound the wrong instrument for 102 of 1,897 real citations, changed its answer between repeats on 34 of 200 rows, and took 13.5 s at p50. Used as a citation parser in front of the router's index, it was the most accurate pipeline measured: 1,972 of 2,112 rows as expected, with 0 wrong and 0 invented bindings. The model supplies recall; the index decides what is bound. Report: comparison-uk.md.

These figures measure the binding step, which §10.1 names as the residual risk, on UK citations. They do not test §8's claim inside a vector store, and they say nothing about Spanish law. Method, seals and reports are in the repository.

---

6. Projected Latency and Cost

6.1 Projected Latency

With one statute bound, the agent receives no conflicting evidence and has nothing to loop on (§8).

$$T_{\text{before}} = T_{\text{embed}} + T_{\text{HNSW}} + T_{\text{ColBERT}} + T_{\text{thrash}}$$

$$T_{\text{before}} \approx 0.2\text{s} + 0.1\text{s} + 0.3\text{s} + 180\text{s} = 180.6\text{s}$$

$$T_{\text{after}} = T_{\text{router}} + T_{\text{embed}} + T_{\text{HNSW\_filtered}} + T_{\text{ColBERT}} + T_{\text{LLM}}$$

$$T_{\text{after}} \approx 0.007\text{s} + 0.2\text{s} + 0.05\text{s} + 0.15\text{s} + 2\text{s} = 2.4\text{s}$$

$$\boxed{\frac{T_{\text{before}}}{T_{\text{after}}} = \frac{180.6}{2.4} \approx 75\times \text{ (modeled latency reduction)}}$$

Filtered search and reranking are modeled as faster because the filter shrinks the candidate set.

Tip

Almost all of the 75× comes from removing $T_{\text{thrash}}$. The other terms move by tenths of a second. The question to test is whether binding the statute stops the agent looping, which §8 argues and this section does not measure. If the router binds the wrong statute (§10), the user gets a fast wrong answer, which is why the audit record is part of the control.

6.2 Projected Cost

6.3 Accuracy

The left column is what the live system did. The right column is what the control is designed to do, and why.

---

7. Deployment Specification

The router is specified for a shadow migration with no downtime.

Phase 1: Shadow Routing

Production runs unchanged. All production traffic is mirrored asynchronously to a staging instance with the router. No user sees staging output. Staging logs:

• whether the router detected a collision
• whether the query named a resolvable statute
• whether the staging answer matched production
• the latency difference

Phase 2: Acceptance Gates

At least 72 hours, with cutover gated on these thresholds:

Note

Gate 2 is set at 0% because the collision index is a pre-computed mapping over a finite corpus. A missed collision is a data-generation bug with a mechanical fix. Gate 3 is a statistical target, because extraction from free text is statistical. The two gates separate what the design guarantees from what it achieves on average.

Phase 3: Cutover

A feature flag moves routing to the staging pipeline. The unfiltered path is removed after a bake period. Rollback is one flag change.

The filter assumes statute_id and article_num already exist in the vector database schema. That is a standard field set for a statutory corpus, but unverified for the audited product. Without them a re-index is needed, and the "no database rebuild" line in §9 does not hold.

---

8. Why a Larger Embedding Model Does Not Fix This

Every By design claim here rests on this argument. It is about the representation, so it needs no traffic or benchmark. A reader who rejects it has reason to reject the specification.

The Embedding Argument (Informal)

A dense model maps text to one vector $\mathbf{v} \in \mathbb{R}^d$. Similarity between query $\mathbf{q}$ and document $\mathbf{d}$ is:

$$\text{sim}(\mathbf{q}, \mathbf{d}) = \frac{\mathbf{q} \cdot \mathbf{d}}{\|\mathbf{q}\| \cdot \|\mathbf{d}\|}$$

Two documents that share the anchor "Artículo 42" share a large component along it:

$$\mathbf{d}_{\text{CdC}} = \alpha \cdot \mathbf{e}_{\text{Art42}} + \beta \cdot \mathbf{e}_{\text{commercial}}$$

$$\mathbf{d}_{\text{LGT}} = \alpha \cdot \mathbf{e}_{\text{Art42}} + \gamma \cdot \mathbf{e}_{\text{tax}}$$

With $\alpha \gg \beta, \gamma$, the $\alpha$ term dominates both similarities:

$$\text{sim}(\mathbf{q}, \mathbf{d}_{\text{CdC}}) \approx \text{sim}(\mathbf{q}, \mathbf{d}_{\text{LGT}}) \approx \frac{\alpha^2}{\|\mathbf{q}\| \cdot \|\mathbf{d}\|}$$

The terms that tell the statutes apart, $\beta \cdot \mathbf{e}_{\text{commercial}}$ and $\gamma \cdot \mathbf{e}_{\text{tax}}$, are small next to $\alpha$.

More dimensions do not change this. The shared anchor still carries most of the dot product. A single vector cannot express "Art. 42 of the CdC only" as a hard condition.

Note

This is an argument about a dominant shared term, not a proof with bounds on $\alpha/\beta$. It predicts the score collapse that §3 measured. A model trained to weight statute names more heavily would raise $\beta$. The claim is that this is a tuning race against a boolean condition that already wins, not that the race cannot be won.

Why the Metadata Filter Holds

The filter works outside the embedding space, as a condition on the index:

$$\text{Candidates} = \{d \in \mathcal{D} \mid \texttt{statute\_id}(d) = \text{``CdC''} \wedge \texttt{article\_num}(d) = \text{``42''}\}$$

It is a boolean condition, not a score. Lexical overlap cannot move it.

$$\boxed{\text{Metadata filtering} \in O(1) \text{ precision.} \quad \text{Dense similarity} \in O\!\left(\frac{1}{\text{collision count}}\right) \text{ precision.}}$$

This is the basis of the 0% claim. If $\texttt{statute\_id}(d) \neq \text{``CdC''}$, then $d \notin \text{Candidates}$. It is not ranked lower; it is outside the set. For a correctly bound query the wrong-statute rate is zero, whatever the corpus size, model or traffic.

The remaining risk is the binding step: whether the router picks the right statute_id. That risk is statistical, and §10 covers it.

---

9. Conclusion

A legal-tech platform marketed a sub-1% hallucination rate. On black-box probes against Spanish statutory law, it returned the wrong statute on 3 of 3 article-collision queries: tax liability quoted as commercial code, an invented sub-clause, and a repealed statute cited as current. A solicitor relying on any of them would breach SRA Principle 2. This half is Measured, audited system.

The cause was the retrieval tier. Dense search and ColBERT reranking both score colliding articles almost equally, so the model received a mixed evidence set. The agent then looped for minutes and answered from the wrong law.

The control is one added step: a router that binds the statute and a boolean filter that restricts retrieval to it, before any vector similarity is computed (§8). This half is By design. A reference implementation exists for UK legislation, legal-rag-router (§5.5). It has not been deployed in front of the audited system and does not yet cover Spanish law.

Results

This table compares the audited system with the specification. The reference implementation's results are on UK legislation, so they are stated separately.

Reference Implementation (UK)

legal-rag-router 0.1.0 implements the binding step and the filter for UK legislation; 0.1.1 changes no binding. Measured, reference implementation, blind sealed run, 29 September 2026, 2,112 labelled queries, index snapshot 28 September 2026: 0 of 40 same-numbered sections bound to the wrong Act; 0 of 1,030 held-out real citations bound wrongly; 0 of 68 invented Acts bound; 82 of 1,030 real citations refused. Routing latency p99 2.10 ms on Apple M4, 30 September 2026. Details in §5.5; listed on /tools.

Lessons

1. An output check does not see a retrieval failure. Most AI compliance frameworks, including the NIST AI RMF and ISO 42001, place controls on the output: hallucination detection, filtering, human review. A system can pass all of them and still fail every probe of a predictable class of retrieval queries. If the retriever returns tax law for a commercial-law question, the model summarises tax law. An audit without retrieval probes misses this.

2. Dense embeddings cannot hold a hard identity condition. When one signal, such as "Artículo 42", is shared by different statutes, similarity scores converge whatever the model or dimension. A metadata filter works outside the embedding space and does not degrade with the number of collisions.

3. An agent without retrieval constraints amplifies the error. A ReAct loop fetches more context when unsure. If retrieval is the source of the error, more context adds more of it: more steps, more compute, a confident wrong answer. Binding the statute first removes the loop.

4. The control is small. Specified as a Python service with spaCy and a lookup table, about 7 ms modeled. Built for UK legislation as a dependency-free library with no model in the path: p99 2.10 ms on Apple M4, and no rebuild where chunks already carry instrument and provision identifiers. It binds the citations it can read and refuses or asks on the rest.

5. New corpora break pipeline assumptions. Embed, search, rerank, generate assumes similarity tracks document identity. That holds for unique identifiers such as French codes (L233-3) and fails for shared ones such as Spanish Artículo 42. A RAG system entering a new legal domain needs a check for identifier collision first.

6. An aggregate hallucination rate does not cover a predictable class. The vendor's sub-1% figure may hold over all traffic and still fail every collision query tried. Regulated buyers need a control and an audit record per known failure mode.

7. Label how each claim is supported. The strongest claim here, 0% wrong-statute retrieval, has no measurement behind it; it follows from a boolean predicate. The largest numbers, 75× and $6,525 a month, are the weakest, because they rest on assumptions. One unlabelled results table would hide that difference.

---

10. Residual Risks and Falsifiers

The filter is the reliable part. The risk is in the binding step before it, and one of the three failures seen in §3 is outside this control.

10.1 The Binding Step Is Statistical

§8 holds for a correctly bound query. Extracting statute_id from free text carries normal extraction error:

• Statute names have abbreviations and variants. "CdC", "C. de C.", "Código de Comercio", "el código mercantil" and "mercantil" all mean the same statute. Recall depends on how complete the gazetteer is.
• Queries can be colloquial or in another language. Catalan, Galician and Basque statute names, and mixed Spanish and English, sit outside a gazetteer built from official titles.
• A wrong binding is worse than a loop. If the router binds the wrong statute, the filter returns clean chunks from the wrong law and the user gets a fast, well-grounded wrong answer with no sign of doubt. The audit record makes this traceable; Gate 3 in §7 is there to catch it before cutover.

Measured, reference implementation, UK legislation (§5.5): 0 of 1,030 held-out real citations were bound wrongly, and 7 of 100,000 citations in the sweep, all 7 from wrong link targets in the source text. The cost of refusing instead of guessing is 82 of 1,030 held-out citations refused (8.0%) and 8.54% of the sweep not recognised. UK citations usually carry a title and a year. Spanish citations such as "art. 42 CdC" or a bare "Artículo 42" carry less, so the Spanish rates are unknown.

10.2 Interception Has a Product Cost

Every collision query without a resolvable statute name becomes a question to the user. That is the human oversight of EU AI Act Art. 14, and it is friction. Where the ceiling sits is a product decision.

10.3 Temporal Validity

Test 3 cited Ley 62/2003, repealed by Ley 16/2007. The router does not address whether a provision is in force. That needs a separate control: corpus versioning with in_force_from and repealed_by metadata and a date predicate alongside the statute predicate. It extends the same filtering approach, but this specification does not deliver it, which leaves one of the three observed failures outside the control.

A point-in-time control of this kind has since been built and measured for three sections of the UK Employment Rights Act 1996, on a different audited system (agentic thrashing case study, §5.5). Repeal of Spanish statutes is still open.

10.4 What Would Falsify the Core Claim

---

Regulatory Applicability

These are design-level mappings: how the control would meet each requirement once built. They are not attestations of a certified deployment.

---

Audit Trail

Every query through the router writes a structured record. Below is a specimen of the specified schema, an illustration of the format, not an extract from a running system:

{
  "query_id": "q-20260415-0042",
  "timestamp": "2026-04-15T14:32:07Z",
  "extracted_article": "42",
  "extracted_statute_keyword": "CdC",
  "collision_detected": true,
  "collision_set": ["CdC", "LGT", "ET", "Ley_15_1988"],
  "resolution_method": "keyword_match",
  "bound_statute": "CdC",
  "binding_confidence": 0.97,
  "metadata_filter_applied": {"statute_id": "CdC", "article_num": "42"},
  "routing_latency_ms": 6.8,
  "human_disambiguation_required": false
}

The schema covers the TPRM questions on decision traceability (which statute was selected and why), anomaly detection (collision rate over time), incident review (the routing decision behind any reported answer) and continuous monitoring (input to a retrieval integrity dashboard).

binding_confidence is in the specification because a wrong binding is the main residual risk (§10.1). The built router records a status and a reason instead (§5.5).

---

TL;DR

Measured, audited system, April 2026: a live legal-RAG platform that markets sub-1% hallucination returned the wrong statute on 3 of 3 article-collision probes: tax law quoted as commercial code, an invented sub-clause, and a repealed statute cited as current. Dense search and ColBERT cannot separate statutes that share an article number, and the model cannot correct evidence it was never given.

The control (By design) is a router that binds the statute, and a boolean metadata filter that restricts retrieval to it, before any embedding is compared. Wrong-statute retrieval is then zero for a correctly bound query (§8). Latency and cost figures are modeled. The remaining risk is extraction (§10). Temporal validity needs a separate control (§10.3).

A reference implementation for UK legislation is built as legal-rag-router (on /tools). Measured, reference implementation, sealed run of 29 September 2026: 0 of 40 same-numbered sections bound to the wrong Act, 0 of 1,030 held-out citations bound wrongly, 0 of 68 invented Acts bound. Over 100,000 citations from the statute book, 7 bound to another instrument (§5.5). The Spanish probes have not been re-run.