AIRegulation-DocAnalysis/docs/superpowers/specs/2026-06-08-compliance-enhancement-design.md

# Compliance Analysis Enhancement Design

**Date:** 2026-06-08
**Directions:** A (Analysis Quality) + B (History & Reports) + C (Deep Chat)
**Approach:** Three independent but coordinated feature sets sharing one DB schema (method one / structured tables).

---

## Goals

1. **A — Analysis Quality:** Parallel clause processing (3-5× speed), fix `highlight_terms` bug (always returns empty), add LLM retry with tenacity, reserve `PassThroughReranker` for future cross-encoder work.
2. **B — Analysis History & Reports:** Auto-save every completed analysis to PostgreSQL, history rail in UI, per-record DOCX export, delete with confirmation.
3. **C — Deep Chat:** Per-finding persistent chat threads grounded in real retrieved text, LLM-generated suggestion questions, multi-turn memory.

---

## Architecture Overview

### Layering Rules (must not be violated)

```
api/routes/         →  thin HTTP handlers, SSE generators only
application/        →  orchestration logic (pipeline.py)
domain/ports/       →  ABCs, no implementation
infrastructure/     →  DB, docx, external calls
shared/bootstrap.py →  composition root, wires everything
```

New business logic goes in `application/compliance/pipeline.py` and domain ports. Never in `services/*` or `workflows/*`.

### Shared Database Schema (B + C)

Three tables, created together so C's FK references are valid from day one:

```sql
CREATE TABLE compliance_analyses (
    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    created_by      VARCHAR(255),
    doc_name        VARCHAR(500),
    standard_name   VARCHAR(500),
    risk_score      INTEGER,
    conclusion      TEXT,
    actions         JSONB,
    para_text       TEXT,
    highlight_terms JSONB
);

CREATE TABLE compliance_findings (
    id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    analysis_id UUID NOT NULL REFERENCES compliance_analyses(id) ON DELETE CASCADE,
    seq         INTEGER NOT NULL,
    title       VARCHAR(500),
    description TEXT,
    status      VARCHAR(50),
    clause_ref  VARCHAR(200)
);

CREATE TABLE finding_chat_messages (
    id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    analysis_id UUID NOT NULL REFERENCES compliance_analyses(id) ON DELETE CASCADE,
    finding_id  UUID NOT NULL REFERENCES compliance_findings(id) ON DELETE CASCADE,
    role        VARCHAR(20) NOT NULL,  -- 'user' | 'assistant'
    content     TEXT NOT NULL,
    created_at  TIMESTAMPTZ NOT NULL DEFAULT now()
);
```

---

## Direction A — Analysis Quality

### A1: Parallel Clause Processing

**Current:** Route handler has a sequential `for i, clause in enumerate(clauses)` loop. Each iteration calls `retrieve_for_clause()` then `check_clause_compliance()` synchronously via `asyncio.to_thread`.

**Change:** Extract a `process_single_clause(clause, idx, ...) -> dict` function in `pipeline.py`, then replace the loop with `asyncio.gather`:

```python
async def run_clauses_parallel(clauses, retrieval_svc, llm_client, standard_name, para_text):
    tasks = [
        asyncio.to_thread(process_single_clause, clause, i, retrieval_svc, llm_client, standard_name, para_text)
        for i, clause in enumerate(clauses)
    ]
    return await asyncio.gather(*tasks, return_exceptions=True)
```

Results are yielded to the SSE stream in original order. Exceptions from individual clauses are caught and emitted as `{type: "error", clause_index: i}` events rather than crashing the whole stream.

### A2: Fix highlight_terms

**Root cause:** `synthesize_conclusion()` passes the LLM response through `json.loads()` but the LLM often wraps output in markdown fences (` ```json ... ``` `), causing a parse failure and silent fallback to `[]`.

**Fix in `pipeline.py`:**

```python
import re

def _extract_json(text: str) -> dict:
    """Strip markdown fences then parse JSON. Raises ValueError on failure."""
    cleaned = re.sub(r"^```(?:json)?\s*|\s*```$", "", text.strip(), flags=re.MULTILINE)
    return json.loads(cleaned)
```

Apply `_extract_json` in `synthesize_conclusion()` instead of bare `json.loads`. Wrap with `@retry` (see A3) so transient parse failures get a second attempt.

### A3: LLM Retry with tenacity

`tenacity` is already in `requirements.txt` but unused. Add to all LLM calls in `pipeline.py`:

```python
from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type

@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=1, max=4),
    retry=retry_if_exception_type((httpx.HTTPError, ValueError)),
    reraise=True,
)
def _call_llm_with_retry(client, prompt: str) -> str:
    """Call LLM and return raw text. Retries on HTTP errors and JSON parse failures."""
    ...
```

On final failure, the calling function catches and emits `{type: "error", text: "LLM call failed after 3 attempts"}` to the SSE stream.

### A4: PassThroughReranker (future-ready stub)

`domain/retrieval/ports.py` already defines a `Reranker` ABC. Add the no-op implementation:

**New file:** `backend/app/infrastructure/retrieval/reranker.py`

```python
from app.domain.retrieval.ports import Reranker, RetrievedChunk

class PassThroughReranker(Reranker):
    """No-op reranker. Replace with CrossEncoderReranker when a local model is available."""

    def rerank(self, query: str, chunks: list[RetrievedChunk], top_k: int) -> list[RetrievedChunk]:
        return chunks[:top_k]
```

Register in `shared/bootstrap.py` as the default `Reranker` implementation.

### A — Files Changed

| File | Action |
|------|--------|
| `backend/app/application/compliance/pipeline.py` | Add `process_single_clause`, `run_clauses_parallel`, `_extract_json`, `_call_llm_with_retry` |
| `backend/app/api/routes/compliance.py` | Replace sequential loop with `await run_clauses_parallel(...)` |
| `backend/app/infrastructure/retrieval/reranker.py` | New — `PassThroughReranker` |
| `backend/app/shared/bootstrap.py` | Register `PassThroughReranker` |

---

## Direction B — History & Reports

### B1: Domain Port

**New file:** `backend/app/domain/compliance/ports.py`

```python
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from datetime import datetime
from typing import Optional

@dataclass
class FindingRecord:
    id: str
    analysis_id: str
    seq: int
    title: str
    description: str
    status: str
    clause_ref: Optional[str] = None

@dataclass
class AnalysisRecord:
    id: str
    created_at: datetime
    created_by: Optional[str]
    doc_name: str
    standard_name: str
    risk_score: int
    conclusion: str
    actions: list
    para_text: str
    highlight_terms: list
    findings: list[FindingRecord] = field(default_factory=list)

class ComplianceRepository(ABC):
    @abstractmethod
    def save_analysis(self, record: AnalysisRecord) -> str: ...
    @abstractmethod
    def list_analyses(self, limit: int = 50, offset: int = 0) -> list[AnalysisRecord]: ...
    @abstractmethod
    def get_analysis(self, analysis_id: str) -> Optional[AnalysisRecord]: ...
    @abstractmethod
    def delete_analysis(self, analysis_id: str) -> None: ...
    @abstractmethod
    def save_message(self, analysis_id: str, finding_id: str, role: str, content: str) -> str: ...
    @abstractmethod
    def get_messages(self, finding_id: str) -> list[dict]: ...
```

### B2: PostgresComplianceRepository

**New file:** `backend/app/infrastructure/compliance/repository.py`

Implements `ComplianceRepository` using `psycopg2` (already in requirements). Connection string from `settings.DATABASE_URL`. Key methods:

- `save_analysis`: INSERT into `compliance_analyses`, then bulk INSERT findings into `compliance_findings`, return `analysis_id` (UUID string).
- `list_analyses`: SELECT with JOIN on findings count, ORDER BY `created_at DESC`, supports limit/offset.
- `get_analysis`: SELECT analysis + all findings by `analysis_id`.
- `delete_analysis`: DELETE cascades to findings and chat messages via FK.
- `save_message` / `get_messages`: INSERT/SELECT on `finding_chat_messages`.

Uses a connection pool (simple `psycopg2.pool.ThreadedConnectionPool`, min=1, max=5).

### B3: Auto-save Hook

In the SSE generator in `compliance.py`, after the `done` event is assembled:

```python
# After yielding the done event
if repo is not None:
    record = AnalysisRecord(
        id="",  # will be assigned by DB
        created_at=datetime.utcnow(),
        created_by=current_user,
        doc_name=doc_name,
        standard_name=standard_name,
        risk_score=done_payload["risk_score"],
        conclusion=done_payload["conclusion"],
        actions=done_payload["actions"],
        para_text=done_payload["para_text"],
        highlight_terms=done_payload["highlight_terms"],
        findings=[FindingRecord(...) for f in accumulated_findings],
    )
    analysis_id = await asyncio.to_thread(repo.save_analysis, record)
    # Emit an extra SSE event so frontend receives the analysis_id
    yield f"data: {json.dumps({'type': 'saved', 'analysis_id': analysis_id})}\n\n"
```

### B4: New API Endpoints

Added to `backend/app/api/routes/compliance.py`:

```
GET    /api/v1/compliance/history
       Query params: limit=20&offset=0
       Response: [{id, created_at, doc_name, standard_name, risk_score, finding_count}]

GET    /api/v1/compliance/history/{analysis_id}
       Response: full AnalysisRecord including findings list

DELETE /api/v1/compliance/history/{analysis_id}
       Response: 204 No Content

GET    /api/v1/compliance/history/{analysis_id}/download
       Response: DOCX file (application/vnd.openxmlformats-officedocument.wordprocessingml.document)
```

### B5: DOCX Export

**New file:** `backend/app/infrastructure/compliance/docx_export.py`

Uses `python-docx` (already in requirements). Generates a structured report:

- Cover: document name, standard, date, risk score badge
- Executive summary: conclusion paragraph
- Findings table: seq / title / status / clause_ref / description
- Action items: numbered list
- Footer: generated by AI Regulation Analysis System

```python
def generate_docx(record: AnalysisRecord) -> bytes:
    """Generate a DOCX compliance report and return as bytes."""
    doc = Document()
    # ... build document ...
    buf = BytesIO()
    doc.save(buf)
    return buf.getvalue()
```

### B6: Frontend — History Rail

`CompliancePage.tsx` gains a left rail (same layout pattern as RagChat's `history-pane`):

```
┌──────────────┬─────────────────────────────────┐
│ History      │  Main Analysis Area              │
│ ──────────   │                                  │
│ 2026-06-08   │  (current analysis or loaded     │
│ doc.pdf      │   read-only historical record)   │
│ ⚠ 72  [↓][×]│                                  │
│ ──────────   │                                  │
│ 2026-06-07   │                                  │
│ csms.pdf     │                                  │
│ ✓ 15  [↓][×]│                                  │
└──────────────┴─────────────────────────────────┘
```

- `[↓]` triggers `GET /history/{id}/download` and saves the DOCX file
- `[×]` shows a confirmation dialog, then calls `DELETE /history/{id}`
- Clicking a row loads that analysis into the main area in read-only mode
- `PageStateContext.ComplianceState` gains `analysisId: string | null` and `isReadOnly: boolean`

On mount, the rail calls `GET /history?limit=20` to populate the list. The list re-fetches after delete or after a new analysis completes (triggered by the `saved` SSE event).

### B — Files Changed

| File | Action |
|------|--------|
| `backend/app/domain/compliance/ports.py` | New — `ComplianceRepository` ABC + data classes |
| `backend/app/infrastructure/compliance/repository.py` | New — `PostgresComplianceRepository` |
| `backend/app/infrastructure/compliance/docx_export.py` | New — `generate_docx()` |
| `backend/app/api/routes/compliance.py` | Add history endpoints + auto-save hook |
| `backend/app/shared/bootstrap.py` | Register `PostgresComplianceRepository` |
| `frontend/src/pages/Compliance/CompliancePage.tsx` | Add History Rail |
| `frontend/src/contexts/PageStateContext.tsx` | Add `analysisId`, `isReadOnly` to `ComplianceState` |

---

## Direction C — Deep Chat

### C1: New Chat Endpoints

Replace the existing `/compliance/chat/{segment_id}` (kept for backward compatibility but deprecated) with finding-scoped endpoints:

```
POST /api/v1/compliance/analyses/{analysis_id}/findings/{finding_id}/chat
     Body: {query: string}
     Response: SSE stream — chunk / done / error events

GET  /api/v1/compliance/analyses/{analysis_id}/findings/{finding_id}/chat
     Response: [{id, role, content, created_at}]

POST /api/v1/compliance/analyses/{analysis_id}/findings/{finding_id}/suggestions
     Response: {questions: [string, string, string]}
```

### C2: Grounded Context Construction

New function in `pipeline.py`:

```python
def build_finding_context(finding: FindingRecord, analysis: AnalysisRecord) -> str:
    """
    Build a grounded system context string for a finding chat thread.
    Combines finding details with analysis metadata for LLM grounding.
    """
    return (
        f"Document: {analysis.doc_name}\n"
        f"Standard: {analysis.standard_name}\n"
        f"Finding [{finding.seq}]: {finding.title}\n"
        f"Status: {finding.status}\n"
        f"Clause reference: {finding.clause_ref or 'N/A'}\n"
        f"Description: {finding.description}\n"
        f"Overall conclusion: {analysis.conclusion}\n"
    )
```

This string is prepended to the system prompt for every chat call — replacing the fragile `segment_context` approach.

### C3: Multi-turn Context

Chat handler fetches existing messages from `finding_chat_messages` via `repo.get_messages(finding_id)` and prepends them to the LLM call as `[{"role": "user"/"assistant", "content": "..."}]` message history. Max history: 10 most recent messages (5 turns) to avoid token overflow.

After each LLM response, both the user message and assistant message are saved via `repo.save_message()`.

### C4: Suggestion Generation

New function in `pipeline.py`:

```python
SUGGESTION_PROMPTS = {
    "non_compliant": "Generate 3 questions focused on remediation steps and timeline.",
    "partial": "Generate 3 questions focused on identifying the compliance gap.",
    "compliant": "Generate 3 questions focused on maintaining and evidencing compliance.",
}

def generate_suggestions(finding: FindingRecord, analysis: AnalysisRecord, llm_client) -> list[str]:
    """
    Generate 3 context-aware follow-up questions for a finding chat thread.
    Returns a list of 3 question strings. Falls back to generic questions on error.
    """
    focus = SUGGESTION_PROMPTS.get(finding.status, SUGGESTION_PROMPTS["partial"])
    context = build_finding_context(finding, analysis)
    prompt = f"{context}\n\n{focus}\nReturn JSON: {{\"questions\": [\"...\", \"...\", \"...\"]}}"
    # ... call LLM, parse JSON, return list ...
    # Fallback on error:
    return ["What are the specific requirements?", "What is the remediation timeline?", "Which regulation clause applies?"]
```

### C5: Frontend — Finding Chat Drawer

New component: `frontend/src/pages/Compliance/FindingChatDrawer.tsx`

Drawer slides in from the right (CSS: `position: fixed; right: 0; width: 420px`), reusing existing CSS variables (`--surface`, `--border`, `--accent`).

Structure:
- Header: finding title + close button
- Suggestions section: 3 chip buttons (only shown before first user message; hidden after)
- Message list: scrollable, same bubble style as RagChat
- Composer: textarea + send button, same pattern as RagChat composer

State managed in `PageStateContext.ComplianceState`:
- `activeFindingId: string | null` — which finding's drawer is open
- Drawer open/close controlled by `activeFindingId !== null`

On open:
1. `GET /analyses/{id}/findings/{fid}/chat` → restore history
2. If history is empty: `POST /findings/{fid}/suggestions` → show chips

Each finding card in `CompliancePage.tsx` gains a `💬 Chat` button that sets `activeFindingId`.

### C — Files Changed

| File | Action |
|------|--------|
| `backend/app/api/routes/compliance.py` | Add 3 new finding-chat endpoints |
| `backend/app/application/compliance/pipeline.py` | Add `build_finding_context`, `generate_suggestions` |
| `backend/app/infrastructure/compliance/repository.py` | Add `save_message`, `get_messages` (already in port) |
| `frontend/src/pages/Compliance/FindingChatDrawer.tsx` | New component |
| `frontend/src/pages/Compliance/CompliancePage.tsx` | Add Chat button to finding cards, render drawer |
| `frontend/src/contexts/PageStateContext.tsx` | Add `activeFindingId` to `ComplianceState` |

---

## Implementation Order

Direction A must be completed first (parallel processing changes the route handler that B's auto-save hook attaches to). B must be completed before C (C's FK references require B's tables and repository).

```
A (parallel + bug fixes + reranker stub)
  └→ B (schema migration + history + DOCX)
       └→ C (finding chat + suggestions)
```

---

## Non-Goals

- PDF export (DOCX only; users convert via Word/WPS)
- Cross-encoder reranking (stub reserved, not implemented)
- Scheduled/automatic crawling
- User-level history isolation (all users share history — global visibility)
- Prompt version management or A/B testing

---

## Constraints

- Backend comments and docstrings: English only
- No new top-level libraries beyond those already in `requirements.txt` (`tenacity`, `python-docx`, `psycopg2-binary` are all present)
- `DOCUMENT_REPOSITORY_BACKEND=postgres` → `PostgresComplianceRepository`; any other value → raise `NotImplementedError` with a clear message (no mock fallback for compliance history)
- Git commits are made by the user, never automated