diff --git a/docs/superpowers/specs/2026-07-29-mcp-search-regulations-design.md b/docs/superpowers/specs/2026-07-29-mcp-search-regulations-design.md new file mode 100644 index 0000000..f7569c6 --- /dev/null +++ b/docs/superpowers/specs/2026-07-29-mcp-search-regulations-design.md @@ -0,0 +1,182 @@ +# MCP Regulation Search Server — Design + +**Date:** 2026-07-29 +**Scope:** Expose the existing compliance knowledge base as a standalone Model Context Protocol (MCP) server module, mounted into the existing FastAPI backend, so external MCP clients (Claude Desktop, GitHub Copilot, Cursor, etc.) can call a single `search_regulations` tool over the network. +**Relationship to existing roadmap:** This is the first half ("Direction A" — expose our data) of the MCP integration opportunity identified during the 2026-07-29 brainstorming session. "Direction B" (consuming external MCP servers, e.g. for US/UK regulatory data) was researched and explicitly rejected for this iteration — no existing open-source regulation MCP server covers this platform's actual sources (国标委/GB standards, CATARC, EUR-Lex); the closest match (`lamcearber-spec/eu-legal-mcp`) is a 0-star, month-old project that only duplicates EUR-Lex data this platform already crawls itself. Direction B is deferred until a concrete need for a jurisdiction this platform doesn't already cover arises. + +--- + +## Goals + +1. Expose exactly one MCP tool, `search_regulations`, backed by the **existing** `AgentConversationService.ask()` application service (`backend/app/application/agent/services.py`) — the same code path already used by the `/api/v1/agent/ask` REST endpoint. Zero new retrieval/answering logic. +2. Package the MCP server as its own self-contained module (`backend/app/mcp/`), then mount it into the existing FastAPI app (`backend/app/api/main.py`) so it ships with the current deployment — no new process, no new deployment pipeline. +3. Use the Streamable HTTP transport (not stdio) — the backend is deployed remotely (6.86.80.9), so external MCP clients must connect over the network, not via a locally-spawned subprocess. +4. Reuse the existing JWT auth mechanism — no new auth system. Any authenticated user (any of the four roles) may call `search_regulations`, matching the existing `/agent/ask` endpoint's access level and the `UserRole` docstring ("knowledge query" is available to all four roles including `READONLY`). + +## Non-Goals + +- Direction B (this platform's agent consuming external MCP servers) — deferred, see rejection rationale above. +- Additional tools beyond `search_regulations` (e.g. perception event queries, compliance checks) — explicit user decision to ship the minimal viable version first. +- Role-based restriction of the tool (e.g. ADMIN-only) — all four roles already have knowledge-query access per the existing RBAC model; no new restriction needed. +- stdio transport / local-only usage — not useful for a remotely-deployed backend. +- Rate limiting or per-client quotas on the MCP endpoint — no existing precedent in this codebase for any endpoint; out of scope until a concrete abuse case appears. + +--- + +## Architecture + +### Module layout + +``` +backend/app/mcp/ + __init__.py + server.py # FastMCP instance, search_regulations tool, auth wrapper, ASGI app builder +``` + +This sits as a new top-level package alongside `app/api/`, `app/application/`, `app/services/`, `app/shared/` — not nested inside `app/api/routes/`, because MCP tool registration (decorator-based schema binding) and its own sub-ASGI-app are a fundamentally different transport mechanism from the FastAPI `APIRouter` REST routes there. Keeping it as its own top-level module satisfies "list MCP as its own module" and keeps the REST route directory free of non-REST concerns. + +`server.py` contains **zero new business logic** — it is a protocol adapter that calls the existing composition root (`app.shared.bootstrap.get_agent_conversation_service()`), the same function `backend/app/api/routes/agent.py` already calls. This is consistent with the architecture rule that new business orchestration belongs in `application/`, not scattered across transport adapters — there is no new orchestration here at all. + +### Tool definition + +```python +# backend/app/mcp/server.py +from mcp.server.fastmcp import FastMCP +from app.shared.bootstrap import get_agent_conversation_service + +mcp = FastMCP("ai-regulations") + +@mcp.tool() +def search_regulations(query: str, top_k: int = 5) -> dict: + """检索法规知识库，返回基于检索结果生成的答案及引用来源。 + + query: 自然语言检索问题，例如"国六排放标准最新要求"。 + top_k: 返回的引用来源条数上限，默认5条。 + """ + _, result = get_agent_conversation_service().ask(query=query, top_k=top_k) + return { + "answer": result.answer, + "sources": [source.__dict__ for source in result.sources], + } +``` + +Calling `ask()` **without** `session_id` is intentional: it skips all `ConversationStore` reads/writes (see `AgentConversationService.ask()` — history/session logic is only engaged when `session_id` is passed), so each MCP tool call is stateless and side-effect-free, matching the "search" semantics (not a multi-turn chat). + +### Transport & mounting + +```python +# backend/app/mcp/server.py (continued) +def build_mcp_asgi_app(): + """Return the mounted MCP ASGI app (Streamable HTTP transport).""" + return mcp.streamable_http_app() +``` + +```python +# backend/app/api/main.py (modified) +from contextlib import AsyncExitStack +from app.mcp.server import build_mcp_asgi_app, MCPAuthMiddleware + +mcp_app = build_mcp_asgi_app() +mcp_app.add_middleware(MCPAuthMiddleware) # see Auth section + +@asynccontextmanager +async def lifespan(app: FastAPI): + """Application lifecycle hooks.""" + async with AsyncExitStack() as stack: + # FastMCP's streamable_http_app() owns a session manager that must be + # started via its own lifespan context. app.mount() does NOT propagate + # nested ASGI lifespans automatically (confirmed Starlette/ASGI limitation: + # https://github.com/modelcontextprotocol/python-sdk/issues/1367) — without + # this, every search_regulations call would fail because the MCP session + # manager was never started. + await stack.enter_async_context(mcp_app.router.lifespan_context(mcp_app)) + + logger.info(f"启动 {settings.app_name} v{settings.app_version}") + preload_runtime_dependencies() + yield + cleanup_runtime_dependencies() + +app.mount("/mcp", mcp_app) +``` + +This is the one non-obvious infrastructure detail in this design: naively mounting `mcp.streamable_http_app()` via `app.mount()` without wiring its lifespan results in tool calls failing at runtime because the MCP session manager was never started — this is not a hypothetical, it is a confirmed, documented limitation of nested ASGI apps. Using stdlib `AsyncExitStack` inside the **existing** `lifespan()` function avoids adding any new dependency to solve it. + +### Auth + +The `/mcp` mount point is protected by a small ASGI middleware (not a FastAPI `Depends`, since the mounted app is not a `FastAPI`/`APIRouter` instance) that: + +1. Reads the `Authorization: Bearer ` header from the incoming ASGI scope. +2. When `settings.auth_enabled` is `False` (dev mode) — passes through unchanged, matching the existing `get_current_user` dev bypass behavior. +3. When `settings.auth_enabled` is `True` — validates the token via the **existing** `get_jwt_handler().decode_token(token)` (`backend/app/infrastructure/auth/jwt_handler.py`). On `ValueError` (expired/invalid/missing), returns an HTTP 401 before the request ever reaches the MCP protocol handler. On success, the request proceeds — no role check, since all roles already have knowledge-query access. + +```python +# backend/app/mcp/server.py (continued) +from starlette.types import ASGIApp, Receive, Scope, Send +from starlette.responses import PlainTextResponse +from app.config.settings import settings +from app.shared.bootstrap import get_jwt_handler + +class MCPAuthMiddleware: + """Reject unauthenticated requests to the mounted MCP app before they reach FastMCP.""" + + def __init__(self, app: ASGIApp) -> None: + self.app = app + + async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: + if scope["type"] != "http" or not settings.auth_enabled: + await self.app(scope, receive, send) + return + + headers = dict(scope["headers"]) + auth_header = headers.get(b"authorization", b"").decode() + token = auth_header.removeprefix("Bearer ").strip() + try: + get_jwt_handler().decode_token(token) + except ValueError as exc: + response = PlainTextResponse(str(exc), status_code=401) + await response(scope, receive, send) + return + + await self.app(scope, receive, send) +``` + +**Operational consequence:** to connect an external MCP client (Claude Desktop, Copilot, etc.) to this server, the user must configure a static bearer token (a JWT obtained via the existing `/api/v1/auth/login` flow) in that client's MCP server config, e.g.: + +```json +{ + "mcpServers": { + "ai-regulations": { + "url": "http://6.86.80.9:8000/mcp", + "headers": { "Authorization": "Bearer " } + } + } +} +``` + +JWTs expire after `expire_minutes` (480 by default, see `JWTHandler`) — long-lived external tool connections will need a token refresh story, but that is an existing limitation of the JWT scheme generally (not new to MCP), so it is not addressed differently here. + +--- + +## Error Handling + +- **Auth failure** (missing/expired/invalid token, when `auth_enabled=True`): HTTP 401 from `MCPAuthMiddleware`, before the MCP protocol layer is invoked at all. +- **Tool execution failure** (e.g. the underlying retrieval/LLM call raises): FastMCP's own tool-call error handling catches exceptions raised inside `@mcp.tool()`-decorated functions and returns them as a normal MCP tool-error result to the calling client — no special handling needed in `search_regulations` itself, consistent with how `/agent/ask`'s REST handler already lets the global FastAPI exception handler in `main.py` catch unexpected errors. +- **Lifespan startup failure** (e.g. `mcp_app`'s session manager fails to start): surfaces the same way any other `lifespan()` failure does today — the app fails to start, visible immediately in logs, not a silent partial-degradation. + +## Testing + +- `backend/tests/mcp/test_search_regulations_tool.py` — unit test for the tool function with a mocked `AgentConversationService` (same mocking style as existing `application/agent` tests): asserts `search_regulations()` calls `.ask(query=..., top_k=...)` with no `session_id`, and shapes the returned dict correctly (`answer`, `sources`). +- `backend/tests/mcp/test_mcp_auth_middleware.py` — unit test for `MCPAuthMiddleware`: no token → 401; invalid/expired token → 401; valid token → request passed through to the wrapped app; `auth_enabled=False` → always passed through. Uses Starlette's `TestClient` against a minimal dummy inner ASGI app, no real MCP protocol handshake needed. +- **Manual end-to-end verification** (not automated): use the official `mcp` Python client (`mcp.client.streamable_http.streamablehttp_client` + `mcp.ClientSession`) to connect to a locally running instance, call `list_tools()`, then call `search_regulations` with a real query, confirming a real answer + sources come back. This is a one-time manual check, not a CI test — full MCP protocol handshake testing would require standing up the `mcp` client SDK as a test dependency for marginal additional confidence beyond the two unit tests above. + +## Dependencies + +- Add `mcp` (official Model Context Protocol Python SDK, provides `mcp.server.fastmcp.FastMCP`) to `backend/requirements.txt`. No existing dependency implements the MCP protocol (JSON-RPC 2.0 framing + Streamable HTTP transport + capability negotiation); hand-rolling this would be substantially more code and more fragile than the official SDK. + +## Out of Scope (deferred to future iterations) + +- Direction B: consuming external MCP servers from this platform's own Agentic RAG pipeline. +- Additional MCP tools (perception event queries, compliance checks). +- Per-role tool restrictions. +- Token refresh / long-lived credential story for external MCP clients beyond the existing JWT expiry behavior. +- Rate limiting on the `/mcp` endpoint.