fix: harden MCP endpoint after code review

Critical: the MCP SDK auto-enables DNS-rebinding protection when its host
parameter is left at the 127.0.0.1 default, hard-coding a loopback-only Host
allow-list. Every remote client (the only deployment this feature targets) was
refused with HTTP 421 before auth or the tool ran. Now driven by a new
MCP_ALLOWED_HOSTS setting, with '*' as an explicit, logged opt-out.

Also bounds query/top_k to match AskRequest (top_k is amplified 4x downstream,
so an unbounded value was a resource-exhaustion vector), decodes the
Authorization header as latin-1 per the ASGI spec instead of raising a 500 on
malformed bytes, and returns WWW-Authenticate on 401 per RFC 7235.

Moves the psycopg2 import guard into backend/tests/conftest.py: duplicated
across four test modules, it only worked because of alphabetical collection
order, and any earlier-sorting package would have reintroduced a live
connection attempt against the production database.

Registers the mcp module in the authoritative backend architecture doc.

84 backend tests pass. Verified against a live server: allowed remote Host
returns a valid initialize result, unknown Host returns 421, missing token
returns 401 with WWW-Authenticate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

backend/tests/mcp/test_mcp_auth_middleware.py

@@ -7,23 +7,7 @@ independent of FastMCP internals.
 
 from __future__ import annotations
 
-import sys
-from unittest.mock import MagicMock, patch
-
-# app.mcp.server imports app.shared.bootstrap at module scope (needed for
-# get_agent_conversation_service/get_jwt_handler), which in turn eagerly
-# imports several Postgres store modules that do `import psycopg2` at their
-# own module scope. Since this is the only test file in backend/tests/mcp/
-# that imports app.mcp.server at module scope (not inside a test function),
-# it is the first thing to trigger that chain during pytest collection —
-# guard psycopg2 here the same way backend/tests/observability/
-# test_model_usage_bootstrap.py and test_model_usage_persistence.py already
-# do, so a real (network-connecting) psycopg2 never gets bound first.
-mock_psycopg2 = MagicMock()
-mock_psycopg2.extras = MagicMock()
-sys.modules.setdefault("psycopg2", mock_psycopg2)
-sys.modules.setdefault("psycopg2.extras", mock_psycopg2.extras)
-sys.modules.setdefault("psycopg2.pool", MagicMock())
+from unittest.mock import patch
 
 from starlette.applications import Starlette
 from starlette.responses import PlainTextResponse
@@ -83,3 +67,27 @@ def test_auth_disabled_always_passes_through():
         client = TestClient(_dummy_app())
         response = client.get("/ping")
     assert response.status_code == 200
+
+
+def test_401_includes_www_authenticate_header():
+    """RFC 7235 requires WWW-Authenticate on 401 so clients can tell why they failed."""
+    with patch("app.mcp.server.settings") as fake_settings:
+        fake_settings.auth_enabled = True
+        client = TestClient(_dummy_app())
+        response = client.get("/ping")
+    assert response.status_code == 401
+    assert response.headers["WWW-Authenticate"] == "Bearer"
+
+
+def test_non_utf8_authorization_header_is_rejected_not_crashed():
+    """A non-UTF-8 header byte must yield a clean 401, not an unhandled 500.
+
+    ASGI header values are latin-1 bytes, so any remote client could otherwise
+    trigger a UnicodeDecodeError inside the middleware at will.
+    """
+    with patch("app.mcp.server.settings") as fake_settings:
+        fake_settings.auth_enabled = True
+        client = TestClient(_dummy_app(), raise_server_exceptions=False)
+        # Bypass the http client's own header encoding by writing raw bytes.
+        response = client.get("/ping", headers={"Authorization": b"Bearer \xff\xfe"})
+    assert response.status_code == 401