catonline_ai/vw-agentic-rag/docs/development.md

💻 Development Guide

This guide provides comprehensive information for developers working on the Agentic RAG system, including setup, code structure, development workflows, and best practices.

Development Environment Setup

Prerequisites

• Python 3.12+ - Download Python
• Node.js 18+ - Download Node.js
• uv - Python package manager (Install uv)
• Git - Version control
• VS Code (recommended) - Download VS Code

Initial Setup

# Clone the repository
git clone <repository-url>
cd agentic-rag-4

# Install Python dependencies
uv sync --dev

# Install frontend dependencies
cd web && npm install

# Copy configuration template
cp config.yaml config.local.yaml

# Set up environment variables
export OPENAI_API_KEY="your-key"
export RETRIEVAL_API_KEY="your-key"

VS Code Configuration

Recommended VS Code extensions:

{
  "recommendations": [
    "ms-python.python",
    "ms-python.black-formatter",
    "charliermarsh.ruff",
    "ms-python.mypy-type-checker",
    "bradlc.vscode-tailwindcss",
    "ms-vscode.vscode-typescript-next",
    "esbenp.prettier-vscode"
  ]
}

Create .vscode/settings.json:

{
  "python.defaultInterpreterPath": "./.venv/bin/python",
  "python.linting.enabled": true,
  "python.linting.ruffEnabled": true,
  "python.formatting.provider": "black",
  "python.testing.pytestEnabled": true,
  "python.testing.pytestArgs": ["tests/"],
  "editor.formatOnSave": true,
  "editor.codeActionsOnSave": {
    "source.organizeImports": true
  },
  "files.exclude": {
    "**/__pycache__": true,
    "**/.pytest_cache": true,
    "**/.mypy_cache": true
  }
}

Architecture Deep Dive

Backend Architecture (FastAPI + LangGraph)

service/
├── main.py                    # FastAPI application entry point
├── config.py                 # Configuration management
├── ai_sdk_adapter.py         # Data Stream Protocol adapter
├── ai_sdk_chat.py            # AI SDK compatible endpoints
├── llm_client.py             # LLM provider abstractions
├── sse.py                    # Server-Sent Events utilities
├── graph/                    # LangGraph workflow
│   ├── graph.py              # Agent workflow definition  
│   ├── state.py              # State management (TurnState, AgentState)
│   └── message_trimmer.py    # Context window management
├── memory/                   # Session persistence
│   ├── postgresql_memory.py  # PostgreSQL checkpointer
│   └── store.py              # Memory abstractions
├── retrieval/                # Information retrieval
│   └── agentic_retrieval.py  # Tool implementations
├── schemas/                  # Data models
│   └── messages.py           # Pydantic models
└── utils/                    # Shared utilities
    ├── logging.py            # Structured logging
    └── templates.py          # Prompt templates

Frontend Architecture (Next.js + assistant-ui)

web/src/
├── app/
│   ├── layout.tsx            # Root layout with providers
│   ├── page.tsx              # Main chat interface
│   ├── globals.css           # Global styles + assistant-ui
│   └── api/                  # Server-side API routes
│       ├── chat/route.ts     # Chat proxy endpoint
│       └── langgraph/        # LangGraph API proxy
├── components/               # Reusable components
├── hooks/                    # Custom React hooks
└── lib/                      # Utility libraries

Development Workflow

1. Start Development Services

# Terminal 1: Start backend in development mode
make dev-backend
# or
./scripts/start_service.sh --dev

# Terminal 2: Start frontend development server  
make dev-web
# or
cd web && npm run dev

# Alternative: Start both simultaneously
make dev

2. Development URLs

• Backend API: http://localhost:8000
• API Documentation: http://localhost:8000/docs
• Frontend: http://localhost:3000
• Health Check: http://localhost:8000/health

3. Hot Reloading

Both backend and frontend support hot reloading:

• Backend: uvicorn auto-reloads on Python file changes
• Frontend: Next.js hot-reloads on TypeScript/CSS changes

Code Style and Standards

Python Code Style

We use the following tools for Python code quality:

# Format code with Black
uv run black service/ tests/

# Lint with Ruff  
uv run ruff check service/ tests/

# Type checking with MyPy
uv run mypy service/

# Run all quality checks
make lint

Python Coding Standards

# Example: Proper function documentation
async def stream_chat_response(request: ChatRequest) -> AsyncGenerator[str, None]:
    """
    Stream chat response using agent workflow with PostgreSQL session memory.
    
    Args:
        request: Chat request containing messages and session_id
        
    Yields:
        str: SSE formatted events for streaming response
        
    Raises:
        HTTPException: If workflow execution fails
    """
    try:
        # Implementation...
        pass
    except Exception as e:
        logger.error(f"Stream chat error: {e}", exc_info=True)
        raise

TypeScript/React Standards

// Example: Proper component structure
interface ChatInterfaceProps {
  sessionId?: string;
  initialMessages?: Message[];
}

export function ChatInterface({ 
  sessionId, 
  initialMessages = [] 
}: ChatInterfaceProps) {
  // Component implementation...
}

Configuration Management

Use environment-based configuration:

# config.py example
from pydantic_settings import BaseSettings
from typing import Optional

class Config(BaseSettings):
    provider: str = "openai"
    openai_api_key: Optional[str] = None
    retrieval_endpoint: str
    
    class Config:
        env_file = ".env"
        env_prefix = "AGENTIC_"

Testing Strategy

Running Tests

# Run all tests
make test

# Run specific test types
make test-unit           # Unit tests only
make test-integration    # Integration tests only
make test-e2e           # End-to-end tests

# Run with coverage
uv run pytest --cov=service --cov-report=html tests/

# Run specific test file
uv run pytest tests/unit/test_retrieval.py -v

# Run tests with debugging
uv run pytest -s -vvv tests/integration/test_api.py::test_chat_endpoint

Test Structure

tests/
├── unit/                     # Unit tests (fast, isolated)
│   ├── test_config.py
│   ├── test_retrieval.py
│   ├── test_memory.py
│   └── test_graph.py
├── integration/              # Integration tests (with dependencies)
│   ├── test_api.py
│   ├── test_streaming.py
│   ├── test_full_workflow.py
│   └── test_e2e_tool_ui.py
└── conftest.py              # Shared test fixtures

Writing Tests

# Example unit test
import pytest
from service.retrieval.agentic_retrieval import RetrievalTool

class TestRetrievalTool:
    @pytest.fixture
    def tool(self):
        return RetrievalTool(
            endpoint="http://test-endpoint",
            api_key="test-key"
        )
    
    async def test_search_standards(self, tool, httpx_mock):
        # Mock HTTP response
        httpx_mock.add_response(
            url="http://test-endpoint/search",
            json={"results": [{"title": "Test Standard"}]}
        )
        
        # Test the tool
        result = await tool.search_standards("test query")
        
        # Assertions
        assert len(result["results"]) == 1
        assert result["results"][0]["title"] == "Test Standard"

# Example integration test
class TestChatAPI:
    @pytest.mark.asyncio
    async def test_streaming_response(self, client):
        request_data = {
            "messages": [{"role": "user", "content": "test question"}],
            "session_id": "test_session"
        }
        
        response = client.post("/api/chat", json=request_data)
        
        assert response.status_code == 200
        assert response.headers["content-type"] == "text/event-stream"

API Development

Adding New Endpoints

1. Define the schema in service/schemas/:

# schemas/new_feature.py
from pydantic import BaseModel
from typing import List, Optional

class NewFeatureRequest(BaseModel):
    query: str
    options: Optional[List[str]] = []

class NewFeatureResponse(BaseModel):
    result: str
    metadata: dict

2. Implement the logic in appropriate module:

# service/new_feature.py
async def process_new_feature(request: NewFeatureRequest) -> NewFeatureResponse:
    # Implementation
    return NewFeatureResponse(
        result="processed",
        metadata={"took_ms": 100}
    )

3. Add the endpoint in service/main.py:

@app.post("/api/new-feature")
async def new_feature_endpoint(request: NewFeatureRequest):
    try:
        result = await process_new_feature(request)
        return result
    except Exception as e:
        logger.error(f"New feature error: {e}")
        raise HTTPException(status_code=500, detail=str(e))

4. Add tests:

# tests/unit/test_new_feature.py
def test_new_feature_endpoint(client):
    response = client.post("/api/new-feature", json={
        "query": "test",
        "options": ["option1"]
    })
    assert response.status_code == 200

LangGraph Agent Development

Adding New Tools

1. Define the tool in service/retrieval/:

# agentic_retrieval.py
@tool
def new_search_tool(query: str, filters: Optional[dict] = None) -> dict:
    """
    New search tool for specific domain.
    
    Args:
        query: Search query string
        filters: Optional search filters
        
    Returns:
        Search results with metadata
    """
    # Implementation
    return {"results": [], "metadata": {}}

2. Register the tool in service/graph/graph.py:

def build_graph() -> CompiledGraph:
    # Add the new tool to tools list
    tools = [
        retrieve_standard_regulation,
        retrieve_doc_chunk_standard_regulation,
        new_search_tool  # Add new tool
    ]
    
    # Rest of graph building...

3. Update the system prompt to include the new tool:

# config.yaml
llm:
  rag:
    agent_system_prompt: |
      You have access to the following tools:
      - retrieve_standard_regulation: Search standards/regulations
      - retrieve_doc_chunk_standard_regulation: Search document chunks  
      - new_search_tool: Search specific domain

Modifying Agent Workflow

The agent workflow is defined in service/graph/graph.py:

def agent_node(state: TurnState, config: RunnableConfig) -> TurnState:
    """Main agent decision-making node"""
    
    # Get conversation history
    messages = state.get("messages", [])
    
    # Call LLM with tools
    response = llm_with_tools.invoke(messages, config)
    
    # Update state
    new_messages = messages + [response]
    return {"messages": new_messages}

def should_continue(state: TurnState) -> str:
    """Decide whether to continue or finish"""
    
    last_message = state["messages"][-1]
    
    # If LLM called tools, continue to tools
    if last_message.tool_calls:
        return "tools"
    
    # Otherwise, finish
    return "post_process"

Frontend Development

assistant-ui Integration

The frontend uses @assistant-ui/react for the chat interface:

// app/page.tsx
import { Thread } from "@assistant-ui/react";
import { makeDataStreamRuntime } from "@assistant-ui/react-data-stream";

export default function ChatPage() {
  const runtime = makeDataStreamRuntime({
    api: "/api/chat",
  });

  return (
    <div className="h-screen">
      <Thread runtime={runtime} />
    </div>
  );
}

Adding Custom Tool UI

// components/ToolUI.tsx
import { ToolCall, ToolCallContent } from "@assistant-ui/react";

export function CustomToolUI() {
  return (
    <ToolCall toolName="retrieve_standard_regulation">
      <ToolCallContent>
        {({ result }) => (
          <div className="border rounded p-4">
            <h3>Search Results</h3>
            {result?.results?.map((item, index) => (
              <div key={index} className="mt-2">
                <strong>{item.title}</strong>
                <p>{item.description}</p>
              </div>
            ))}
          </div>
        )}
      </ToolCallContent>
    </ToolCall>
  );
}

Styling with Tailwind CSS

The project uses Tailwind CSS with assistant-ui plugin:

// tailwind.config.ts
import { assistant } from "@assistant-ui/react/tailwindcss";

export default {
  content: [
    "./src/**/*.{js,ts,jsx,tsx,mdx}",
  ],
  theme: {
    extend: {},
  },
  plugins: [
    assistant,  // assistant-ui plugin
  ],
};

Database Development

Working with PostgreSQL Memory

The system uses PostgreSQL for session persistence via LangGraph's checkpointer:

# memory/postgresql_memory.py
from langgraph.checkpoint.postgres import PostgresSaver

class PostgreSQLMemoryManager:
    def __init__(self, connection_string: str):
        self.connection_string = connection_string
        self.checkpointer = None
    
    def get_checkpointer(self):
        if not self.checkpointer:
            self.checkpointer = PostgresSaver.from_conn_string(
                self.connection_string
            )
            # Setup tables
            self.checkpointer.setup()
        return self.checkpointer

Database Migrations

For schema changes, update the PostgreSQL setup:

-- migrations/001_add_metadata.sql
ALTER TABLE checkpoints 
ADD COLUMN metadata JSONB DEFAULT '{}';

CREATE INDEX idx_checkpoints_metadata 
ON checkpoints USING GIN (metadata);

Debugging

Backend Debugging

1. Enable debug logging:

export LOG_LEVEL=DEBUG
make dev-backend

2. Use Python debugger:

# Add to code where you want to break
import pdb; pdb.set_trace()

# Or use breakpoint() in Python 3.7+
breakpoint()

3. VS Code debugging:

Create .vscode/launch.json:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "FastAPI Debug",
      "type": "python",
      "request": "launch",
      "program": "${workspaceFolder}/.venv/bin/uvicorn",
      "args": [
        "service.main:app",
        "--reload",
        "--host", "127.0.0.1",
        "--port", "8000"
      ],
      "console": "integratedTerminal",
      "env": {
        "PYTHONPATH": "${workspaceFolder}",
        "LOG_LEVEL": "DEBUG"
      }
    }
  ]
}

Frontend Debugging

1. Browser DevTools: Use React DevTools and Network tab

2. Next.js debugging:

# Start with debug mode
cd web && npm run dev -- --inspect

# Or use VS Code debugger

3. Console logging:

// Add debug logs
console.log("Chat API request:", { messages, sessionId });
console.log("Backend response:", response);

Performance Optimization

Backend Performance

1. Database connection pooling:

# config.yaml
postgresql:
  pool_size: 20
  max_overflow: 10
  pool_timeout: 30

2. Async request handling:

# Use async/await properly
async def handle_request():
    # Good: concurrent execution
    results = await asyncio.gather(
        tool1.search(query),
        tool2.search(query)
    )
    
    # Avoid: sequential execution
    # result1 = await tool1.search(query)
    # result2 = await tool2.search(query)

3. Memory management:

# Limit conversation history
def trim_conversation(messages: List[Message], max_tokens: int = 32000):
    # Implementation to keep conversations under token limit
    pass

Frontend Performance

1. Code splitting:

// Lazy load components
const HeavyComponent = lazy(() => import('./HeavyComponent'));

2. Optimize bundle size:

cd web && npm run build
npm run analyze  # If you have bundle analyzer

Common Development Tasks

Adding Configuration Options

1. Update config schema:

# config.py
class AppConfig(BaseSettings):
    new_feature_enabled: bool = False
    new_feature_timeout: int = 30

2. Use in code:

config = get_config()
if config.app.new_feature_enabled:
    # Feature implementation
    pass

Adding New Dependencies

1. Python dependencies:

# Add to pyproject.toml
uv add fastapi-users[sqlalchemy]

# For development dependencies
uv add --dev pytest-xdist

2. Frontend dependencies:

cd web
npm install @types/lodash
npm install --save-dev @testing-library/react

Environment Management

Create environment-specific configs:

# Development
cp config.yaml config.dev.yaml

# Production  
cp config.yaml config.prod.yaml

# Use specific config
export CONFIG_FILE=config.dev.yaml
make dev-backend

Troubleshooting Development Issues

Common Issues

1. Port conflicts:

# Check what's using port 8000
make port-check

# Kill processes on common ports
make port-kill

2. Python import errors:

# Ensure PYTHONPATH is set
export PYTHONPATH="${PWD}:${PYTHONPATH}"

# Or use uv run
uv run python -m service.main

3. Database connection issues:

# Test PostgreSQL connection
psql -h localhost -U user -d database -c "SELECT 1;"

# Check connection string format
echo $DATABASE_URL

4. Frontend build errors:

# Clear Next.js cache
cd web && rm -rf .next

# Reinstall dependencies
rm -rf node_modules package-lock.json
npm install

Development Best Practices

1. Use feature branches:

git checkout -b feature/new-feature
# Make changes
git commit -m "Add new feature"
git push origin feature/new-feature

2. Write tests first (TDD approach):

# Write test first
def test_new_feature():
    assert new_feature("input") == "expected"

# Then implement
def new_feature(input: str) -> str:
    return "expected"

3. Keep commits small and focused:

# Good commit messages
git commit -m "Add PostgreSQL connection pooling"
git commit -m "Fix citation parsing edge case"
git commit -m "Update frontend dependencies"

4. Document as you go:

def complex_function(param: str) -> dict:
    """
    Brief description of what this function does.
    
    Args:
        param: Description of parameter
        
    Returns:
        Description of return value
        
    Example:
        >>> result = complex_function("test")
        >>> assert result["status"] == "success"
    """

---

This development guide provides the foundation for contributing to the Agentic RAG project. For specific questions or advanced topics, refer to the code comments and existing implementations as examples.