doris-mcp-server/test/README.md

# Doris MCP Server Testing System

## Overview

This testing system adopts a layered architecture, including unit tests, integration tests, and client-server tests. The testing system assumes the server is already properly started and focuses on testing functionality rather than startup configuration.

## Testing Architecture

### 1. Unit Tests
- **Location**: `test/security/`, `test/utils/`, `test/tools/`
- **Purpose**: Test individual module functionality
- **Features**: Uses Mock objects, no dependency on external services

### 2. Integration Tests
- **Location**: `test/integration/`
- **Purpose**: Test collaboration between modules
- **Features**: Test complete workflows

### 3. Client-Server Tests
- **Location**: `test/tools/test_tools_client_server.py`, `test/utils/test_query_executor_client_server.py`
- **Purpose**: Test actual server functionality through MCP client
- **Features**: Assumes server is running, skips tests if server is not available

## Configuration Files

### test_config.json
Test configuration file defines how to connect to the running server:

```json
{
  "server_endpoints": {
    "http": {
      "url": "http://localhost:3000/mcp",
      "timeout": 30
    },
    "stdio": {
      "command": "uv",
      "args": ["run", "python", "-m", "doris_mcp_server.main", "--transport", "stdio"],
      "timeout": 30
    }
  },
  "test_settings": {
    "default_transport": "http",
    "retry_attempts": 3,
    "retry_delay": 1.0,
    "test_timeout": 60,
    "enable_performance_tests": true,
    "enable_security_tests": true
  }
}
```

## Usage

### 1. Start the Server

Before running client-server tests, you need to start the server first:

#### HTTP Mode (Recommended)
```bash
# Start HTTP server
./start_server.sh
# or
uv run python -m doris_mcp_server.main --transport http --port 3000
```

#### Stdio Mode
```bash
# Stdio mode is started directly by the client, no need to pre-start
```

### 2. Run Tests

#### Run All Tests
```bash
python -m pytest test/ -v
```

#### Run Unit Tests
```bash
# Security module tests
python -m pytest test/security/ -v

# Tools module tests
python -m pytest test/tools/test_tools_manager.py -v

# Query executor tests
python -m pytest test/utils/test_query_executor.py -v
```

#### Run Integration Tests
```bash
python -m pytest test/integration/ -v
```

#### Run Client-Server Tests
```bash
# Tools Client-Server tests
python -m pytest test/tools/test_tools_client_server.py -v

# QueryExecutor Client-Server tests
python -m pytest test/utils/test_query_executor_client_server.py -v
```

### 3. Test Configuration

#### Modify Server Endpoints
Edit the `test/test_config.json` file:

```json
{
  "server_endpoints": {
    "http": {
      "url": "http://your-server:port/mcp"
    }
  }
}
```

#### Enable/Disable Specific Tests
```json
{
  "test_settings": {
    "enable_performance_tests": false,  // Disable performance tests
    "enable_security_tests": true       // Enable security tests
  }
}
```

## Test Status

### ✅ Completed Test Modules

1. **Security Module** (100% Pass)
   - Authentication tests: 5/5 passed
   - Authorization tests: 7/7 passed
   - Data masking tests: 13/13 passed
   - SQL validation tests: 10/10 passed
   - Security manager tests: 7/7 passed
   - Coverage: 88%

2. **Client-Server Test Architecture** (Implemented)
   - Automatic server connection status detection
   - Automatically skip tests when server is not running
   - Support for both HTTP and Stdio transport modes

### 🔄 Tests Requiring Server Running

1. **Tools Client-Server Tests**
   - Tool list retrieval
   - SQL query execution
   - Database list retrieval
   - Table schema queries
   - Performance statistics
   - Error handling
   - Security authentication

2. **QueryExecutor Client-Server Tests**
   - Simple query execution
   - Database queries
   - Information schema queries
   - Parameterized queries
   - Error handling
   - Security authentication

## Testing Best Practices

### 1. Server Startup Check
All client-server tests automatically check server connection status:
- If server is running normally, execute actual tests
- If server is not running, skip tests and display appropriate message

### 2. Test Isolation
- Unit tests use Mock objects, no dependency on external services
- Integration tests use controlled test environments
- Client-server tests connect to actually running servers

### 3. Error Handling
- Tests don't assume specific success/failure results
- Verify response structure rather than specific content
- Gracefully handle connection failures and timeouts

### 4. Configuration Management
- Use configuration files to manage test parameters
- Support configuration switching for different environments
- Provide reasonable default values

## Troubleshooting

### 1. Server Connection Failure
```
ERROR: Server is not running or not accessible
```
**Solution**: Ensure the server is started and listening on the correct port

### 2. Import Errors
```
ImportError: cannot import name 'DorisUnifiedClient'
```
**Solution**: Check Python path and dependency installation

### 3. Test Timeouts
```
TimeoutError: Test execution timeout
```
**Solution**: Increase timeout settings in `test_config.json`

## Development Guide

### Adding New Client-Server Tests

1. Add test methods in the appropriate test file
2. Use `@pytest.mark.asyncio` decorator
3. Get test client through `client` fixture
4. Implement test callback function
5. Verify response structure

Example:
```python
@pytest.mark.asyncio
async def test_new_feature_via_client(self, client, test_config):
    """Test new feature through client"""
    async def test_callback(client_instance):
        result = await client_instance.call_tool("new_tool", {
            "param": "value"
        })
        
        assert "success" in result
        return result
    
    result = await client.connect_and_run(test_callback)
    assert "success" in result
```

### Modifying Test Configuration

Edit the `test/test_config.json` file to adjust:
- Server endpoints
- Timeout settings
- Test data
- Feature switches

## Summary

This testing system provides complete test coverage, from unit tests to end-to-end client-server tests. Through reasonable configuration and automated connection detection, it ensures tests can run stably in different environments.