v0.4.2 preview

2025-06-26 20:23:54 +08:00
parent 97006a756d
commit 8526cb75fe
6 changed files with 151 additions and 72 deletions
--- a/README.md
+++ b/README.md
@@ -21,15 +21,16 @@ under the License.

 Doris MCP (Model Context Protocol) Server is a backend service built with Python and FastAPI. It implements the MCP, allowing clients to interact with it through defined "Tools". It's primarily designed to connect to Apache Doris databases, potentially leveraging Large Language Models (LLMs) for tasks like converting natural language queries to SQL (NL2SQL), executing queries, and performing metadata management and analysis.

-## 🚀 What's New in v0.4.0
+## 🚀 What's New in v0.4.2

- **🛠️ Enhanced Monitoring Tools Module**: Advanced memory tracking, metrics collection, and BE node discovery with modular, extensible design
- **🔍 Mature Query Information Tools**: Enhanced SQL explain and profiling with configurable content truncation, file export for LLM attachments, and advanced query analytics
- **⚙️ Unified Configuration Framework**: Centralized configuration management through `config.py` with comprehensive validation and standardized parameter naming
- **🗃️ Smart Default Database Handling**: Automatic fallback to `information_schema` database, eliminating startup configuration requirements and improving reliability
- **🗑️ Production-Ready Architecture**: Removed experimental tools in favor of battle-tested, production-ready alternatives with better performance and monitoring
+- **🔒 Enhanced Security Framework**: Comprehensive SQL security validation with configurable blocked keywords, SQL injection protection, and unified security configuration management
+- **🛠️ Connection Stability Improvements**: Fixed critical `at_eof` connection errors with advanced connection health monitoring, automatic retry mechanisms, and proactive connection cleanup
+- **⚙️ Flexible Security Configuration**: Environment variable support for security policies (`BLOCKED_KEYWORDS`, `ENABLE_SECURITY_CHECK`) with unified configuration architecture eliminating code duplication
+- **🎯 Centralized Configuration Management**: All security keywords now managed through single configuration source with consistent enforcement across all components
+- **🔧 MCP Version Compatibility**: Resolved MCP library version conflicts with intelligent compatibility layer supporting both MCP 1.8.x and 1.9.x versions
+- **🚀 Production Reliability**: Enhanced error handling, connection diagnostics, and automatic recovery from database connection issues

-> **⚠️ Breaking Changes**: Experimental tools (`column_analysis`, `performance_stats`) have been removed and replaced with enhanced monitoring and query information tools.
+> **🔧 Key Improvements**: Resolved connection stability issues, unified security keyword management, added comprehensive environment variable configuration for security policies, and fixed MCP library version compatibility conflicts.

 ## Core Features

@@ -38,18 +39,18 @@ Doris MCP (Model Context Protocol) Server is a backend service built with Python
 *   **Stdio Communication**: Standard input/output mode for direct integration with MCP clients like Cursor.
 *   **Enterprise-Grade Architecture**: Modular design with comprehensive functionality:
    *   **Tools Manager**: Centralized tool registration and routing with unified interfaces (`doris_mcp_server/tools/tools_manager.py`)
-    *   **Enhanced Monitoring Tools Module**: Advanced memory tracking, metrics collection, and flexible BE node discovery (Enhanced in v0.4.0)
-    *   **Query Information Tools**: Enhanced SQL explain and profiling with configurable content management and LLM file export capabilities (Enhanced in v0.4.0)
+    *   **Enhanced Monitoring Tools Module**: Advanced memory tracking, metrics collection, and flexible BE node discovery with modular, extensible design
+    *   **Query Information Tools**: Enhanced SQL explain and profiling with configurable content truncation, file export for LLM attachments, and advanced query analytics
    *   **Resources Manager**: Resource management and metadata exposure (`doris_mcp_server/tools/resources_manager.py`)
    *   **Prompts Manager**: Intelligent prompt templates for data analysis (`doris_mcp_server/tools/prompts_manager.py`)
 *   **Advanced Database Features**:
-    *   **Query Execution**: High-performance SQL execution with advanced caching and optimization (`doris_mcp_server/utils/query_executor.py`)
-    *   **Security Management**: Comprehensive SQL security validation, data masking, and access control (`doris_mcp_server/utils/security.py`)
+    *   **Query Execution**: High-performance SQL execution with advanced caching and optimization, enhanced connection stability and automatic retry mechanisms (`doris_mcp_server/utils/query_executor.py`)
+    *   **Security Management**: Comprehensive SQL security validation with configurable blocked keywords, SQL injection protection, data masking, and unified security configuration management (`doris_mcp_server/utils/security.py`)
    *   **Metadata Extraction**: Comprehensive database metadata with catalog federation support (`doris_mcp_server/utils/schema_extractor.py`)
    *   **Performance Analysis**: Advanced column analysis, performance monitoring, and data analysis tools (`doris_mcp_server/utils/analysis_tools.py`)
 *   **Catalog Federation Support**: Full support for multi-catalog environments (internal Doris tables and external data sources like Hive, MySQL, etc.)
-*   **Enterprise Security**: Comprehensive security framework with authentication, authorization, SQL injection protection, and data masking capabilities (`doris_mcp_server/utils/security.py`)
-*   **Flexible Configuration**: Comprehensive configuration management with environment variables, file-based config, and validation (`doris_mcp_server/utils/config.py`)
+*   **Enterprise Security**: Comprehensive security framework with authentication, authorization, SQL injection protection, and data masking capabilities with environment variable configuration support
+*   **Unified Configuration Framework**: Centralized configuration management through `config.py` with comprehensive validation, standardized parameter naming, and smart default database handling with automatic fallback to `information_schema`

 ## System Requirements

@@ -65,7 +66,7 @@ Doris MCP (Model Context Protocol) Server is a backend service built with Python
 pip install mcp-doris-server

 # Install specific version
-pip install mcp-doris-server==0.4.0
+pip install mcp-doris-server==0.4.2
 ```

 > **💡 Command Compatibility**: After installation, both `doris-mcp-server` and `mcp-doris-server` commands are available for backward compatibility. You can use either command interchangeably.
@@ -174,6 +175,8 @@ cp .env.example .env
 *   **Security Configuration**:
    *   `AUTH_TYPE`: Authentication type (token/basic/oauth, default: token)
    *   `TOKEN_SECRET`: Token secret key
+    *   `ENABLE_SECURITY_CHECK`: Enable/disable SQL security validation (default: true, New in v0.4.2)
+    *   `BLOCKED_KEYWORDS`: Comma-separated list of blocked SQL keywords (New in v0.4.2)
    *   `ENABLE_MASKING`: Enable data masking (default: true)
    *   `MAX_RESULT_ROWS`: Maximum result rows (default: 10000)
 *   **Performance Configuration**:
@@ -404,16 +407,25 @@ The system automatically validates SQL queries for security risks:

 #### Blocked Operations

-Configure blocked SQL operations:
+Configure blocked SQL operations using environment variables (New in v0.4.2):

 ```bash
-# Environment variable
-BLOCKED_SQL_OPERATIONS=DROP,DELETE,TRUNCATE,ALTER,CREATE,INSERT,UPDATE,GRANT,REVOKE
+# Enable/disable SQL security check (New in v0.4.2)
+ENABLE_SECURITY_CHECK=true
+
+# Customize blocked keywords via environment variable (New in v0.4.2)
+BLOCKED_KEYWORDS="DROP,DELETE,TRUNCATE,ALTER,CREATE,INSERT,UPDATE,GRANT,REVOKE,EXEC,EXECUTE,SHUTDOWN,KILL"

 # Maximum query complexity score
 MAX_QUERY_COMPLEXITY=100
 ```

+**Default Blocked Keywords (Unified in v0.4.2):**
+- **DDL Operations**: DROP, CREATE, ALTER, TRUNCATE
+- **DML Operations**: DELETE, INSERT, UPDATE  
+- **DCL Operations**: GRANT, REVOKE
+- **System Operations**: EXEC, EXECUTE, SHUTDOWN, KILL
+
 #### SQL Injection Protection

 The system automatically detects and blocks:
@@ -968,6 +980,60 @@ Recommendations:
   DORIS_MAX_CONNECTIONS=20
   ```

+### Q: How to resolve `at_eof` connection errors? (Fixed in v0.4.2)
+
+**A:** Version 0.4.2 has resolved the critical `at_eof` connection errors. The improvements include:
+
+1. **Enhanced Connection Health Monitoring**: Strict connection state validation before operations
+2. **Automatic Retry Mechanism**: Failed queries are automatically retried up to 2 times
+3. **Proactive Connection Cleanup**: Automatic detection and cleanup of problematic connections
+4. **Connection Diagnostics**: Comprehensive connection health analysis and reporting
+
+If you still encounter connection issues after upgrading to v0.4.2:
+```bash
+# Check connection diagnostics
+# The system now automatically handles connection recovery
+# Monitor logs for connection health reports
+tail -f logs/doris_mcp_server.log | grep "connection"
+```
+
+### Q: How to resolve MCP library version compatibility issues? (Fixed in v0.4.2)
+
+**A:** Version 0.4.2 introduced an intelligent MCP compatibility layer that supports both MCP 1.8.x and 1.9.x versions:
+
+**The Problem:**
+- MCP 1.9.3 introduced breaking changes to the `RequestContext` class (changed from 2 to 3 generic parameters)
+- This caused `TypeError: Too few arguments for RequestContext` errors
+
+**The Solution (v0.4.2):**
+- **Intelligent Version Detection**: Automatically detects the installed MCP version
+- **Compatibility Layer**: Gracefully handles API differences between versions
+- **Flexible Version Support**: `mcp>=1.8.0,<2.0.0` in dependencies
+
+**Supported MCP Versions:**
+```bash
+# Both versions now work seamlessly
+pip install mcp==1.8.0  # Stable version (recommended)
+pip install mcp==1.9.3  # Latest version with new features
+```
+
+**Version Information:**
+```bash
+# Check which MCP version is being used
+doris-mcp-server --transport stdio
+# The server will log: "Using MCP version: x.x.x"
+```
+
+If you encounter MCP-related startup errors:
+```bash
+# Recommended: Use stable version
+pip uninstall mcp
+pip install mcp==1.8.0
+
+# Or upgrade to latest compatible version
+pip install --upgrade mcp-doris-server==0.4.2
+```
+
 ### Q: How to view server logs?

 **A:** Log files are located in the `logs/` directory. You can:
--- a/doris_mcp_server/main.py
+++ b/doris_mcp_server/main.py
@@ -28,6 +28,17 @@ import json
 import logging
 from typing import Any

+# MCP version compatibility check
+try:
+    import mcp
+    MCP_VERSION = getattr(mcp, '__version__', 'unknown')
+    logger = logging.getLogger(__name__)
+    logger.info(f"Using MCP version: {MCP_VERSION}")
+except Exception as e:
+    logger = logging.getLogger(__name__)
+    logger.warning(f"Could not determine MCP version: {e}")
+    MCP_VERSION = 'unknown'
+
 from mcp.server import Server
 from mcp.server.models import InitializationOptions

@@ -75,6 +86,47 @@ class DorisServer:
        self.logger = logging.getLogger(f"{__name__}.DorisServer")
        self._setup_handlers()

+    def _get_mcp_capabilities(self):
+        """Get MCP capabilities with version compatibility"""
+        try:
+            # For MCP 1.9.x and newer
+            from mcp.server.lowlevel.server import NotificationOptions
+            
+            return self.server.get_capabilities(
+                notification_options=NotificationOptions(
+                    prompts_changed=True,
+                    resources_changed=True,
+                    tools_changed=True
+                ),
+                experimental_capabilities={}
+            )
+        except TypeError:
+            try:
+                # For MCP 1.8.x
+                from mcp.server.lowlevel.server import NotificationOptions
+                
+                return self.server.get_capabilities(
+                    notification_options=NotificationOptions(
+                        prompts_changed=True,
+                        resources_changed=True,
+                        tools_changed=True
+                    ),
+                    experimental_capabilities={}
+                )
+            except Exception as e:
+                self.logger.warning(f"Could not get capabilities with NotificationOptions: {e}")
+                # Fallback for older versions
+                try:
+                    return self.server.get_capabilities()
+                except Exception as fallback_e:
+                    self.logger.error(f"Failed to get capabilities: {fallback_e}")
+                    # Return minimal capabilities
+                    return {
+                        "resources": {},
+                        "tools": {},
+                        "prompts": {}
+                    }
+
    def _setup_handlers(self):
        """Setup MCP protocol handlers"""

@@ -193,18 +245,8 @@ class DorisServer:
                    read_stream, write_stream = streams
                    self.logger.info("stdio_server streams created successfully")
                    
-                    # Create initialization options
-                    # MCP 1.8.0 requires parameters for get_capabilities
-                    from mcp.server.lowlevel.server import NotificationOptions
-                    
-                    capabilities = self.server.get_capabilities(
-                        notification_options=NotificationOptions(
-                            prompts_changed=True,
-                            resources_changed=True,
-                            tools_changed=True
-                        ),
-                        experimental_capabilities={}
-                    )
+                    # Create initialization options with version compatibility
+                    capabilities = self._get_mcp_capabilities()
                    
                    init_options = InitializationOptions(
                        server_name="doris-mcp-server",
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -20,7 +20,7 @@ build-backend = "hatchling.build"

 [project]
 name = "mcp-doris-server"
-version = "0.4.0"
+version = "0.4.2"
 description = "Enterprise-grade Model Context Protocol (MCP) server implementation for Apache Doris"
 authors = [
    {name = "Yijia Su", email = "freeoneplus@apache.org"}
@@ -42,7 +42,7 @@ classifiers = [

 dependencies = [
    # Core MCP dependencies
-    "mcp>=1.8.0",
+    "mcp>=1.8.0,<2.0.0",
    # Database drivers
    "aiomysql>=0.2.0",
    "PyMySQL>=1.1.0",
--- a/requirements-dev.txt
+++ b/requirements-dev.txt
@@ -1,21 +1,5 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-#   http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-# 开发依赖 - 从 pyproject.toml 自动生成
-# 安装命令: pip install -r requirements-dev.txt
+# Development dependencies - auto-generated from pyproject.toml
+# Installation command: pip install -r requirements-dev.txt

 pytest>=7.4.0
 pytest-asyncio>=0.23.0
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,24 +1,8 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-#   http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-# 主要依赖 - 从 pyproject.toml 自动生成
-# 请不要手动编辑此文件，使用 python generate_requirements.py 重新生成
+# Main dependencies - auto-generated from pyproject.toml
+# Do not edit this file manually, use 'python generate_requirements.py' to regenerate

-# === 核心依赖 ===
-mcp>=1.0.0
+# === Core Dependencies ===
+mcp>=1.8.0,<2.0.0
 aiomysql>=0.2.0
 PyMySQL>=1.1.0
 asyncio-mqtt>=0.16.0
@@ -53,8 +37,11 @@ click>=8.1.0
 typer>=0.9.0
 requests>=2.31.0
 tqdm>=4.66.0
+pytest>=8.4.0
+pytest-asyncio>=1.0.0
+pytest-cov>=6.1.1

-# === 开发依赖 ===
+# === Development Dependencies ===
 pytest>=7.4.0
 pytest-asyncio>=0.23.0
 pytest-cov>=4.1.0
--- a/uv.lock
+++ b/uv.lock
@@ -948,7 +948,7 @@ wheels = [

 [[package]]
 name = "mcp-doris-server"
-version = "0.4.0"
+version = "0.4.1.1"
 source = { editable = "." }
 dependencies = [
    { name = "aiofiles" },
@@ -1054,7 +1054,7 @@ requires-dist = [
    { name = "httpx", specifier = ">=0.26.0" },
    { name = "isort", marker = "extra == 'dev'", specifier = ">=5.13.0" },
    { name = "jaeger-client", marker = "extra == 'monitoring'", specifier = ">=4.8.0" },
-    { name = "mcp", specifier = ">=1.8.0" },
+    { name = "mcp", specifier = ">=1.8.0,<2.0.0" },
    { name = "mypy", marker = "extra == 'dev'", specifier = ">=1.8.0" },
    { name = "myst-parser", marker = "extra == 'dev'", specifier = ">=2.0.0" },
    { name = "myst-parser", marker = "extra == 'docs'", specifier = ">=2.0.0" },