Update API docs: document judge_language in all score endpoints
- POST /api/score: add judge_language section with default/usage - POST /api/score/async: mention judge_language in docstring - POST /api/score/session_async: add to description string + JSON example - ScoreRequest / SessionScoreRequest: add judge_language to schema example - request_summary snapshots: include judge_language for observability Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -91,6 +91,12 @@ def score_sample(
|
||||
- `factual_correctness` — 回答与参考答案的事实准确性(需 ground_truth)
|
||||
- `semantic_similarity` — 回答与参考答案的语义相似度(需 ground_truth)
|
||||
|
||||
**中文评判 `judge_language`**:
|
||||
- 默认 `"en"`(使用英文评判 prompt)。
|
||||
- 传 `"zh"` 开启中文评判 prompt,提升对中文语料的判定一致性(架构设计 §10.5)。
|
||||
- 可通过 `.env` 设置 `RAGAS_JUDGE_LANGUAGE=zh` 作为全局默认,请求字段优先级更高。
|
||||
- 注意:`noise_sensitivity` 使用内置函数式 prompt,不受此参数影响,始终保持英文。
|
||||
|
||||
**推荐模型配置**:
|
||||
- `judge_model`: `gpt-5`
|
||||
- `embedding_model`: `text-embedding-3-small`
|
||||
|
||||
@@ -45,10 +45,13 @@ def submit_async_score(request: ScoreRequest) -> AsyncScoreJobResponse:
|
||||
**适合 Dify 工作流**:HTTP 节点无需等待评分完成(无超时风险),
|
||||
工作流立即继续,评分结果在 RAGAS 平台「运行列表」中查看。
|
||||
|
||||
评分完成后自动生成:
|
||||
接受与 `POST /api/score` 完全相同的请求体(含 `judge_language`),
|
||||
但以异步方式在后台运行,完成后自动生成:
|
||||
- 各指标得分(`scores.csv`)
|
||||
- 摘要报告(`summary.md`)
|
||||
- LLM 优化建议(`optimization_advice.md`)
|
||||
|
||||
**`judge_language`**:传 `"zh"` 开启中文评判 prompt;默认 `null`(使用全局配置,出厂为 `"en"`)。
|
||||
"""
|
||||
logger.info(
|
||||
"[score_async] submit metrics=%s has_ctx=%s has_gt=%s",
|
||||
|
||||
@@ -54,7 +54,8 @@ logger = logging.getLogger("webapp.api.session_score_jobs")
|
||||
"- `contexts`:检索片段拼接字符串,按 `context_separator` 拆分。\n"
|
||||
"- `ground_truth`:标准答案,可选;缺失时会自动跳过依赖它的指标。\n"
|
||||
"- `metrics`:本次需要计算的指标列表。\n"
|
||||
"- `judge_model` / `embedding_model`:可选;为空时回退到系统默认配置。\n\n"
|
||||
"- `judge_model` / `embedding_model`:可选;为空时回退到系统默认配置。\n"
|
||||
"- `judge_language`:评判 prompt 语言,`\"zh\"` 开启中文评判,默认 `null`(全局配置,出厂为 `\"en\"`)。\n\n"
|
||||
"**处理行为**\n"
|
||||
"1. 服务端立即返回 `202 Accepted`,并生成本次调用的 `job_id`。\n"
|
||||
"2. 系统根据 `session_id` 计算固定 `run_id`,格式为 `session-<sanitized-session_id>`。\n"
|
||||
@@ -75,7 +76,8 @@ logger = logging.getLogger("webapp.api.session_score_jobs")
|
||||
" \"context_separator\": \" |||| \",\n"
|
||||
" \"metrics\": [\"answer_relevancy\", \"faithfulness\"],\n"
|
||||
" \"judge_model\": \"gpt-5.5\",\n"
|
||||
" \"embedding_model\": \"text-embedding-3-small\"\n"
|
||||
" \"embedding_model\": \"text-embedding-3-small\",\n"
|
||||
" \"judge_language\": \"zh\"\n"
|
||||
"}\n"
|
||||
"```"
|
||||
),
|
||||
@@ -113,6 +115,8 @@ def submit_session_async_score(request: SessionScoreRequest) -> SessionScoreJobR
|
||||
|
||||
**适合 Dify 工作流**:在循环节点中批量调用,所有轮次共用同一 `session_id`,
|
||||
最终在 RAGAS 平台「运行列表」中查看完整的批量评估报告。
|
||||
|
||||
**`judge_language`**:传 `"zh"` 开启中文评判 prompt;默认 `null`(使用全局配置,出厂为 `"en"`)。
|
||||
"""
|
||||
logger.info(
|
||||
"[session_async] submit session_id=%s metrics=%s has_ctx=%s has_gt=%s",
|
||||
|
||||
@@ -443,6 +443,7 @@ class ScoreRequest(BaseModel):
|
||||
],
|
||||
"judge_model": "gpt-5",
|
||||
"embedding_model": "text-embedding-3-small",
|
||||
"judge_language": "zh",
|
||||
}
|
||||
}
|
||||
)
|
||||
@@ -573,6 +574,7 @@ class SessionScoreRequest(ScoreRequest):
|
||||
"metrics": ["answer_relevancy", "faithfulness"],
|
||||
"judge_model": "gpt-5.5",
|
||||
"embedding_model": "text-embedding-3-small",
|
||||
"judge_language": "zh",
|
||||
},
|
||||
}
|
||||
]
|
||||
|
||||
@@ -73,6 +73,7 @@ class ScoreJobManager:
|
||||
"metrics": list(request.metrics),
|
||||
"judge_model": request.judge_model or "",
|
||||
"embedding_model": request.embedding_model or "",
|
||||
"judge_language": request.judge_language or "",
|
||||
"has_contexts": bool(request.contexts),
|
||||
"has_ground_truth": bool(request.ground_truth),
|
||||
},
|
||||
|
||||
@@ -113,6 +113,7 @@ class SessionScoreJobManager:
|
||||
"metrics": list(request.metrics),
|
||||
"judge_model": request.judge_model or "",
|
||||
"embedding_model": request.embedding_model or "",
|
||||
"judge_language": request.judge_language or "",
|
||||
"has_contexts": bool(request.contexts),
|
||||
"has_ground_truth": bool(request.ground_truth),
|
||||
"session_id": session_id,
|
||||
|
||||
Reference in New Issue
Block a user