Update API docs: document judge_language in all score endpoints

- POST /api/score: add judge_language section with default/usage

- POST /api/score/async: mention judge_language in docstring

- POST /api/score/session_async: add to description string + JSON example

- ScoreRequest / SessionScoreRequest: add judge_language to schema example

- request_summary snapshots: include judge_language for observability

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
wangwei
2026-07-01 20:21:04 +08:00
co-authored by Copilot
parent 24a8688a34
commit 3a82d8c487
6 changed files with 20 additions and 3 deletions
+6
View File
@@ -91,6 +91,12 @@ def score_sample(
- `factual_correctness` — 回答与参考答案的事实准确性(需 ground_truth - `factual_correctness` — 回答与参考答案的事实准确性(需 ground_truth
- `semantic_similarity` — 回答与参考答案的语义相似度(需 ground_truth - `semantic_similarity` — 回答与参考答案的语义相似度(需 ground_truth
**中文评判 `judge_language`**
- 默认 `"en"`(使用英文评判 prompt)。
- 传 `"zh"` 开启中文评判 prompt,提升对中文语料的判定一致性(架构设计 §10.5)。
- 可通过 `.env` 设置 `RAGAS_JUDGE_LANGUAGE=zh` 作为全局默认,请求字段优先级更高。
- 注意:`noise_sensitivity` 使用内置函数式 prompt,不受此参数影响,始终保持英文。
**推荐模型配置** **推荐模型配置**
- `judge_model`: `gpt-5` - `judge_model`: `gpt-5`
- `embedding_model`: `text-embedding-3-small` - `embedding_model`: `text-embedding-3-small`
+4 -1
View File
@@ -45,10 +45,13 @@ def submit_async_score(request: ScoreRequest) -> AsyncScoreJobResponse:
**适合 Dify 工作流**:HTTP 节点无需等待评分完成(无超时风险), **适合 Dify 工作流**:HTTP 节点无需等待评分完成(无超时风险),
工作流立即继续,评分结果在 RAGAS 平台「运行列表」中查看。 工作流立即继续,评分结果在 RAGAS 平台「运行列表」中查看。
评分完成后自动生成: 接受与 `POST /api/score` 完全相同的请求体(含 `judge_language`),
但以异步方式在后台运行,完成后自动生成:
- 各指标得分(`scores.csv` - 各指标得分(`scores.csv`
- 摘要报告(`summary.md` - 摘要报告(`summary.md`
- LLM 优化建议(`optimization_advice.md` - LLM 优化建议(`optimization_advice.md`
**`judge_language`**:传 `"zh"` 开启中文评判 prompt;默认 `null`(使用全局配置,出厂为 `"en"`)。
""" """
logger.info( logger.info(
"[score_async] submit metrics=%s has_ctx=%s has_gt=%s", "[score_async] submit metrics=%s has_ctx=%s has_gt=%s",
+6 -2
View File
@@ -54,7 +54,8 @@ logger = logging.getLogger("webapp.api.session_score_jobs")
"- `contexts`:检索片段拼接字符串,按 `context_separator` 拆分。\n" "- `contexts`:检索片段拼接字符串,按 `context_separator` 拆分。\n"
"- `ground_truth`:标准答案,可选;缺失时会自动跳过依赖它的指标。\n" "- `ground_truth`:标准答案,可选;缺失时会自动跳过依赖它的指标。\n"
"- `metrics`:本次需要计算的指标列表。\n" "- `metrics`:本次需要计算的指标列表。\n"
"- `judge_model` / `embedding_model`:可选;为空时回退到系统默认配置。\n\n" "- `judge_model` / `embedding_model`:可选;为空时回退到系统默认配置。\n"
"- `judge_language`:评判 prompt 语言,`\"zh\"` 开启中文评判,默认 `null`(全局配置,出厂为 `\"en\"`)。\n\n"
"**处理行为**\n" "**处理行为**\n"
"1. 服务端立即返回 `202 Accepted`,并生成本次调用的 `job_id`。\n" "1. 服务端立即返回 `202 Accepted`,并生成本次调用的 `job_id`。\n"
"2. 系统根据 `session_id` 计算固定 `run_id`,格式为 `session-<sanitized-session_id>`。\n" "2. 系统根据 `session_id` 计算固定 `run_id`,格式为 `session-<sanitized-session_id>`。\n"
@@ -75,7 +76,8 @@ logger = logging.getLogger("webapp.api.session_score_jobs")
" \"context_separator\": \" |||| \",\n" " \"context_separator\": \" |||| \",\n"
" \"metrics\": [\"answer_relevancy\", \"faithfulness\"],\n" " \"metrics\": [\"answer_relevancy\", \"faithfulness\"],\n"
" \"judge_model\": \"gpt-5.5\",\n" " \"judge_model\": \"gpt-5.5\",\n"
" \"embedding_model\": \"text-embedding-3-small\"\n" " \"embedding_model\": \"text-embedding-3-small\",\n"
" \"judge_language\": \"zh\"\n"
"}\n" "}\n"
"```" "```"
), ),
@@ -113,6 +115,8 @@ def submit_session_async_score(request: SessionScoreRequest) -> SessionScoreJobR
**适合 Dify 工作流**:在循环节点中批量调用,所有轮次共用同一 `session_id` **适合 Dify 工作流**:在循环节点中批量调用,所有轮次共用同一 `session_id`
最终在 RAGAS 平台「运行列表」中查看完整的批量评估报告。 最终在 RAGAS 平台「运行列表」中查看完整的批量评估报告。
**`judge_language`**:传 `"zh"` 开启中文评判 prompt;默认 `null`(使用全局配置,出厂为 `"en"`)。
""" """
logger.info( logger.info(
"[session_async] submit session_id=%s metrics=%s has_ctx=%s has_gt=%s", "[session_async] submit session_id=%s metrics=%s has_ctx=%s has_gt=%s",
+2
View File
@@ -443,6 +443,7 @@ class ScoreRequest(BaseModel):
], ],
"judge_model": "gpt-5", "judge_model": "gpt-5",
"embedding_model": "text-embedding-3-small", "embedding_model": "text-embedding-3-small",
"judge_language": "zh",
} }
} }
) )
@@ -573,6 +574,7 @@ class SessionScoreRequest(ScoreRequest):
"metrics": ["answer_relevancy", "faithfulness"], "metrics": ["answer_relevancy", "faithfulness"],
"judge_model": "gpt-5.5", "judge_model": "gpt-5.5",
"embedding_model": "text-embedding-3-small", "embedding_model": "text-embedding-3-small",
"judge_language": "zh",
}, },
} }
] ]
+1
View File
@@ -73,6 +73,7 @@ class ScoreJobManager:
"metrics": list(request.metrics), "metrics": list(request.metrics),
"judge_model": request.judge_model or "", "judge_model": request.judge_model or "",
"embedding_model": request.embedding_model or "", "embedding_model": request.embedding_model or "",
"judge_language": request.judge_language or "",
"has_contexts": bool(request.contexts), "has_contexts": bool(request.contexts),
"has_ground_truth": bool(request.ground_truth), "has_ground_truth": bool(request.ground_truth),
}, },
+1
View File
@@ -113,6 +113,7 @@ class SessionScoreJobManager:
"metrics": list(request.metrics), "metrics": list(request.metrics),
"judge_model": request.judge_model or "", "judge_model": request.judge_model or "",
"embedding_model": request.embedding_model or "", "embedding_model": request.embedding_model or "",
"judge_language": request.judge_language or "",
"has_contexts": bool(request.contexts), "has_contexts": bool(request.contexts),
"has_ground_truth": bool(request.ground_truth), "has_ground_truth": bool(request.ground_truth),
"session_id": session_id, "session_id": session_id,