117 lines
2.9 KiB
Markdown
117 lines
2.9 KiB
Markdown
|
# 异步评分记录功能设计
|
|||
|
|
|
|||
|
|
**日期**: 2026-06-24
|
|||
|
|
**状态**: 已批准，待实现
|
|||
|
|
**范围**: 新增 `POST /api/score/async` 异步评分端点，评分结果持久化到磁盘，前端新增「评分记录」页面展示。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 1. 目标
|
|||
|
|
|
|||
|
|
- Dify 工作流调用 `/api/score/async` 立即返回 `job_id`（202），不等待评分完成
|
|||
|
|
- 后台异步执行 RAGAS 评分，结果写入 `outputs/score-jobs/<job_id>.json`
|
|||
|
|
- RAGAS 平台新增「评分记录」导航页，列表展示所有评分记录及状态
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 2. 架构
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Dify → POST /api/score/async → 202 {job_id, status:"queued"}
|
|||
|
|
↓
|
|||
|
|
ScoreJobManager (线程池)
|
|||
|
|
↓
|
|||
|
|
InlineScorer.score()
|
|||
|
|
↓
|
|||
|
|
outputs/score-jobs/<job_id>.json
|
|||
|
|
↓
|
|||
|
|
GET /api/score/jobs ← 前端「评分记录」页轮询
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 3. 存储格式
|
|||
|
|
|
|||
|
|
`outputs/score-jobs/<job_id>.json`:
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
{
|
|||
|
|
"job_id": "abc123def456",
|
|||
|
|
"status": "completed",
|
|||
|
|
"created_at": "2026-06-24T09:00:00+00:00",
|
|||
|
|
"finished_at": "2026-06-24T09:00:15+00:00",
|
|||
|
|
"request": {
|
|||
|
|
"question": "双源CT的时间分辨率是多少?",
|
|||
|
|
"answer": "双源CT的单扇区时间分辨率为75ms。",
|
|||
|
|
"contexts": null,
|
|||
|
|
"ground_truth": null,
|
|||
|
|
"metrics": ["answer_relevancy"],
|
|||
|
|
"judge_model": "gpt-5",
|
|||
|
|
"embedding_model": "text-embedding-3-small"
|
|||
|
|
},
|
|||
|
|
"scores": {"answer_relevancy": 0.9075},
|
|||
|
|
"weighted_score": 0.9075,
|
|||
|
|
"latency_ms": 12500,
|
|||
|
|
"skipped_metrics": [],
|
|||
|
|
"error": null
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 4. API 端点
|
|||
|
|
|
|||
|
|
### `POST /api/score/async`
|
|||
|
|
|
|||
|
|
请求体与 `POST /api/score` 完全相同（`ScoreRequest`）。
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
// 立即返回 202
|
|||
|
|
{"job_id": "abc123def456", "status": "queued"}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### `GET /api/score/jobs`
|
|||
|
|
|
|||
|
|
返回所有评分记录，按创建时间倒序：
|
|||
|
|
```json
|
|||
|
|
{"jobs": [{...ScoreJobStatus...}]}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### `GET /api/score/jobs/{job_id}`
|
|||
|
|
|
|||
|
|
返回单条评分记录详情。
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 5. 新增文件
|
|||
|
|
|
|||
|
|
| 文件 | 职责 |
|
|||
|
|
|------|------|
|
|||
|
|
| `webapp/services/score_job_manager.py` | ScoreJobManager：线程池 + JSON 持久化 |
|
|||
|
|
| `webapp/api/score_jobs.py` | 3 个端点路由 |
|
|||
|
|
| `webapp/static/js/score_jobs.js` | 前端列表逻辑 + 轮询 |
|
|||
|
|
|
|||
|
|
## 6. 修改文件
|
|||
|
|
|
|||
|
|
| 文件 | 改动 |
|
|||
|
|
|------|------|
|
|||
|
|
| `webapp/models.py` | 新增 `AsyncScoreJobStatus`、`AsyncScoreJobResponse` |
|
|||
|
|
| `webapp/server.py` | 注册 score_jobs router，更新 OPENAPI_TAGS |
|
|||
|
|
| `webapp/static/index.html` | 新增导航项 + section |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 7. 前端「评分记录」页
|
|||
|
|
|
|||
|
|
列表列：时间 / 问题摘要（前40字）/ 指标 / 得分 / 状态
|
|||
|
|
|
|||
|
|
- 进入页面自动刷新
|
|||
|
|
- `queued/running` 记录每 5 秒轮询 `GET /api/score/jobs/{id}` 更新状态
|
|||
|
|
- 得分按 scoreClass（good/warn/bad）着色
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 8. Dify 改造
|
|||
|
|
|
|||
|
|
只改 HTTP 节点 URL：`/api/score` → `/api/score/async`，删除解析响应的代码节点。
|