feat(session-async): add /api/score/session_async with incremental session report aggregation
- New POST /api/score/session_async endpoint: same session_id calls append to one shared report
- New GET /api/score/sessions/{session_id}: returns call_count, metric_means, all job records
- New GET /api/score/session/jobs/{job_id}: individual call status
- SessionScoreJobManager: deterministic run_id from session_id, per-session mutex for CSV append, advisor regenerated on every call
- SessionScoreRequest (extends ScoreRequest + session_id), SessionScoreJobResponse, SessionStatus models added
- 24 new tests, all passing
chore(weighted-score): comment out 综合加权得分 display and computation
- report.js: hide 综合加权得分 card in report detail page
- score_jobs.js: hide 综合 chip in async job list
- report_builder.py: overall_ws=None (computation disabled)
- summary.py: weighted_score summary line disabled
- evaluator.py: weighted_score/sample_weight columns no longer written to scores.csv
- score.py /api/score: weighted_score always returns null
- score_job_manager.py + session_score_manager.py: weighted=None
- Updated 3 tests to match new behaviour (6 pre-existing failures unchanged)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -0,0 +1,53 @@
|
||||
<h2>优化顾问模块 — 整体架构与数据流</h2>
|
||||
<p class="subtitle">新增 rag_eval/advisor/ 包，插入 run_scenario() 末尾，复用已有 llm 实例</p>
|
||||
|
||||
<div class="mockup">
|
||||
<div class="mockup-header">执行链路（变更前 → 变更后）</div>
|
||||
<div class="mockup-body" style="font-family:monospace;font-size:13px;line-height:2">
|
||||
<span style="color:#94a3b8">run_scenario()</span><br>
|
||||
→ load_scenario() <span style="color:#94a3b8"># 读 YAML，解析 Scenario + optimization_advisor 字段</span><br>
|
||||
→ build_models() <span style="color:#94a3b8"># 已有：创建 llm, embeddings</span><br>
|
||||
→ build_metric_pipeline() <span style="color:#94a3b8"># 已有</span><br>
|
||||
→ Evaluator.evaluate() <span style="color:#94a3b8"># 已有：打分 → EvaluationResult</span><br>
|
||||
→ write_run_artifacts() <span style="color:#94a3b8"># 已有：scores.csv / summary.md / ...</span><br>
|
||||
<span style="color:#4ade80;font-weight:bold">→ run_advisor(result, scenario, llm) # 新增 3 行</span><br>
|
||||
<span style="color:#4ade80"> → rules.diagnose(score_rows) # 规则引擎：识别异常指标 + 方向</span><br>
|
||||
<span style="color:#4ade80"> → llm_analyzer.analyze(diag, samples) # LLM：结合低分样本生成中文建议</span><br>
|
||||
<span style="color:#4ade80"> → writer.write(advice, paths) # 写 optimization_advice.md + 日志</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h3>新增文件一览</h3>
|
||||
<div class="mockup">
|
||||
<div class="mockup-body" style="font-family:monospace;font-size:13px;line-height:1.9">
|
||||
rag_eval/advisor/<br>
|
||||
__init__.py <span style="color:#94a3b8">← 暴露 run_advisor()，是外部唯一入口</span><br>
|
||||
rules.py <span style="color:#94a3b8">← 纯函数，无 LLM，可单独单测</span><br>
|
||||
llm_analyzer.py <span style="color:#94a3b8">← 接收 llm 实例 + 诊断结构 → 中文 Markdown</span><br>
|
||||
writer.py <span style="color:#94a3b8">← 写 optimization_advice.md，打日志摘要</span><br>
|
||||
<br>
|
||||
rag_eval/shared/models.py <span style="color:#fbbf24">← 修改：Scenario 加 optimization_advisor 字段</span><br>
|
||||
rag_eval/config/schema.py <span style="color:#fbbf24">← 修改：ScenarioModel 加字段</span><br>
|
||||
rag_eval/execution/runner.py <span style="color:#fbbf24">← 修改：末尾加 3 行调用</span><br>
|
||||
rag_eval/reporting/artifacts.py <span style="color:#fbbf24">← 修改：RunArtifactPaths 加 advice_md 路径</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h3>输出产物</h3>
|
||||
<div class="mockup">
|
||||
<div class="mockup-body" style="font-family:monospace;font-size:13px;line-height:1.9">
|
||||
outputs/online/siemens-pdf-question-bank/<run_id>/<br>
|
||||
scenario.snapshot.yaml<br>
|
||||
scores.csv<br>
|
||||
invalid.csv<br>
|
||||
summary.md<br>
|
||||
metadata.json<br>
|
||||
<span style="color:#4ade80;font-weight:bold">optimization_advice.md ← 新增</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<p style="margin-top:1rem;color:#94a3b8;font-size:13px">整体看起来 OK 吗？这是新模块与现有链路的接入方式。</p>
|
||||
Reference in New Issue
Block a user