54 lines
3.9 KiB
HTML
54 lines
3.9 KiB
HTML
|
<h2>优化顾问模块 — 整体架构与数据流</h2>
|
|||
|
|
<p class="subtitle">新增 rag_eval/advisor/ 包，插入 run_scenario() 末尾，复用已有 llm 实例</p>
|
|||
|
|
|
|||
|
|
<div class="mockup">
|
|||
|
|
<div class="mockup-header">执行链路（变更前 → 变更后）</div>
|
|||
|
|
<div class="mockup-body" style="font-family:monospace;font-size:13px;line-height:2">
|
|||
|
|
<span style="color:#94a3b8">run_scenario()</span><br>
|
|||
|
|
→ load_scenario() <span style="color:#94a3b8"># 读 YAML，解析 Scenario + optimization_advisor 字段</span><br>
|
|||
|
|
→ build_models() <span style="color:#94a3b8"># 已有：创建 llm, embeddings</span><br>
|
|||
|
|
→ build_metric_pipeline() <span style="color:#94a3b8"># 已有</span><br>
|
|||
|
|
→ Evaluator.evaluate() <span style="color:#94a3b8"># 已有：打分 → EvaluationResult</span><br>
|
|||
|
|
→ write_run_artifacts() <span style="color:#94a3b8"># 已有：scores.csv / summary.md / ...</span><br>
|
|||
|
|
<span style="color:#4ade80;font-weight:bold">→ run_advisor(result, scenario, llm) # 新增 3 行</span><br>
|
|||
|
|
<span style="color:#4ade80"> → rules.diagnose(score_rows) # 规则引擎：识别异常指标 + 方向</span><br>
|
|||
|
|
<span style="color:#4ade80"> → llm_analyzer.analyze(diag, samples) # LLM：结合低分样本生成中文建议</span><br>
|
|||
|
|
<span style="color:#4ade80"> → writer.write(advice, paths) # 写 optimization_advice.md + 日志</span>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="section">
|
|||
|
|
<h3>新增文件一览</h3>
|
|||
|
|
<div class="mockup">
|
|||
|
|
<div class="mockup-body" style="font-family:monospace;font-size:13px;line-height:1.9">
|
|||
|
|
rag_eval/advisor/<br>
|
|||
|
|
__init__.py <span style="color:#94a3b8">← 暴露 run_advisor()，是外部唯一入口</span><br>
|
|||
|
|
rules.py <span style="color:#94a3b8">← 纯函数，无 LLM，可单独单测</span><br>
|
|||
|
|
llm_analyzer.py <span style="color:#94a3b8">← 接收 llm 实例 + 诊断结构 → 中文 Markdown</span><br>
|
|||
|
|
writer.py <span style="color:#94a3b8">← 写 optimization_advice.md，打日志摘要</span><br>
|
|||
|
|
<br>
|
|||
|
|
rag_eval/shared/models.py <span style="color:#fbbf24">← 修改：Scenario 加 optimization_advisor 字段</span><br>
|
|||
|
|
rag_eval/config/schema.py <span style="color:#fbbf24">← 修改：ScenarioModel 加字段</span><br>
|
|||
|
|
rag_eval/execution/runner.py <span style="color:#fbbf24">← 修改：末尾加 3 行调用</span><br>
|
|||
|
|
rag_eval/reporting/artifacts.py <span style="color:#fbbf24">← 修改：RunArtifactPaths 加 advice_md 路径</span>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="section">
|
|||
|
|
<h3>输出产物</h3>
|
|||
|
|
<div class="mockup">
|
|||
|
|
<div class="mockup-body" style="font-family:monospace;font-size:13px;line-height:1.9">
|
|||
|
|
outputs/online/siemens-pdf-question-bank/<run_id>/<br>
|
|||
|
|
scenario.snapshot.yaml<br>
|
|||
|
|
scores.csv<br>
|
|||
|
|
invalid.csv<br>
|
|||
|
|
summary.md<br>
|
|||
|
|
metadata.json<br>
|
|||
|
|
<span style="color:#4ade80;font-weight:bold">optimization_advice.md ← 新增</span>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<p style="margin-top:1rem;color:#94a3b8;font-size:13px">整体看起来 OK 吗？这是新模块与现有链路的接入方式。</p>
|