wangwei
754a30ad59
- New POST /api/score/session_async endpoint: same session_id calls append to one shared report
- New GET /api/score/sessions/{session_id}: returns call_count, metric_means, all job records
- New GET /api/score/session/jobs/{job_id}: individual call status
- SessionScoreJobManager: deterministic run_id from session_id, per-session mutex for CSV append, advisor regenerated on every call
- SessionScoreRequest (extends ScoreRequest + session_id), SessionScoreJobResponse, SessionStatus models added
- 24 new tests, all passing
chore(weighted-score): comment out 综合加权得分 display and computation
- report.js: hide 综合加权得分 card in report detail page
- score_jobs.js: hide 综合 chip in async job list
- report_builder.py: overall_ws=None (computation disabled)
- summary.py: weighted_score summary line disabled
- evaluator.py: weighted_score/sample_weight columns no longer written to scores.csv
- score.py /api/score: weighted_score always returns null
- score_job_manager.py + session_score_manager.py: weighted=None
- Updated 3 tests to match new behaviour (6 pre-existing failures unchanged)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
69 lines
2.6 KiB
HTML
69 lines
2.6 KiB
HTML
<h2>优化顾问在什么情况下运行？</h2>
|
|
<p class="subtitle">这决定了模块与现有评测流程的集成方式</p>
|
|
|
|
<div class="options">
|
|
<div class="option" data-choice="a" onclick="toggleSelect(this)">
|
|
<div class="letter">A</div>
|
|
<div class="content">
|
|
<h3>每次评测自动运行</h3>
|
|
<p>run_scenario() 结束后自动调用，无需任何额外配置。</p>
|
|
<div class="pros-cons">
|
|
<div class="pros"><h4>优点</h4><ul>
|
|
<li>零感知，开箱即用</li>
|
|
<li>每次跑完都有建议报告</li>
|
|
</ul></div>
|
|
<div class="cons"><h4>缺点</h4><ul>
|
|
<li>每次都多一次 LLM 调用，不管是否需要</li>
|
|
<li>无法关闭</li>
|
|
</ul></div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="option" data-choice="b" onclick="toggleSelect(this)">
|
|
<div class="letter">B</div>
|
|
<div class="content">
|
|
<h3>YAML 场景中显式开启（推荐）</h3>
|
|
<p>在 scenario YAML 里加一行 <code>optimization_advisor: true</code>，默认关闭。</p>
|
|
<div class="mockup">
|
|
<div class="mockup-header">siemens-pdf-question-bank-online.yaml</div>
|
|
<div class="mockup-body" style="font-family:monospace;font-size:13px;line-height:1.8">
|
|
metrics:<br>
|
|
- faithfulness<br>
|
|
- noise_sensitivity<br>
|
|
...<br>
|
|
<span style="color:#4ade80;font-weight:bold">optimization_advisor: true # 新增</span>
|
|
</div>
|
|
</div>
|
|
<div class="pros-cons">
|
|
<div class="pros"><h4>优点</h4><ul>
|
|
<li>显式可见，按需开启</li>
|
|
<li>与现有 YAML 驱动风格一致</li>
|
|
<li>可为不同场景独立配置</li>
|
|
</ul></div>
|
|
<div class="cons"><h4>缺点</h4><ul>
|
|
<li>需要手动在 YAML 里加一行</li>
|
|
</ul></div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="option" data-choice="c" onclick="toggleSelect(this)">
|
|
<div class="letter">C</div>
|
|
<div class="content">
|
|
<h3>阈值触发（任一指标低于警戒线时自动激活）</h3>
|
|
<p>规则引擎先算，若发现有指标低于阈值则自动启动 LLM 分析；一切正常则跳过。</p>
|
|
<div class="pros-cons">
|
|
<div class="pros"><h4>优点</h4><ul>
|
|
<li>"有问题才报警"，符合直觉</li>
|
|
<li>高分场景无额外成本</li>
|
|
</ul></div>
|
|
<div class="cons"><h4>缺点</h4><ul>
|
|
<li>阈值需要维护，不同场景可能不同</li>
|
|
<li>正常分数时无建议，但用户可能仍想看优化空间</li>
|
|
</ul></div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|