AIRegulation/siemens_ragas
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
64 Commits 1 Branch 0 Tags
e1751447dfe2042706725f802b58ede898f71887
Commit Graph

9 Commits

Author SHA1 Message Date
wangwei
4fd515d2d9 feat: async score jobs — POST /api/score/async + 评分记录 page 
Each async score job:
- Runs InlineScorer.score() in thread pool
- Writes standard run artifacts (metadata.json, scores.csv, summary.md)
- Runs optimization_advisor => optimization_advice.md
- Result appears in 运行列表 and 报告详情 with full report

New endpoints:
- POST /api/score/async  (202, job_id immediate)
- GET  /api/score/jobs   (list all jobs)
- GET  /api/score/jobs/{id} (single job status)

Frontend:
- 评分记录 nav page with card list
- 5s auto-polling for queued/running jobs
- 查看报告 button navigates to existing 报告详情 page

Dify: change /api/score -> /api/score/async, no response parsing needed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-24 17:24:22 +08:00
wangwei
e4d4e4968b feat: add InlineScorer service with LLM client caching 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-22 15:03:43 +08:00
wangwei
36e5506e2a feat: report_builder uses weighted means; ReportData gains weighted_score_mean 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 17:16:09 +08:00
wangwei
835614189e feat: ScenarioInfo exposes metric_weights and doc_weights from YAML 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 17:05:26 +08:00
wangwei
ce0d2291b0 feat: yaml_patcher and ProfileApplyRequest support metric_weights and doc_weights 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 17:02:21 +08:00
wangwei
91c0dab4f9 fix(advisor): fix LLM API call, wire advice_markdown to webapp, update .env.example timeouts 
- llm_analyzer.py: use llm.langchain_llm.ainvoke() (correct RAGAS 0.4.3 API)
- webapp/models.py: add advice_markdown field to ReportData
- webapp/services/run_reader.py: add read_advice_markdown() reading optimization_advice.md
- webapp/services/report_builder.py: pass advice_markdown into ReportData
- .env.example: OPENAI_TIMEOUT_SECONDS 30→180, RAGAS_METRIC_TIMEOUT_SECONDS 45→300

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-16 17:12:32 +08:00
wangwei
e329f59139 feat: add yaml_patcher service to apply LLM profiles to scenario YAML 2026-06-16 16:21:19 +08:00
wangwei
5d09deb420 feat: add ProfileManager service with JSON persistence 2026-06-16 16:14:31 +08:00
wangwei
e89695e490 Add RAGAS evaluation web console (FastAPI + vanilla JS) 
- webapp/: FastAPI backend with runs/scenarios/evaluations API routers;
  services for run_reader, report_builder, scenario_scanner, task_manager
  (lazy ragas import — server boots even without ragas); Pydantic models
- webapp/static/: single-page console (layout A: left-nav + main area);
  report detail with metric cards, Chart.js distribution histogram,
  grouping table, lowest-score sample review; trigger evaluation + log polling
- webmain.py: uvicorn entry point (alongside existing main.py CLI)
- start.bat: Windows one-click launcher with env checks and auto-browser open
- rag_eval/datasets/: implement missing loader + normalizer modules
  (load_dataset_records, normalize_records) required by evaluator
- scripts/seed_sample_run.py: generate realistic demo run artifacts
- .gitignore: exclude datasets/ data files but keep rag_eval/datasets/ source

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
 2026-06-15 15:53:57 +08:00