AIRegulation/siemens_ragas
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
42 Commits 1 Branch 0 Tags
a03a24be4ea5d26d7bda4feadc187f3f8a265e59
Commit Graph

22 Commits

Author SHA1 Message Date
wangwei
a03a24be4e feat: add POST /api/score endpoint for Dify real-time scoring 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-22 15:14:19 +08:00
wangwei
e4d4e4968b feat: add InlineScorer service with LLM client caching 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-22 15:03:43 +08:00
wangwei
761faf9c42 feat: add ScoreRequest/ScoreResponse models and SCORE_API_TOKEN setting 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-22 15:00:05 +08:00
wangwei
f9e3ba0f64 feat: add weight config panel to 新建评估 and weighted_score card to report 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 17:28:15 +08:00
wangwei
36e5506e2a feat: report_builder uses weighted means; ReportData gains weighted_score_mean 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 17:16:09 +08:00
wangwei
835614189e feat: ScenarioInfo exposes metric_weights and doc_weights from YAML 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 17:05:26 +08:00
wangwei
ce0d2291b0 feat: yaml_patcher and ProfileApplyRequest support metric_weights and doc_weights 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 17:02:21 +08:00
wangwei
9ad2daff73 feat: restore API文档 nav item (iframe /docs) without touching other 4 modules 2026-06-17 11:24:16 +08:00
wangwei
e8af5b906c chore: remove API docs iframe nav item, rename title to RAGAS 评估控制台 2026-06-17 11:18:01 +08:00
wangwei
8ea2b9c7d2 feat: add API文档 nav item with embedded Swagger UI iframe 2026-06-17 11:09:55 +08:00
wangwei
074800b741 feat: add history report switcher dropdown in report detail view 2026-06-17 10:35:56 +08:00
wangwei
3019390592 feat: add export-to-PDF via browser print with @media print CSS 2026-06-17 10:28:01 +08:00
wangwei
ca01e44ad2 feat(webapp): add session persistence via URL hash routing + sessionStorage 
- app.js: hash-based router (#runs / #new / #profiles / #report/{runId})
  - navigate() pushes history entries for back/forward support
  - _restoreSession() reads hash on load and popstate
  - sessionStorage fallback for same-tab refreshes
  - run-card highlights selected run (.run-card.selected)
- runner.js: use App.navigate() for report redirect; persist lastRunId to sessionStorage
- index.html: report nav button starts disabled (enabled on run select/restore)
- app.css: .run-card.selected with petrol border + ring

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-16 17:55:07 +08:00
wangwei
1a2cc534b8 feat(webapp): add optimization advice section to report UI 
- index.html: add section ⑤ advice block (hidden by default, shown when advice_markdown present)
- report.js: add renderAdvice() called in render(), simple Markdown→HTML converter
- app.js: add noise_sensitivity / factual_correctness / semantic_similarity to shortMetric map
- app.css: add .advice-panel, .advice-badge, .advice-md styles (purple left-border theme)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-16 17:26:37 +08:00
wangwei
91c0dab4f9 fix(advisor): fix LLM API call, wire advice_markdown to webapp, update .env.example timeouts 
- llm_analyzer.py: use llm.langchain_llm.ainvoke() (correct RAGAS 0.4.3 API)
- webapp/models.py: add advice_markdown field to ReportData
- webapp/services/run_reader.py: add read_advice_markdown() reading optimization_advice.md
- webapp/services/report_builder.py: pass advice_markdown into ReportData
- .env.example: OPENAI_TIMEOUT_SECONDS 30→180, RAGAS_METRIC_TIMEOUT_SECONDS 45→300

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-16 17:12:32 +08:00
wangwei
5b60ed12ea feat: add LLM role-assignment panel to 新建评估 view 2026-06-16 16:27:00 +08:00
wangwei
dc8baf8662 feat: add LLM配置 management page (profiles view) 2026-06-16 16:25:20 +08:00
wangwei
e329f59139 feat: add yaml_patcher service to apply LLM profiles to scenario YAML 2026-06-16 16:21:19 +08:00
wangwei
b19054bd66 feat: add /api/llm-profiles CRUD router 2026-06-16 16:18:40 +08:00
wangwei
5d09deb420 feat: add ProfileManager service with JSON persistence 2026-06-16 16:14:31 +08:00
wangwei
b98af29449 feat: add LLMProfile pydantic models 2026-06-16 16:10:37 +08:00
wangwei
e89695e490 Add RAGAS evaluation web console (FastAPI + vanilla JS) 
- webapp/: FastAPI backend with runs/scenarios/evaluations API routers;
  services for run_reader, report_builder, scenario_scanner, task_manager
  (lazy ragas import — server boots even without ragas); Pydantic models
- webapp/static/: single-page console (layout A: left-nav + main area);
  report detail with metric cards, Chart.js distribution histogram,
  grouping table, lowest-score sample review; trigger evaluation + log polling
- webmain.py: uvicorn entry point (alongside existing main.py CLI)
- start.bat: Windows one-click launcher with env checks and auto-browser open
- rag_eval/datasets/: implement missing loader + normalizer modules
  (load_dataset_records, normalize_records) required by evaluator
- scripts/seed_sample_run.py: generate realistic demo run artifacts
- .gitignore: exclude datasets/ data files but keep rag_eval/datasets/ source

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
 2026-06-15 15:53:57 +08:00