Fix: build_models resolves separate AsyncOpenAI clients for judge and embedding models

Previously, only the judge_model's profile was looked up and its AsyncOpenAI

client was shared with embedding_factory. When embedding_model has a different

base_url/api_key (e.g. Qwen3-Embedding-4B on SiliconFlow vs gpt-5 on another

gateway), the embedding calls silently used the wrong URL, causing 402/404 errors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
wangwei
2026-07-01 21:15:09 +08:00
co-authored by Copilot
parent 3a82d8c487
commit f6e10145cd
2 changed files with 147 additions and 6 deletions
+13 -6
View File
@@ -98,19 +98,26 @@ def build_models(
) -> tuple[Any, Any]:
"""Create the LLM and embedding clients required by the selected RAGAS metrics.
Dynamically resolves connection settings from the stored LLM Profiles first
(matched by model name), falling back to .env settings when no profile matches.
Resolves connection settings independently for the judge LLM and the embedding
model by looking up each in the stored LLM Profiles (matched by model name).
This allows judge_model and embedding_model to use different gateways / API keys.
Falls back to .env settings when no matching profile is found.
"""
client_kwargs = _resolve_openai_client_kwargs(judge_model, settings)
client = AsyncOpenAI(**client_kwargs)
llm_kwargs = _resolve_openai_client_kwargs(judge_model, settings)
emb_kwargs = _resolve_openai_client_kwargs(embedding_model, settings)
llm_client = AsyncOpenAI(**llm_kwargs)
# Only allocate a second client when the embedding model needs different settings.
emb_client = AsyncOpenAI(**emb_kwargs) if emb_kwargs != llm_kwargs else llm_client
# RAGAS structured-output judge calls can be truncated by the upstream default
# 1024 completion budget, especially for faithfulness and GPT-5 family models.
llm = llm_factory(
judge_model,
client=client,
client=llm_client,
max_tokens=max(1, int(settings.ragas_llm_max_tokens)),
)
embeddings = embedding_factory(provider="openai", model=embedding_model, client=client)
embeddings = embedding_factory(provider="openai", model=embedding_model, client=emb_client)
return llm, embeddings