Fix: build_models resolves separate AsyncOpenAI clients for judge and embedding models
Previously, only the judge_model's profile was looked up and its AsyncOpenAI client was shared with embedding_factory. When embedding_model has a different base_url/api_key (e.g. Qwen3-Embedding-4B on SiliconFlow vs gpt-5 on another gateway), the embedding calls silently used the wrong URL, causing 402/404 errors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -98,19 +98,26 @@ def build_models(
|
||||
) -> tuple[Any, Any]:
|
||||
"""Create the LLM and embedding clients required by the selected RAGAS metrics.
|
||||
|
||||
Dynamically resolves connection settings from the stored LLM Profiles first
|
||||
(matched by model name), falling back to .env settings when no profile matches.
|
||||
Resolves connection settings independently for the judge LLM and the embedding
|
||||
model by looking up each in the stored LLM Profiles (matched by model name).
|
||||
This allows judge_model and embedding_model to use different gateways / API keys.
|
||||
Falls back to .env settings when no matching profile is found.
|
||||
"""
|
||||
client_kwargs = _resolve_openai_client_kwargs(judge_model, settings)
|
||||
client = AsyncOpenAI(**client_kwargs)
|
||||
llm_kwargs = _resolve_openai_client_kwargs(judge_model, settings)
|
||||
emb_kwargs = _resolve_openai_client_kwargs(embedding_model, settings)
|
||||
|
||||
llm_client = AsyncOpenAI(**llm_kwargs)
|
||||
# Only allocate a second client when the embedding model needs different settings.
|
||||
emb_client = AsyncOpenAI(**emb_kwargs) if emb_kwargs != llm_kwargs else llm_client
|
||||
|
||||
# RAGAS structured-output judge calls can be truncated by the upstream default
|
||||
# 1024 completion budget, especially for faithfulness and GPT-5 family models.
|
||||
llm = llm_factory(
|
||||
judge_model,
|
||||
client=client,
|
||||
client=llm_client,
|
||||
max_tokens=max(1, int(settings.ragas_llm_max_tokens)),
|
||||
)
|
||||
embeddings = embedding_factory(provider="openai", model=embedding_model, client=client)
|
||||
embeddings = embedding_factory(provider="openai", model=embedding_model, client=emb_client)
|
||||
return llm, embeddings
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user