Cortex Agent and Cortex Analyst evaluations: Default metric version and judge model changes¶
On October 13, 2026, Snowflake changed the evaluation defaults ahead of the end-of-life of the v1 judge model, claude-4-sonnet, on October 14, 2026. It had been in the legacy state since August 12, 2026, so accounts that didn’t use it before that date already couldn’t run v1 evaluations. The new judge models also have context windows of about 1 million tokens, which reduces failures on long agent traces.
Note
This behavior change was released on October 13, 2026. It isn’t part of a behavior change bundle. To use a different version or judge model, set version or model explicitly, as described in Impact.
Behavior change¶
This change affects the following metrics when they omit the setting shown or set it to auto:
- Cortex Agent system metrics (
answer_correctness,logical_consistency,tool_selection_accuracy, andtool_execution_accuracy) and the Cortex Analystsql_correctnessmetric:version - Cortex Agent custom metrics:
model
- Before the change:
System metrics defaulted to
v1, and custom metrics defaulted toclaude-4-sonnet.- After the change:
System metrics default to
v3, and custom metrics default toclaude-sonnet-4-6. For both, Snowflake usesopenai-gpt-5.4ifclaude-sonnet-4-6isn’t allowed or available for your account.
Explicitly specified versions and models don’t change.
Impact¶
Scores from different metric versions aren’t directly comparable. Scores for metrics that use a judge model can change. tool_selection_accuracy doesn’t use a judge model, so its scores don’t change. Evaluation costs can also change because judge models have different credit rates; track them with the CORTEX_REST_API_USAGE_HISTORY view.
No action is required to use the new defaults. If you depend on consistent scores:
- Recalibrate thresholds in dashboards, alerts, and CI/CD checks against
v3scores. - To use a specific version or judge, set a system metric
version, such asv2, or a custom metricmodel. Metrics pinned tov1orclaude-4-sonnetno longer run, becauseclaude-4-sonnetreached end-of-life. - Confirm that
claude-sonnet-4-6oropenai-gpt-5.4is allowed for your account and the role that runs the evaluation, and is available through your cross-region inference configuration. See judge models and cross-region inference.
See also¶
- Cortex Agent evaluation metric versions
- Cortex Analyst evaluation metric versions
- Cortex Agent custom metrics
- Cortex model deprecations for August 2026
- Behavior change policy
Ref: 2442