Cortex LLM REST API¶

Cost considerations¶

Snowflake Cortex LLM REST API requests incur compute costs based on the number of tokens processed. Refer to the Snowflake Service Consumption Table for each function’s cost in credits per million tokens. A token is the smallest unit of text processed by Snowflake Cortex LLM functions, approximately equal to four characters of text. The equivalence of raw input or output text to tokens can vary by model.

The COMPLETE function generates new text given an input prompt. Both input and output tokens incur compute cost. If you use COMPLETE to provide a conversational or chat user experience, all previous prompts and responses are processed to generate each new response, with corresponding costs.

Model availability¶

The following table contains the models that are available in the Cortex LLM REST API.

You can also use any fine-tuned model in any supported region.

Usage quotas¶

To ensure a high standard of performance for all Snowflake customers, Snowflake Cortex LLM REST API requests are subject to usage quotas beyond which requests may be throttled. Snowflake may adjust these quotas from time to time. The quotas in the table below are applied per account and are independently applied for each model.

COMPLETE endpoint¶

The /api/v2/cortex/inference:complete endpoint executes the SQL COMPLETE function. It takes the form:

POST https://<account_identifier>.snowflakecomputing.cn/api/v2/cortex/inference:complete

where account_identifier is the account identifier you use to access Snowsight.

Setting up authentication¶

Authenticating to the Cortex LLM REST API uses key-pair authentication. This requires creating an RSA key pair and assigning its public key to a user, which must be done using the SECURITYADMIN role (or another role that has had SECURITYADMIN granted, such as ACCOUNTADMIN). For step-by-step instructions, see Configuring key-pair authentication.

To make API requests, use the public key to create a JSON Web token (https://jwt.io/) (JWT) and pass it in the headers of the request.

Setting up authorization¶

Once you have created a key pair and assigned its public key to a user, that user’s default role needs to have the snowflake.cortex_user database role, which contains the privileges to use the LLM functions. In most cases, users already have this privilege, because it is granted to the PUBLIC role automatically, and all roles inherit PUBLIC.

If your Snowflake administrator prefers to opt in individual users, he or she might have revoked snowflake.cortex_user from PUBLIC, and must grant this role to the users who should be able to use the Cortex LLM REST API as follows.

GRANT DATABASE ROLE snowflake.cortex_user TO ROLE MY_ROLE;
GRANT ROLE MY_ROLE TO USER MY_USER;

ALTER USER MY_USER SET DEFAULT_ROLE=MY_ROLE

Submitting requests¶

You make a request to the Cortex LLM REST API by POSTing to the API’s REST endpoint. The Authorization header must contain a JSON Web token generated from your public key, which you can do using snowsql via the following command. The generated JWT expires after one hour.

snowsql -a <account_identifier> -u <user> --private-key-path <path>/rsa_key.p8 --generate-jwt

The body of the request is a JSON object that specifies the model, the prompt or conversation history, and options. See the following API Reference for details.

API Reference¶

Example¶

The following example uses curl to make a COMPLETE request. Replace jwt, prompt, and account_identifier with the appropriate values in this command.

curl -X POST \
    -H 'X-Snowflake-Authorization-Token-Type: KEYPAIR_JWT' \
    -H "Authorization: Bearer <jwt>" \
    -H 'Content-Type: application/json' \
    -H 'Accept: application/json, text/event-stream' \
    -d '{
    "model": "mistral-large",
    "messages": [
        {
            "content": "<prompt>"
        }
    ],
    "top_p": 0,
    "temperature": 0
    }' \
https://<account_identifier>.snowflakecomputing.cn/api/v2/cortex/inference:complete

data: {
data:  "id": "65c5e2ac-529b-461e-8a8c-f80655e6bd3f",
data:  "created": 1723493954,
data:  "model": "mistral-7b",
data:  "choices": [
data:    {
data:      "delta": {
data:        "content": "Cor"
data:        }
data:      }
data:     ],
data:  "usage": {
data:    "prompt_tokens": 57,
data:    "completion_tokens": 1,
data:    "total_tokens": 58
data:  }
data: }

data: {
data:  "id": "65c5e2ac-529b-461e-8a8c-f80655e6bd3f",
data:  "created": 1723493954,
data:  "model": "mistral-7b",
data:  "choices": [
data:    {
data:      "delta": {
data:        "content": "tex"
data:        }
data:      }
data:     ],
data:  "usage": {
data:    "prompt_tokens": 57,
data:    "completion_tokens": 2,
data:    "total_tokens": 59
data:  }
data: }

Python API¶

To install the Python API, use:

pip install snowflake-ml-python

The Python API is included in the snowflake-ml-python package starting with version 1.6.1.

from snowflake.snowpark import Session
from snowflake.cortex import Complete

session = Session.builder.configs(...).create()

stream = Complete(
  "mistral-7b",
  "What are unique features of the Snowflake SQL dialect?",
  session=session,
  stream=True)

for update in stream:
  print(update)