类别：

:doc:`/sql-reference/functions-string`（AI 函数）

AI_COUNT_TOKENS

备注

AI_COUNT_TOKENS 是 COUNT_TOKENS (SNOWFLAKE.CORTEX) 的更新版本。要获得最新功能，请使用 AI_COUNT_TOKENS。

Returns an estimate of the number of tokens in a prompt for the specified large language model or task-specific function. For functions that can take additional inputs that affect token count, such as model name or categories/labels, those inputs can also be specified.

语法

The syntax can vary based on the function used. In general, you pass the function name, model name if applicable, input text, and any additional options that affect token count.

AI_COUNT_TOKENS(<function_name>, <input_text> )
AI_COUNT_TOKENS( <function_name>, <model_name> , <input_text> )
AI_COUNT_TOKENS( <function_name>, <input_text>, <options> )
AI_COUNT_TOKENS( <function_name>, <model_name>, <input_text>, <options> )

AI_COUNT_TOKENS uses specific syntax variations for some functions. For example:

AI_COUNT_TOKENS( 'ai_similarity', <input_text_1>, <input_text_2>, <options> )
AI_COUNT_TOKENS( 'ai_classify', <input_text>, <categories> )
AI_COUNT_TOKENS( 'ai_translate', <input_text>, <source_language>, <target_language> )

See Examples for function specific usage patterns.

实参

必填：

function_name

包含要作为词元计数基础的函数名称的字符串，例如 'ai_complete' 或 'ai_sentiment'。函数的名称必须以“ai_”开头，并且只能使用小写字母。

可用性 表提供了支持的函数的完整列表。

input_text or input_text_1, input_text_2

输入文本，计算其中的令牌。

可选：

model_name

包含要作为词元内容基础的模型名称的字符串。如果 function_name 指定的函数要求您选择要使用的模型（例如 AI_COMPLETE 或 AI_EMBED），则为必需项。

A list of available LLM models is available in the 可用性 table. However, not all models are currently supported. Snowflake intends to add support for additional models over time.

AI_COMPLETE 不支持以下模型：

  • claude-4-opus

  • claude-4-sonnet

  • claude-3-7-sonnet

  • claude-3-5-sonnet

  • openai-gpt-4.1

  • openai-o4-mini

categories

VARIANT 值的数组，为需要此数据的函数指定要使用的一个或多个类别或标签。类别计入输入词元计数。

options

A VARIANT that specifies additional options that affect how the function processes the input. For functions that take two text inputs, such as AI_SIMILARITY, options are used to specify the model.

返回

INTEGER 值，使用给定参数值计算的输入文本词元数。

使用说明

  • Although function names are usually written in all uppercase, use only lowercase letters in function and model names.

  • COUNT_TOKENS does not work with LLM functions in the SNOWFLAKE.CORTEX namespace or with fine-tuned models. You must specify a function name that begins with "ai_".

  • COUNT_TOKENS accepts only text, not image, audio, or video inputs.

  • COUNT_TOKENS only incurs compute costs and does not bill based on token count.

  • 即使某些模型在特定区域不可用，COUNT_TOKENS 也可在所有区域使用。

示例

AI_COMPLETE 示例

以下 SQL 语句计算 AI_COMPLETE 和 llama3.3-70b 模型提示中的词元数：

SELECT AI_COUNT_TOKENS('ai_complete', 'llama3.3-70b', 'Summarize the insights from this
call transcript in 20 words: "I finally splurged on these after months of hesitation about
the price, and I\'m mostly impressed. The Nulu fabric really is as buttery-soft as everyone says,
and they\'re incredibly comfortable for yoga and lounging. The high-rise waistband stays put
and doesn\'t dig in, which is rare for me. However, I\'m already seeing some pilling after
just a few wears, and they definitely require gentle care. They\'re also quite delicate -
I snagged them slightly on my gym bag zipper. Great for low-impact activities, but I wouldn\'t
recommend for high-intensity workouts. Worth it for the comfort factor"');

响应：

158

AI_EMBED 示例

以下 SQL 语句计算使用 AI_EMBED 函数和 nv-embed-qa-4' 模型嵌入的文本中的词元数：

SELECT AI_COUNT_TOKENS('ai_embed', 'nv-embed-qa-4', '"I finally splurged on these after months
of hesitation about the price, and I\'m mostly impressed. The Nulu fabric really is as buttery-soft
as everyone says, and they\'re incredibly comfortable for yoga and lounging. The high-rise waistband
stays put and doesn\'t dig in, which is rare for me. However, I\'m already seeing some pilling after
just a few wears, and they definitely require gentle care. They\'re also quite delicate - I snagged
them slightly on my gym bag zipper. Great for low-impact activities, but I wouldn\'t recommend for
high-intensity workouts. Worth it for the comfort factor"');

响应：

142

AI_CLASSIFY 示例

此示例计算使用给定输入和标签进行文本分类所需的输入词元总数：

SELECT AI_COUNT_TOKENS('ai_classify',
  'One day I will see the world and learn to cook my favorite dishes',
  [
      {'label': 'travel'},
      {'label': 'cooking'},
      {'label': 'reading'},
      {'label': 'driving'}
  ]
);

响应：

187

以下示例在上一个示例的基础上添加了每个标签的描述和总体任务描述：

SELECT AI_COUNT_TOKENS('ai_classify',
  'One day I will see the world and learn to cook my favorite dishes',
  [
    {'label': 'travel', 'description': 'content related to traveling'},
    {'label': 'cooking','description': 'content related to food preparation'},
    {'label': 'reading','description': 'content related to reading'},
    {'label': 'driving','description': 'content related to driving a car'}
  ],
  {
    'task_description': 'Determine topics related to the given text'
  };

响应：

254

以下示例在前两个示例的基础上添加了标签示例：

SELECT AI_COUNT_TOKENS('ai_classify',
  'One day I will see the world and learn to cook my favorite dishes',
  [
    {'label': 'travel', 'description': 'content related to traveling'},
    {'label': 'cooking','description': 'content related to food preparation'},
    {'label': 'reading','description': 'content related to reading'},
    {'label': 'driving','description': 'content related to driving a car'}
  ],
  {
    'task_description': 'Determine topics related to the given text',
    'examples': [
      {
        'input': 'i love traveling with a good book',
        'labels': ['travel', 'reading'],
        'explanation': 'the text mentions traveling and a good book which relates to reading'
      }
    ]
  }
);

响应：

298

AI_SENTIMENT 示例

以下 SQL 语句计算使用 AI_SENTIMENT 函数对其进行情绪分析的文本中的词元数量：

SELECT AI_COUNT_TOKENS('ai_sentiment',
  'This place makes the best truffle pizza in the world! Too bad I cannot afford it');

响应：

139

以下示例为上一个示例添加了标签：

SELECT AI_COUNT_TOKENS('ai_sentiment',
  'This place makes the best truffle pizza in the world! Too bad I cannot afford it',
  [
    {'label': 'positive'},
    {'label': 'negative'},
    {'label': 'neutral'}
  ]
);

响应：

148

AI_SIMILARITY examples

The following SQL statement counts the number of tokens in an AI_SIMILARITY call that uses the default model.

SELECT AI_COUNT_TOKENS('ai_similarity',
  'The plot is fast and the characters feel real. This book kept me awake all night
  because the mystery is so deep. I love how the author  handles the ending. It is a
  great read for anyone who likes suspense.',
  'The story is quick and the people feel true. This novel kept me awake all night
  because the puzzle is so big. I love how the writer handles the finale. It is a
  solid choice for anyone who enjoys suspense.');

响应：

101

The following SQL statement counts the number of tokens in an AI_SIMILARITY that uses the e5-base-v2 model:

SELECT AI_COUNT_TOKENS('ai_similarity',
  'The plot is fast and the characters feel real. This book kept me awake all night
  because the mystery is so deep. I love how the author handles the ending. It is a
  great read for anyone who likes suspense.',
  'The story is quick and the people feel true. This novel kept me awake all night
  because the puzzle is so big. I love how the writer handles the finale. It is a
  solid choice for anyone who enjoys suspense.', {'model': 'e5-base-v2'})

响应：

92

AI_TRANSLATE example

The following SQL statement counts the number of tokens used by AI_TRANSLATE when translating text from English to German.

SELECT AI_COUNT_TOKENS('ai_translate',
  'The plot is fast and the characters feel real. This book kept me awake all night
  because the mystery is so deep. I love how the author handles the ending. It is a
  great read for anyone who likes suspense.', 'en', 'de');

响应：

51