ML 的容器运行时¶
概述¶
ML 的容器运行时是一组预配置的可定制环境,专为 Snowpark Container Services 上的机器学习构建,涵盖交互式实验和批量 ML 工作负载,例如模型训练、超参数调优、批量推理和微调。它们包括最热门的机器学习和深度学习框架。与 Snowflake 笔记本一起使用,可提供端到端的 ML 体验。
执行环境¶
ML 的容器运行时提供了一个环境,其中装有支持 Snowflake 内各种 ML 开发任务的包和库。除了预安装的包外,您还可以从外部来源(如公共 PyPI 存储库)或内部托管的包存储库导入包,这些包存储库提供了经批准可在您组织内部使用的包列表。
自定义 Python ML 工作负载和受支持的训练 APIs 在 Snowpark Container Services 中执行,该服务可在 CPU 或 GPU 计算池上运行。使用 Snowflake ML APIs 时,ML 的容器运行时会将处理任务分配到可用资源中。
分布式处理¶
Snowflake ML 建模和数据加载 APIs 建立在 Snowflake ML 的分布式处理框架之上,通过充分利用可用的计算能力来最大程度地提高资源利用率。默认情况下,该框架在多 GPU 节点上使用所有 GPUs,与开源的包相比,性能显著提高,并缩短了整体运行时间。

机器学习工作负载(包括数据加载)在 Snowflake 管理的计算环境中执行。该框架允许根据当前任务的具体要求动态扩展资源,例如训练模型或加载数据。每个任务的资源数量(包括 GPU 和内存分配)可通过提供的 APIs 轻松配置。
优化的数据加载¶
容器运行时提供了一系列数据连接器 APIs,支持将 Snowflake 数据源(包括表、DataFrames 和 Datasets)连接到流行的 ML 框架(例如 PyTorch 和 TensorFlow),充分利用多个核心或 GPUs。一旦加载,数据可以使用开源包或任何 Snowflake ML APIs(包括下面所述的分布式版本)进行处理。这些 APIs 可以在 snowflake.ml.data
命名空间中找到。
snowflake.ml.data.data_connector.DataConnector
类将 Snowpark DataFrames 或 Snowflake ML Datasets 连接到 TensorFlow 或 PyTorch DataSets 或 Pandas DataFrames。使用以下类方法之一实例化连接器:
DataConnector.from_dataframe
接受 Snowpark DataFrame。
DataConnector.from_dataset
:接受 Snowflake ML 数据集,按名称和版本指定。
DataConnector.from_sources
:接受源列表,每个源可以是 DataFrame 或数据集。
您实例化了连接器(例如,调用实例 data_connector
)之后,请调用以下方法以生成所需类型的输出。
data_connector.to_tf_dataset
:返回适合与 TensorFlow 一起使用的 TensorFlow 数据集。data_connector.to_torch_dataset
:返回适合与 PyTorch 一起使用的 PyTorch 数据集。
有关这些 APIs 的更多信息,请参阅 Snowflake ML API 参考。
使用开源构建¶
凭借预先填充热门 ML 包的基础 CPU 和 GPU 图像以及灵活性,以使用 pip
安装其他库,用户可以在 Snowflake Notebooks 中使用熟悉和创新的开源框架,而无需将数据移出 Snowflake。您可以将用于数据加载、训练和超参数优化的 Snowflake 分布式 APIs 与热门 OSS 包的熟悉 APIs 结合使用,只需对接口进行小型更改以允许扩展配置,从而扩展处理。
以下代码演示如何使用这些 APIs 创建 XGBoost 分类器:
from snowflake.snowpark.context import get_active_session
from snowflake.ml.data.data_connector import DataConnector
import pandas as pd
import xgboost as xgb
from sklearn.model_selection import train_test_split
session = get_active_session()
# Use the DataConnector API to pull in large data efficiently
df = session.table("my_dataset")
pandas_df = DataConnector.from_dataframe(df).to_pandas()
# Build with open source
X = df_pd[['feature1', 'feature2']]
y = df_pd['label']
# Split data into test and train in memory
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15, random_state=34)
# Train in memory
model = xgb.XGBClassifier()
model.fit(X_train, y_train)
# Predict
y_pred = model.predict(X_test)
CPU 容器运行时的包与 GPU 容器运行时的包不同。以下章节列出了每个容器运行时中可用的包。
Snowflake 运行时包¶
Snowflake 运行时 CPU 包¶
以下是可用于 Snowflake ML 运行时 CPU 版本的包。
包 |
版本 |
---|---|
absl-py |
1.4.0 |
aiobotocore |
2.23.2 |
aiohappyeyeballs |
2.6.1 |
aiohttp |
3.12.15 |
aiohttp-cors |
0.8.1 |
aioitertools |
0.12.0 |
aiosignal |
1.4.0 |
altair |
5.5.0 |
annotated-types |
0.7.0 |
anyio |
4.10.0 |
appdirs |
1.4.4 |
arviz |
0.22.0 |
asn1crypto |
1.5.1 |
asttokens |
3.0.0 |
async-timeout |
5.0.1 |
attrs |
25.3.0 |
bayesian-optimization |
1.5.1 |
blinker |
1.9.0 |
boto3 |
1.39.8 |
botocore |
1.39.8 |
cachetools |
5.5.2 |
CausalPy |
0.5.0 |
certifi |
2025.8.3 |
cffi |
1.17.1 |
charset-normalizer |
3.4.3 |
click |
8.2.1 |
clikit |
0.6.2 |
cloudpickle |
3.0.0 |
cmdstanpy |
1.2.5 |
colorama |
0.4.6 |
colorful |
0.5.7 |
comm |
0.2.3 |
cons |
0.4.7 |
contourpy |
1.3.2 |
crashtest |
0.3.1 |
cryptography |
43.0.3 |
cycler |
0.12.1 |
datasets |
4.0.0 |
debugpy |
1.8.16 |
decorator |
5.2.1 |
已弃用 |
1.2.18 |
dill |
0.3.8 |
distlib |
0.4.0 |
etuples |
0.3.10 |
evaluate |
0.4.5 |
exceptiongroup |
1.3.0 |
executing |
2.2.0 |
fastapi |
0.116.1 |
filelock |
3.19.1 |
FLAML |
2.3.6 |
Flask |
3.1.2 |
fonttools |
4.59.2 |
frozenlist |
1.7.0 |
fsspec |
2025.3.0 |
gitdb |
4.0.12 |
GitPython |
3.1.45 |
google-api-core |
2.25.1 |
google-auth |
2.40.3 |
googleapis-common-protos |
1.70.0 |
graphviz |
0.21 |
grpcio |
1.74.0 |
grpcio-status |
1.62.3 |
grpcio-tools |
1.62.3 |
gunicorn |
23.0.0 |
h11 |
0.16.0 |
h5netcdf |
1.6.4 |
h5py |
3.14.0 |
hf-xet |
1.1.9 |
holidays |
0.79 |
httpcore |
1.0.9 |
httpstan |
4.13.0 |
httpx |
0.28.1 |
huggingface-hub |
0.34.4 |
hypothesis |
6.138.7 |
idna |
3.10 |
importlib_metadata |
8.0.0 |
importlib_resources |
6.5.2 |
ipykernel |
6.30.1 |
ipython |
8.37.0 |
itsdangerous |
2.2.0 |
JayDeBeApi |
1.2.3 |
jedi |
0.19.2 |
Jinja2 |
3.1.6 |
jmespath |
1.0.1 |
joblib |
1.5.2 |
jpype1 |
1.6.0 |
jsonschema |
4.25.1 |
jsonschema-specifications |
2025.4.1 |
jupyter_client |
8.6.3 |
jupyter_core |
5.8.1 |
kiwisolver |
1.4.9 |
lightgbm |
4.5.0 |
lightgbm-ray |
0.1.9 |
llvmlite |
0.44.0 |
logical-unification |
0.4.6 |
markdown-it-py |
4.0.0 |
MarkupSafe |
3.0.2 |
marshmallow |
3.26.1 |
matplotlib |
3.10.5 |
matplotlib-inline |
0.1.7 |
mdurl |
0.1.2 |
miniKanren |
1.0.5 |
mlruntimes_service |
1.8.0 |
modin |
0.35.0 |
mpmath |
1.3.0 |
msgpack |
1.1.1 |
multidict |
6.6.4 |
multipledispatch |
1.0.0 |
multiprocess |
0.70.16 |
narwhals |
2.2.0 |
nest-asyncio |
1.6.0 |
networkx |
3.4.2 |
nltk |
3.9.1 |
numba |
0.61.2 |
numpy |
1.26.4 |
nvidia-nccl-cu12 |
2.27.7 |
opencensus |
0.11.4 |
opencensus-context |
0.1.3 |
opentelemetry-api |
1.26.0 |
opentelemetry-exporter-otlp |
1.26.0 |
opentelemetry-exporter-otlp-proto-common |
1.26.0 |
opentelemetry-exporter-otlp-proto-grpc |
1.26.0 |
opentelemetry-exporter-otlp-proto-http |
1.26.0 |
opentelemetry-exporter-prometheus |
0.47b0 |
opentelemetry-proto |
1.26.0 |
opentelemetry-sdk |
1.26.0 |
opentelemetry-semantic-conventions |
0.47b0 |
packaging |
24.2 |
pandas |
2.3.2 |
parso |
0.8.5 |
pastel |
0.2.1 |
patsy |
1.0.1 |
pexpect |
4.9.0 |
pillow |
10.4.0 |
platformdirs |
4.4.0 |
plotly |
6.3.0 |
prometheus_client |
0.22.1 |
prompt_toolkit |
3.0.52 |
propcache |
0.3.2 |
prophet |
1.1.7 |
proto-plus |
1.26.1 |
protobuf |
4.25.8 |
psutil |
7.0.0 |
ptyprocess |
0.7.0 |
pure_eval |
0.2.3 |
py-spy |
0.4.1 |
py4j |
0.10.9.7 |
pyarrow |
21.0.0 |
pyasn1 |
0.6.1 |
pyasn1_modules |
0.4.2 |
pycparser |
2.22 |
pydantic |
2.11.7 |
pydantic-settings |
2.10.1 |
pydantic-core |
2.33.2 |
pydeck |
0.9.1 |
Pygments |
2.19.2 |
PyJWT |
2.10.1 |
pylev |
1.4.0 |
pymc |
5.25.1 |
pyOpenSSL |
25.1.0 |
pyparsing |
3.2.3 |
pysimdjson |
6.0.2 |
pystan |
3.10.0 |
pytensor |
2.31.7 |
python-dateutil |
2.9.0.post0 |
python-dotenv |
1.1.1 |
pytimeparse |
1.1.8 |
pytz |
2025.2 |
PyYAML |
6.0.2 |
pyzmq |
27.0.2 |
ray |
2.47.1 |
referencing |
0.36.2 |
regex |
2025.7.34 |
requests |
2.32.5 |
retrying |
1.4.2 |
rich |
13.9.4 |
rpds-py |
0.27.1 |
rsa |
4.9.1 |
s3fs |
2025.3.0 |
s3transfer |
0.13.1 |
safetensors |
0.6.2 |
scikit-learn |
1.5.2 |
scipy |
1.15.3 |
seaborn |
0.13.2 |
shap |
0.48.0 |
six |
1.17.0 |
slicer |
0.0.8 |
smart_open |
7.3.0.post1 |
smmap |
5.0.2 |
sniffio |
1.3.1 |
snowbooks |
1.76.7rc1 |
snowflake |
1.7.0 |
snowflake-connector-python |
3.17.2 |
snowflake-ml-python |
1.11.0 |
snowflake-snowpark-python |
1.37.0 |
snowflake-telemetry-python |
0.7.1 |
snowflake._legacy |
1.0.1 |
snowflake.core |
1.7.0 |
snowpark-connect |
0.20.3 |
sortedcontainers |
2.4.0 |
sqlglot |
27.9.0 |
sqlparse |
0.5.3 |
stack-data |
0.6.3 |
stanio |
0.5.1 |
starlette |
0.47.3 |
statsmodels |
0.14.5 |
streamlit |
1.39.1 |
sympy |
1.13.1 |
tenacity |
9.1.2 |
threadpoolctl |
3.6.0 |
tokenizers |
0.21.4 |
toml |
0.10.2 |
tomlkit |
0.13.3 |
toolz |
1.0.0 |
torch |
2.6.0+cpu |
torchvision |
0.21.0+cpu |
tornado |
6.5.2 |
tqdm |
4.67.1 |
traitlets |
5.14.3 |
transformers |
4.55.4 |
typing-inspection |
0.4.1 |
typing_extensions |
4.15.0 |
tzdata |
2025.2 |
tzlocal |
5.3.1 |
urllib3 |
2.5.0 |
uvicorn |
0.35.0 |
virtualenv |
20.34.0 |
watchdog |
5.0.3 |
wcwidth |
0.2.13 |
webargs |
8.7.0 |
Werkzeug |
3.1.3 |
wrapt |
1.17.3 |
xarray |
2025.6.1 |
xarray-einstats |
0.8.0 |
xgboost |
2.1.4 |
xgboost-ray |
0.1.19 |
xxhash |
3.5.0 |
yarl |
1.20.1 |
zipp |
3.23.0 |
Snowflake ML Runtime GPU packages¶
以下是可用于 Snowflake ML 运行时 GPU 版本的包。
包 |
版本 |
---|---|
absl-py |
1.4.0 |
accelerate |
1.10.1 |
aiobotocore |
2.23.2 |
aiohappyeyeballs |
2.6.1 |
aiohttp |
3.12.15 |
aiohttp-cors |
0.8.1 |
aioitertools |
0.12.0 |
aiosignal |
1.4.0 |
airportsdata |
20250811 |
altair |
5.5.0 |
annotated-types |
0.7.0 |
anyio |
4.10.0 |
appdirs |
1.4.4 |
arviz |
0.22.0 |
asn1crypto |
1.5.1 |
astor |
0.8.1 |
asttokens |
3.0.0 |
async-timeout |
5.0.1 |
attrs |
25.3.0 |
bayesian-optimization |
1.5.1 |
blake3 |
1.0.5 |
blinker |
1.9.0 |
boto3 |
1.39.8 |
botocore |
1.39.8 |
cachetools |
5.5.2 |
CausalPy |
0.5.0 |
certifi |
2025.8.3 |
cffi |
1.17.1 |
charset-normalizer |
3.4.3 |
click |
8.2.1 |
clikit |
0.6.2 |
cloudpickle |
3.0.0 |
cmdstanpy |
1.2.5 |
colorama |
0.4.6 |
colorful |
0.5.7 |
comm |
0.2.3 |
compressed-tensors |
0.9.3 |
cons |
0.4.7 |
contourpy |
1.3.2 |
crashtest |
0.3.1 |
cryptography |
43.0.3 |
cuda-bindings |
12.9.2 |
cuda-pathfinder |
1.1.0 |
cuda-python |
12.9.2 |
cudf-cu12 |
25.6.0 |
cuml-cu12 |
25.6.0 |
cupy-cuda12x |
13.6.0 |
cuvs-cu12 |
25.6.1 |
cycler |
0.12.1 |
dask |
2025.5.0 |
dask-cuda |
25.6.0 |
dask-cudf-cu12 |
25.6.0 |
datasets |
4.0.0 |
debugpy |
1.8.16 |
decorator |
5.2.1 |
已弃用 |
1.2.18 |
depyf |
0.18.0 |
dill |
0.3.8 |
diskcache |
5.6.3 |
distlib |
0.4.0 |
distributed |
2025.5.0 |
distributed-ucxx-cu12 |
0.44.0 |
distro |
1.9.0 |
dnspython |
2.7.0 |
einops |
0.8.1 |
email-validator |
2.3.0 |
etuples |
0.3.10 |
evaluate |
0.4.5 |
exceptiongroup |
1.3.0 |
executing |
2.2.0 |
fastapi |
0.116.1 |
fastapi-cli |
0.0.8 |
fastapi-cloud-cli |
0.1.5 |
fastrlock |
0.8.3 |
filelock |
3.19.1 |
FLAML |
2.3.6 |
Flask |
3.1.2 |
fonttools |
4.59.2 |
frozenlist |
1.7.0 |
fsspec |
2025.3.0 |
gguf |
0.17.1 |
gitdb |
4.0.12 |
GitPython |
3.1.45 |
google-api-core |
2.25.1 |
google-auth |
2.40.3 |
googleapis-common-protos |
1.70.0 |
graphviz |
0.21 |
grpcio |
1.74.0 |
grpcio-status |
1.62.3 |
grpcio-tools |
1.62.3 |
gunicorn |
23.0.0 |
h11 |
0.16.0 |
h5netcdf |
1.6.4 |
h5py |
3.14.0 |
hf-xet |
1.1.9 |
holidays |
0.79 |
httpcore |
1.0.9 |
httpstan |
4.13.0 |
httptools |
0.6.4 |
httpx |
0.28.1 |
huggingface-hub |
0.34.4 |
hypothesis |
6.138.7 |
idna |
3.10 |
importlib_metadata |
8.0.0 |
importlib_resources |
6.5.2 |
interegular |
0.3.3 |
ipykernel |
6.30.1 |
ipython |
8.37.0 |
itsdangerous |
2.2.0 |
JayDeBeApi |
1.2.3 |
jedi |
0.19.2 |
Jinja2 |
3.1.6 |
jiter |
0.10.0 |
jmespath |
1.0.1 |
joblib |
1.5.2 |
jpype1 |
1.6.0 |
jsonschema |
4.25.1 |
jsonschema-specifications |
2025.4.1 |
jupyter_client |
8.6.3 |
jupyter_core |
5.8.1 |
kiwisolver |
1.4.9 |
lark |
1.2.2 |
libcudf-cu12 |
25.6.0 |
libcuml-cu12 |
25.6.0 |
libcuvs-cu12 |
25.6.1 |
libkvikio-cu12 |
25.6.0 |
libraft-cu12 |
25.6.0 |
librmm-cu12 |
25.6.0 |
libucx-cu12 |
1.18.1 |
libucxx-cu12 |
0.44.0 |
lightgbm |
4.5.0 |
lightgbm-ray |
0.1.9 |
llguidance |
0.7.30 |
llvmlite |
0.44.0 |
lm-format-enforcer |
0.10.12 |
locket |
1.0.0 |
logical-unification |
0.4.6 |
markdown-it-py |
4.0.0 |
MarkupSafe |
3.0.2 |
marshmallow |
3.26.1 |
matplotlib |
3.10.5 |
matplotlib-inline |
0.1.7 |
mdurl |
0.1.2 |
miniKanren |
1.0.5 |
mistral_common |
1.8.4 |
mlruntimes_service |
1.8.0 |
modin |
0.35.0 |
mpmath |
1.3.0 |
msgpack |
1.1.1 |
msgspec |
0.19.0 |
multidict |
6.6.4 |
multipledispatch |
1.0.0 |
multiprocess |
0.70.16 |
narwhals |
2.2.0 |
nest-asyncio |
1.6.0 |
networkx |
3.4.2 |
ninja |
1.13.0 |
nltk |
3.9.1 |
numba |
0.61.2 |
numba-cuda |
0.11.0 |
numpy |
1.26.4 |
nvidia-cublas-cu12 |
12.6.4.1 |
nvidia-cuda-cupti-cu12 |
12.6.80 |
nvidia-cuda-nvcc-cu12 |
12.9.86 |
nvidia-cuda-nvrtc-cu12 |
12.6.77 |
nvidia-cuda-runtime-cu12 |
12.6.77 |
nvidia-cudnn-cu12 |
9.5.1.17 |
nvidia-cufft-cu12 |
11.3.0.4 |
nvidia-curand-cu12 |
10.3.7.77 |
nvidia-cusolver-cu12 |
11.7.1.2 |
nvidia-cusparse-cu12 |
12.5.4.2 |
nvidia-cusparselt-cu12 |
0.6.3 |
nvidia-ml-py |
12.575.51 |
nvidia-nccl-cu12 |
2.21.5 |
nvidia-nvjitlink-cu12 |
12.6.85 |
nvidia-nvtx-cu12 |
12.6.77 |
nvtx |
0.2.13 |
openai |
1.102.0 |
opencensus |
0.11.4 |
opencensus-context |
0.1.3 |
opencv-python-headless |
4.11.0.86 |
opentelemetry-api |
1.26.0 |
opentelemetry-exporter-otlp |
1.26.0 |
opentelemetry-exporter-otlp-proto-common |
1.26.0 |
opentelemetry-exporter-otlp-proto-grpc |
1.26.0 |
opentelemetry-exporter-otlp-proto-http |
1.26.0 |
opentelemetry-exporter-prometheus |
0.47b0 |
opentelemetry-proto |
1.26.0 |
opentelemetry-sdk |
1.26.0 |
opentelemetry-semantic-conventions |
0.47b0 |
opentelemetry-semantic-conventions-ai |
0.4.13 |
outlines |
0.1.11 |
outlines_core |
0.1.26 |
packaging |
24.2 |
pandas |
2.2.3 |
parso |
0.8.5 |
partd |
1.4.2 |
partial-json-parser |
0.2.1.1.post6 |
pastel |
0.2.1 |
patsy |
1.0.1 |
peft |
0.17.1 |
pexpect |
4.9.0 |
pillow |
10.4.0 |
platformdirs |
4.4.0 |
plotly |
6.3.0 |
prometheus-fastapi-instrumentator |
7.1.0 |
prometheus_client |
0.22.1 |
prompt_toolkit |
3.0.52 |
propcache |
0.3.2 |
prophet |
1.1.7 |
proto-plus |
1.26.1 |
protobuf |
4.25.8 |
psutil |
7.0.0 |
ptyprocess |
0.7.0 |
pure_eval |
0.2.3 |
py-cpuinfo |
9.0.0 |
py-spy |
0.4.1 |
py4j |
0.10.9.7 |
pyarrow |
19.0.1 |
pyasn1 |
0.6.1 |
pyasn1_modules |
0.4.2 |
pycountry |
24.6.1 |
pycparser |
2.22 |
pydantic |
2.11.7 |
pydantic-extra-types |
2.10.5 |
pydantic-settings |
2.10.1 |
pydantic-core |
2.33.2 |
pydeck |
0.9.1 |
Pygments |
2.19.2 |
PyJWT |
2.10.1 |
pylev |
1.4.0 |
pylibcudf-cu12 |
25.6.0 |
pylibraft-cu12 |
25.6.0 |
pymc |
5.25.1 |
pynvjitlink-cu12 |
0.7.0 |
pynvml |
12.0.0 |
pyOpenSSL |
25.1.0 |
pyparsing |
3.2.3 |
pysimdjson |
6.0.2 |
pystan |
3.10.0 |
pytensor |
2.31.7 |
python-dateutil |
2.9.0.post0 |
python-dotenv |
1.1.1 |
python-json-logger |
3.3.0 |
python-multipart |
0.0.20 |
pytimeparse |
1.1.8 |
pytz |
2025.2 |
PyYAML |
6.0.2 |
pyzmq |
27.0.2 |
raft-dask-cu12 |
25.6.0 |
rapids-dask-dependency |
25.6.0 |
rapids-logger |
0.1.1 |
ray |
2.47.1 |
referencing |
0.36.2 |
regex |
2025.7.34 |
requests |
2.32.5 |
retrying |
1.4.2 |
rich |
13.9.4 |
rich-toolkit |
0.15.0 |
rignore |
0.6.4 |
rmm-cu12 |
25.6.0 |
rpds-py |
0.27.1 |
rsa |
4.9.1 |
s3fs |
2025.3.0 |
s3transfer |
0.13.1 |
safetensors |
0.6.2 |
scikit-learn |
1.5.2 |
scipy |
1.15.3 |
seaborn |
0.13.2 |
sentencepiece |
0.2.1 |
sentry-sdk |
2.35.1 |
shap |
0.48.0 |
shellingham |
1.5.4 |
six |
1.17.0 |
slicer |
0.0.8 |
smart_open |
7.3.0.post1 |
smmap |
5.0.2 |
sniffio |
1.3.1 |
snowbooks |
1.76.7rc1 |
snowflake |
1.7.0 |
snowflake-connector-python |
3.17.2 |
snowflake-ml-python |
1.11.0 |
snowflake-snowpark-python |
1.37.0 |
snowflake-telemetry-python |
0.7.1 |
snowflake._legacy |
1.0.1 |
snowflake.core |
1.7.0 |
snowpark-connect |
0.20.3 |
sortedcontainers |
2.4.0 |
sqlglot |
27.9.0 |
sqlparse |
0.5.3 |
stack-data |
0.6.3 |
stanio |
0.5.1 |
starlette |
0.47.3 |
statsmodels |
0.14.5 |
streamlit |
1.39.1 |
sympy |
1.13.1 |
tblib |
3.1.0 |
tenacity |
9.1.2 |
threadpoolctl |
3.6.0 |
tiktoken |
0.11.0 |
tokenizers |
0.21.4 |
toml |
0.10.2 |
tomlkit |
0.13.3 |
toolz |
1.0.0 |
torch |
2.6.0+cu126 |
torchaudio |
2.6.0+cu126 |
torchvision |
0.21.0+cu126 |
tornado |
6.5.2 |
tqdm |
4.67.1 |
traitlets |
5.14.3 |
transformers |
4.51.3 |
treelite |
4.4.1 |
triton |
3.2.0 |
typer |
0.16.1 |
typing-inspection |
0.4.1 |
typing_extensions |
4.15.0 |
tzdata |
2025.2 |
tzlocal |
5.3.1 |
ucx-py-cu12 |
0.44.0 |
ucxx-cu12 |
0.44.0 |
urllib3 |
2.5.0 |
uvicorn |
0.35.0 |
uvloop |
0.21.0 |
virtualenv |
20.34.0 |
vllm |
0.8.5.post1 |
watchdog |
5.0.3 |
watchfiles |
1.1.0 |
wcwidth |
0.2.13 |
webargs |
8.7.0 |
websockets |
15.0.1 |
Werkzeug |
3.1.3 |
wrapt |
1.17.3 |
xarray |
2025.6.1 |
xarray-einstats |
0.8.0 |
xformers |
0.0.29.post2 |
xgboost |
2.1.4 |
xgboost-ray |
0.1.19 |
xgrammar |
0.1.18 |
xxhash |
3.5.0 |
yarl |
1.20.1 |
zict |
3.0.0 |
zipp |
3.23.0 |
优化的训练¶
ML 的容器运行时提供一系列分布式训练 APIs(包括 LightGBM、PyTorch 和 XGBoost 的分布式版本),充分利用容器环境中的可用资源。这些可以在 snowflake.ml.modeling.distributors
命名空间中找到。分布式类的 APIs 与标准版本的类似。
有关这些 APIs 的更多信息,请参阅 API 参考。
XGBoost¶
主要 XGBoost 类是 snowflake.ml.modeling.distributors.xgboost.XGBEstimator
。相关类包括:
snowflake.ml.modeling.distributors.xgboost.XGBScalingConfig
要查看与此 API 搭配使用的示例,请参阅 Snowflake ML 容器运行时 GitHub 存储库中的 XGBoost on GPU (https://github.com/Snowflake-Labs/sfguide-getting-started-with-container-runtime-apis/blob/main/XGBoost_on_GPU_Quickstart.ipynb) 示例笔记本。
LightGBM¶
主要 LightGBM 类是 snowflake.ml.modeling.distributors.lightgbm.LightGBMEstimator
。相关类包括:
snowflake.ml.modeling.distributors.lightgbm.LightGBMScalingConfig
要查看与此 API 搭配使用的示例,请参阅 Snowflake ML 容器运行时 GitHub 存储库中的 LightGBM on GPU (https://github.com/Snowflake-Labs/sfguide-getting-started-with-container-runtime-apis/blob/main/LightGBM_on_GPU_Quickstart.ipynb) 示例笔记本。
PyTorch¶
主要 PyTorch 类是 snowflake.ml.modeling.distributors.pytorch.PyTorchDistributor
。相关的类和函数包括:
snowflake.ml.modeling.distributors.pytorch.WorkerResourceConfig
snowflake.ml.modeling.distributors.pytorch.PyTorchScalingConfig
snowflake.ml.modeling.distributors.pytorch.Context
snowflake.ml.modeling.distributors.pytorch.get_context
要查看与此 API 搭配使用的示例,请参阅 Snowflake ML 容器运行时 GitHub 存储库中的 PyTorch on GPU (https://github.com/Snowflake-Labs/sfguide-getting-started-with-container-runtime-apis/blob/main/PyTorch_on_GPU_Quickstart.ipynb) 示例笔记本。
后续步骤¶
要使用 ML 的容器运行时试用笔记本,请参阅 ML 的容器运行时的笔记本。