snowflake.snowpark.DataFrame.to_polars

DataFrame.to_polars(*, transport: Literal['arrow', 'parquet'] = 'parquet', max_workers: Optional[int] = None, statement_params: Optional[Dict[str, str]] = None) → polars.DataFrame[source]

Executes the query representing this DataFrame and returns the result as a Polars DataFrame.

The transport is selected by transport:

transport

Description

“parquet”

Parquet unload + eager parallel read (default)

“arrow”

Arrow (full Snowflake type fidelity)
Usage Notes:
  • Transport selection:

    • "parquet" (default): The default transport. Snowflake’s COPY INTO splits the result into multiple Parquet files that are opened and read concurrently. In server-side environments such as stored procedures, each file open is a parallelizable I/O operation, so the concurrent access substantially reduces wall-clock time compared to the Arrow path. On a local client, connection bandwidth is the primary constraint, and the benefit is more modest; for small or medium result sets the COPY INTO setup cost may outweigh the gain, making Arrow the more efficient choice.

    • "arrow": Streams result batches directly from the cursor without staging to disk. Preserves full Snowflake type fidelity. Use when type accuracy is required, or when the result set is small enough that the Parquet path’s COPY INTO overhead is not justified.

  • Parallelism tuning: max_workers controls the thread pool for opening and reading staged Parquet files. Only applies to the "parquet" transport; ignored for "arrow". The default (None) defers to ThreadPoolExecutor.

Example:

>>> df = session.create_dataframe([[1, 2], [3, 4]], schema=["a", "b"])
>>> df.to_polars().shape  
(2, 2)
Parameters:

  • transport – Either "parquet" (default) or "arrow". When "parquet", unload the result to Parquet on the session stage and read the files back in parallel. Can be faster than the "arrow" transport for large, data-transfer-dominated workloads — especially in a stored procedure — but subject to the type-fidelity limits below.

  • max_workers – Maximum number of threads for parallel stage-file opens and reads. Only applies to the "parquet" transport (transport="parquet"). Defaults to None.

  • statement_params – Dictionary of statement level parameters to be set while executing this action.

Note

  1. Requires polars>=1.0.

2. The "parquet" transport does not fully preserve Snowflake types because COPY INTO has restrictions on Parquet unload:

  • TIMESTAMP_LTZ and TIMESTAMP_TZ columns cause the unload to error.

  • TIMESTAMP_NTZ values with sub-millisecond precision (microseconds or nanoseconds) are rounded to milliseconds.

  • FLOAT (double) columns are downcast to float32.

See the COPY INTO <location> usage notes for the full unload type mapping. Use transport="arrow" when full type fidelity is required.