Apache Iceberg™ 表的存储¶

Flat layout¶

This section describes the flat layout in Snowflake for data and metadata directories for Snowflake-managed tables.

When you create a Snowflake-managed table that uses the default flat directory layout (PATH_LAYOUT = FLAT), Snowflake writes all Parquet data files under a single data/ directory and all table metadata data files under a single metadata/ directory. Snowflake also writes metadata for 基于 Delta 的表.

Snowflake 使用以下模式构建路径，具体取决于为 BASE_LOCATION 或 BASE_LOCATION_PREFIX 参数指定的值。如果您指定了 BASE_LOCATION，Snowflake 不会使用路径中的 BASE_LOCATION_PREFIX。

其中：

• STORAGE_BASE_URL 是与外部卷关联的活动存储位置的基本 URL。

• 相对于外部卷位置，BASE_LOCATION 是一个目录路径，Snowflake 会在此路径下写入表文件（在 CREATE ICEBERG TABLE 中指定）。基于 Delta 的表需要指定 BASE_LOCATION。

• randomId 是一个随机的、由 Snowflake 生成的 8 个字符的字符串。

** 使用 BASE_LOCATION 整理表存储**

要在同一 STORAGE_BASE_URL 下为多个 Iceberg 表整理存储中的文件，可考虑在 CREATE ICEBERG TABLE 语句中使用表名作为 BASE_LOCATION。这样，Snowflake 就会将数据和元数据写入包含表名的目录中。

例如：

CREATE OR REPLACE ICEBERG TABLE iceberg_table_1 (
  col_1 int,
  col_2 string
)
  CATALOG = 'SNOWFLAKE'
  EXTERNAL_VOLUME = 'iceberg_external_volume'
  BASE_LOCATION = 'iceberg_table_1';

CREATE OR REPLACE ICEBERG TABLE iceberg_table_2 (
  col_1 int,
  col_2 string
)
  CATALOG = 'SNOWFLAKE'
  EXTERNAL_VOLUME = 'iceberg_external_volume'
  BASE_LOCATION = 'iceberg_table_2';

该语句会在外部云存储中产生以下目录结构：

STORAGE_BASE_URL
|-- iceberg_table_1.<randomId>
|   |-- data/
|   |-- metadata/
|-- iceberg_table_2.<randomId>
|   |-- data/
|   |-- metadata/

Hierarchical layout¶

This section describes the hierarchical layout in Snowflake for data and metadata directories for Snowflake-managed tables.

When you create a Snowflake-managed table that uses the default flat directory layout (PATH_LAYOUT = HIERARCHICAL), Snowflake writes all Parquet data files by organizing them in a hierarchical directory structure under the data/ directory that is based on transforms that you define when you create a table. For instructions on how to enable this layout, see Partitioning with hierarchical paths. Snowflake writes all table metadata data files under a single metadata/ directory.

Snowflake 使用以下模式构建路径，具体取决于为 BASE_LOCATION 或 BASE_LOCATION_PREFIX 参数指定的值。如果您指定了 BASE_LOCATION，Snowflake 不会使用路径中的 BASE_LOCATION_PREFIX。

其中：

• STORAGE_BASE_URL 是与外部卷关联的活动存储位置的基本 URL。

• 相对于外部卷位置，BASE_LOCATION 是一个目录路径，Snowflake 会在此路径下写入表文件（在 CREATE ICEBERG TABLE 中指定）。基于 Delta 的表需要指定 BASE_LOCATION。

• randomId 是一个随机的、由 Snowflake 生成的 8 个字符的字符串。

** 使用 BASE_LOCATION 整理表存储**

要在同一 STORAGE_BASE_URL 下为多个 Iceberg 表整理存储中的文件，可考虑在 CREATE ICEBERG TABLE 语句中使用表名作为 BASE_LOCATION。这样，Snowflake 就会将数据和元数据写入包含表名的目录中。

The following example creates customer_region_summary and orders_by_status tables, which each use a hierarchical path layout for their data files based on the following transforms:

• The customer_region_summary table is partitioned by region

• the orders_by_status table is partitioned by order_status

CREATE OR REPLACE ICEBERG TABLE customer_region_summary (
  customer_id int,
  region string
)
  PARTITION BY (region)
  CATALOG = 'SNOWFLAKE'
  EXTERNAL_VOLUME = 'iceberg_external_volume'
  PATH_LAYOUT = HIERARCHICAL
  BASE_LOCATION = 'customer_region_summary';

CREATE OR REPLACE ICEBERG TABLE orders_by_status (
  order_id int,
  order_status string
)
  PARTITION BY (order_status)
  CATALOG = 'SNOWFLAKE'
  EXTERNAL_VOLUME = 'iceberg_external_volume'
  PATH_LAYOUT = HIERARCHICAL
  BASE_LOCATION = 'orders_by_status';

该语句会在外部云存储中产生以下目录结构：

STORAGE_BASE_URL
|-- customer_region_summary.<randomId>
|   |-- data/
|   |   |-- REGION=US/
|   |   |   |-- part-00001-abc123.parquet
|   |   |-- REGION=EU/
|   |       |-- part-00002-def456.parquet
|   |-- metadata/
|
|-- orders_by_status.<randomId>
    |-- data/
    |   |-- ORDER_STATUS=SHIPPED/
    |   |   |-- part-00001-ghi789.parquet
    |   |-- ORDER_STATUS=PENDING/
    |       |-- part-00002-jkl012.parquet
    |-- metadata/

Catalog-linked database: Flat layout¶

This section describes the flat layout for data and metadata directories for externally managed Iceberg tables that you create in a catalog-linked database.

When you create an externally managed table in a catalog-linked database that uses the default flat directory layout (PATH_LAYOUT = FLAT), Snowflake writes all Parquet data files under a single data/ directory and all table metadata data files under a single metadata/ directory.

Snowflake 使用以下模式构建路径，具体取决于为 BASE_LOCATION 或 BASE_LOCATION_PREFIX 参数指定的值。如果您指定了 BASE_LOCATION，Snowflake 不会使用路径中的 BASE_LOCATION_PREFIX。

其中：

• STORAGE_BASE_URL is the base URL for the active storage location associated with your external volume or vended credentials.

• BASE_LOCATION is the path for a directory where Snowflake should write the table files (specified in CREATE ICEBERG TABLE), relative to your external volume location. If you're using catalog-vended credentials, this must be an absolute path that points to an allowed location defined by the remote catalog. Specifying a BASE_LOCATION is required for Delta-based tables.

• randomId 是一个随机的、由 Snowflake 生成的 8 个字符的字符串。

** 使用 BASE_LOCATION 整理表存储**

要在同一 STORAGE_BASE_URL 下为多个 Iceberg 表整理存储中的文件，可考虑在 CREATE ICEBERG TABLE 语句中使用表名作为 BASE_LOCATION。这样，Snowflake 就会将数据和元数据写入包含表名的目录中。

例如：

CREATE OR REPLACE ICEBERG TABLE iceberg_table_1 (
  col_1 int,
  col_2 string
)
  BASE_LOCATION = 's3://my-bucket/customer_iceberg/my_base_location1';

CREATE OR REPLACE ICEBERG TABLE iceberg_table_2 (
  col_1 int,
  col_2 string
)
  BASE_LOCATION = 's3://my-bucket/customer_iceberg/my_base_location2';

该语句会在外部云存储中产生以下目录结构：

STORAGE_BASE_URL
|-- iceberg_table_1.<randomId>
|   |-- data/
|   |-- metadata/
|-- iceberg_table_2.<randomId>
|   |-- data/
|   |-- metadata/

Catalog-linked database: Hierarchical layout¶

This section describes the hierarchical layout for data and metadata directories for externally managed Iceberg tables that you create in a catalog-linked database.

When you create an externally managed Iceberg table in a catalog-linked database that uses the default flat directory layout (PATH_LAYOUT = HIERARCHICAL), Snowflake writes all Parquet data files by organizing them in a hierarchical directory structure under the data/ directory that is based on transforms that you define when you create a table. For instructions on how to enable this layout, see Partitioning with hierarchical paths. Snowflake writes all table metadata data files under a single metadata/ directory.

For externally managed tables with a hierarchical layout, Snowflake writes Parquet data files and table metadata to your external cloud storage. The Parquet data files are organized in a hierarchical directory structure that is based on transforms that you define when you create a table.

Snowflake 使用以下模式构建路径，具体取决于为 BASE_LOCATION 或 BASE_LOCATION_PREFIX 参数指定的值。如果您指定了 BASE_LOCATION，Snowflake 不会使用路径中的 BASE_LOCATION_PREFIX。

其中：

• STORAGE_BASE_URL is the base URL for the active storage location associated with your external volume or vended credentials.

• BASE_LOCATION is the path for a directory where Snowflake should write the table files (specified in CREATE ICEBERG TABLE), relative to your external volume location. If you're using catalog-vended credentials, this must be an absolute path that points to an allowed location defined by the remote catalog. Specifying a BASE_LOCATION is required for Delta-based tables.

• randomId 是一个随机的、由 Snowflake 生成的 8 个字符的字符串。

** 使用 BASE_LOCATION 整理表存储**

要在同一 STORAGE_BASE_URL 下为多个 Iceberg 表整理存储中的文件，可考虑在 CREATE ICEBERG TABLE 语句中使用表名作为 BASE_LOCATION。这样，Snowflake 就会将数据和元数据写入包含表名的目录中。

The following example creates customer_region_summary and orders_by_status tables, which each use a hierarchical path layout for their data files based on the following transforms:

• The customer_region_summary table is partitioned by region

• the orders_by_status table is partitioned by order_status

CREATE OR REPLACE ICEBERG TABLE customer_region_summary (
  customer_id int,
  region string
)
  PARTITION BY (region)
  PATH_LAYOUT = HIERARCHICAL
  BASE_LOCATION = 's3://my-bucket/customer_iceberg/my_base_location1';

CREATE OR REPLACE ICEBERG TABLE orders_by_status (
  order_id int,
  order_status string
)
  PARTITION BY (order_status)
  BASE_LOCATION = 's3://my-bucket/customer_iceberg/my_base_location2';

该语句会在外部云存储中产生以下目录结构：

STORAGE_BASE_URL
|-- customer_region_summary.<randomId>
|   |-- data/
|   |   |-- REGION=US/
|   |   |   |-- part-00001-abc123.parquet
|   |   |-- REGION=EU/
|   |       |-- part-00002-def456.parquet
|   |-- metadata/
|
|-- orders_by_status.<randomId>
    |-- data/
    |   |-- ORDER_STATUS=SHIPPED/
    |   |   |-- part-00001-ghi789.parquet
    |   |-- ORDER_STATUS=PENDING/
    |       |-- part-00002-jkl012.parquet
    |-- metadata/