Full lifecycle management for converted Apache Iceberg™ tables (Preview)¶

When you convert an externally managed Apache Iceberg™ table into a table that uses Snowflake as the Iceberg catalog, Snowflake handles full lifecycle management for the converted table. This management includes managing the data and metadata files that were created before you converted the table.

Before the change:

When you convert an externally managed Iceberg table to use Snowflake as the catalog, Snowflake doesn’t delete the data and metadata files that were created for the table before you converted the table.

After the change:

When you convert an externally managed Iceberg table to use Snowflake as the catalog, Snowflake manages deleting the data and metadata files that were created for the table before you converted the table. Snowflake deletes these files when they expire, which is when they exceed their retention period.

The metadata files that Snowflake deletes include metadata.json, manifest files, and manifest list files.

This change affects both existing converted tables and tables that are converted after the change is enabled.

Before the change takes effect, do the following:

1. Check whether any workloads or pipelines rely on any Iceberg files under the table’s storage location that were created before you converted the table. Also, check any Iceberg snapshots and files that are older than the retention period set on the table.

2. If you find any of these files and you still need them outside of Snowflake after the change takes effect, copy them to a separate storage location that you manage.

This change allows Snowflake to manage the full lifecycle for converted Iceberg tables. As a result, this change prevents unnecessary storage consumption resulting from pre-conversion data and metadata files that remain in storage after they expire.

Ref: 2219