动态表

动态表是根据定义的查询和目标新鲜度自动刷新的表,从而简化了数据转换和管道管理,无需手动更新或进行自定义安排。

When you create a dynamic table, you define a query that specifies how data should be transformed from base objects. Snowflake handles the refresh schedule of the dynamic table and updates the table automatically to reflect the changes made to the base objects based on the query.

关键注意事项与最佳实践

Immutability constraints: Use immutability constraints to let you control dynamic table updates. The constraints keep specific rows static while enabling incremental updates to the rest of the table. They prevent unwanted changes to marked data while they let normal refreshes occur for other parts of the table. For more information, see Understanding immutability constraints.

Performance considerations: Dynamic tables use incremental processing for workloads that support it, which can improve performance by processing only changed data instead of recomputing entire tables. Performance depends on your query patterns and data organization. For guidance on optimizing dynamic table performance, see 动态表性能和优化.

拆分复杂动态表: 将数据管道拆分为更小、更专注的动态表,可以提升性能并简化故障排查。有关更多信息,请参阅 创建动态表的最佳实践

动态表的工作原理

Snowflake 会运行您在 CREATE DYNAMIC TABLE 语句中指定的定义查询,并通过自动刷新流程更新您的动态表。

The following diagram shows how this process computes the changes made to the base objects and merges them into the dynamic table by using compute resources associated with the table.

基础对象和动态表之间的自动刷新过程的可视化表示

目标滞后

Use target lag to set how fresh you want your data to be. Usually, the table data freshness won't be more than that far behind the base table data freshness. With target lag, you control how often the table refreshes and how up-to-date the data stays. Target lag affects refresh frequency and compute costs.

For more information, see 了解动态表目标滞后. For guidance on balancing data freshness with performance, see 优化动态表性能.

Dynamic table refresh

动态表的目标是在您指定的目标滞后内完成刷新。例如,如果目标滞后为五分钟,则动态表中的数据最多比基础表的数据更新落后五分钟。您在创建表时设置刷新模式,此后刷新可以按计划自动执行,也可以手动触发。

For more information, see 了解动态表初始化和刷新 and 手动刷新动态表.

何时使用动态表

Dynamic tables are ideal for the following scenarios:

  • You want to materialize query results without writing custom code.

  • You want to avoid manually tracking data dependencies and managing refresh schedules. Dynamic tables enable you to define pipeline outcomes declaratively, without managing transformation steps manually.

  • You want to chain together multiple tables for data transformations in a pipeline.

  • You don't need fine-grained control over refresh schedules, and you only need to specify a target freshness for the pipeline. Snowflake handles the orchestration of data refreshes, including scheduling and execution, based on your target freshness requirements.

用例示例

  • 缓慢更改维度 (SCDs): 动态表可用于实现类型 1 和类型 2 SCDs,具体方法是读取变更数据流,并对按更改时间戳排序的记录密钥使用窗口函数。此方法可处理无序插入、删除和更新,从而简化了 SCDs 的创建过程。有关详细信息,请参阅 使用动态表实现缓慢变化维度 (link removed)。

  • Joins and aggregations: To enable fast queries, you can use dynamic tables to incrementally precompute slow joins and aggregations. For guidance on optimizing these operators for incremental refresh, see 优化增量刷新查询.

  • 批处理到数据流过渡:动态表支持使用单个 ALTER DYNAMIC TABLE 命令从批处理无缝过渡到流式处理。您可以控制管道中的刷新频率,以平衡成本和数据新鲜度。

语言: 中文