About Openflow - Snowflake Deployments¶
Openflow - Snowflake Deployment run on Snowpark Container Services (SPCS) and provide a streamlined and integrated solution for data integration and connectivity across interoperable storage like Iceberg and Snowflake native storage. As a fully self-contained service within Snowflake, it’s easy to deploy and manage, offering a convenient and cost-effective environment for running your data flows. A key advantage is its native integration with Snowflake’s security model, which allows seamless authentication, authorization, and network security, and simplified operations.
尽管客户可同时使用 BYOC 与 Snowflake 部署方案，以下场景尤其适合采用 Snowflake 部署：
- 在 Bronze 层整合全精度数据：将多源原始数据直接导入 Snowflake，并通过 Openflow Snowflake 部署进行提取和加载。
- 数据增强：运行管道以增强 Snowflake 内部现有数据表。
- 一站式数据数据引入到洞察：在 Snowflake 生态系统内构建应用程序，使整个数据生命周期（引入、处理和服务）都在同一平台上进行。
- AI 驱动的原始数据洞察转化：引入非结构化数据后，结合用户的其他结构化数据，运用 Snowflake Intelligence 等技术实现智能搜索与深度解析。
- 使用反向 ETL：通过 APIs、消息传输基础设施等将洞察结果共享给外部运营系统，实现洞察生成的闭环。
Understanding Snowflake roles and External Access Integrations¶
Openflow - Snowflake Deployments must be able to interact with data sources and destinations that are typically outside Snowflake. In addition these deployments must also be able to communicate with and access Snowflake itself. Snowflake roles and external access integrations provide this support.
What is a Snowflake role?¶
A Snowflake role is a traditional Snowflake role, associated with a specific Openflow Runtime, and used for the following tasks:
- Grant access to external access integrations (EAIs). These EAIs specify rules that allow the runtime to access the data sources and destinations from within Snowflake itself.
- 授予对 Snowflake 资源的访问权限。
- 授予对特定连接器资源的访问权限
Snowflake roles are linked to Openflow session tokens, avoiding the need for customers to create separate service users and key pairs for authentication to Snowflake.
什么是 Openflow 中的外部访问集成 (EAI)?¶
An External access integration (EAI) is a Snowflake object designed to provide secure access to external resources, like source systems from which Openflow connectors pull external data. Openflow Snowflake Deployments use EAIs and network rules together to define the endpoints an Openflow connector can read from or write to.
Data engineers define and configure EAIs and Snowflake roles specific to a given connector and its underlying runtime.
Typical Openflow - Snowflake Deployment workflow¶
The following sections describe Openflow - Snowflake Deployment concepts and workflows.
| User persona | Task |
|---|---|
| Snowflake administrator |
See Set up Openflow - Snowflake Deployment - Task overview.
The Openflow UI is used to manage deployments and runtime creation and maintenance. The Openflow UI allows users to create, upgrade, and delete runtimes in all deployments. |
| Data engineer (pipeline author, responsible for data ingestion) |
连接器是一种针对特定集成用例的简便解决方案，技术水平较低的用户也可以在无需数据工程师协助的情况下部署它们。 |
| Data engineer (pipeline operator) | Configures flow parameters and runs the flow. |
| Data engineer (responsible for transformation to silver and gold layers) | Responsible for transforming data from the bronze layer that was populated by the pipeline to silver and gold layers for analytics. |
| Business user | Makes use of gold layer objects for analytics. |
限制
- Openflow - Snowflake Deployment is not supported in trial accounts.
- Only a single Openflow - Snowflake Deployment is supported per account. However, an account can have many Openflow - Snowflake Deployment runtimes — each having a separate role and network access — which allows users to separate the workload.
- Users with a default role of ACCOUNTADMIN can’t login to Openflow - Snowflake Deployment runtimes and will get an error message when attempting to do so.
- Customers requiring private connectivity will need to configure outbound PrivateLink. Private Link is available to Business Critical Edition only.