设置 Openflow Connector for Google Drive¶

先决条件¶

确保您满足以下要求：

• 您有拥有超级管理员权限的 Google 用户

• 您有具有以下角色的 Google Cloud 项目：

  • 组织策略管理员

  • 组织管理员

启用服务账户密钥创建¶

默认情况下，Google 会禁用服务账户密钥创建。要使 Openflow 使用服务账户 JSON，必须关闭此密钥创建策略。

1. 使用具有组织策略管理员角色的超级管理员账户登录 Google Cloud Console (https://console.cloud.google.com/)。

2. 确保您参与的是与组织相关的项目，而不是组织中的项目。

3. 点击 Organization Policies。

4. 选择 Disable service account key creation 策略。

5. 点击 Manage Policy 并关闭强制执行。

6. 点击 Set Policy。

创建服务账户和密钥¶

1. 打开 Google Cloud Console (https://console.cloud.google.com/)，使用被授予服务账户创建访问权限的用户进行身份验证。

2. 确保您参与组织的项目。

3. 在左侧导航栏的 IAM & Admin 下方，选择 Service Accounts 选项卡。

4. 点击 Create Service Account。

5. 输入服务账户名称，然后点击 Create and Continue。

6. 点击 Done。在列出服务账户的表中，找到 OAuth 2 Client ID 列。复制客户端 ID，因为稍后在下一部分中设置全域委派时需要该 ID。

7. 在新建的服务账户上，点击表下方的菜单，其中列出了该服务账户的服务账户，然后选择 Manage keys。

8. 选择 Add key，然后选择 Create new key。

9. 保留默认选择 JSON 并点击 Create。

密钥将以 .json 文件的形式下载到浏览器的 Downloads 目录。

为服务账户授予所列作用域的全域委派¶

1. 登录您的 Google 管理员账户。

2. 从 Google Apps selector 中选择 Admin。

3. 在左侧导航栏中，依次展开 Security 和 Access 并选择 Data control，然后点击 API Controls。

4. 在 API Controls 屏幕上，选择 Manage domain wild delegation。

5. 点击 Add new。

6. 输入从“Create Service Account and Key”部分获取的 OAuth 2 客户端 ID，以及以下作用域：

   • https://www.googleapis.com/auth/drive (https://www.googleapis.com/auth/drive)

   • https://www.googleapis.com/auth/drive.metadata.readonly (https://www.googleapis.com/auth/drive.metadata.readonly)

   • https://www.googleapis.com/auth/admin.directory.group.member.readonly (https://www.googleapis.com/auth/admin.directory.group.member.readonly)

   • https://www.googleapis.com/auth/admin.directory.group.readonly (https://www.googleapis.com/auth/admin.directory.group.readonly)

   • https://www.googleapis.com/auth/drive.file (https://www.googleapis.com/auth/drive.file)

   • https://www.googleapis.com/auth/drive.metadata (https://www.googleapis.com/auth/drive.metadata)

7. 点击 Authorize。

示例设置¶

--The following script assumes you'll need to create all required roles, users, and objects.
--However, you may want to reuse some that are already in existence.

--Create a Snowflake service user to manage the connector
USE ROLE USERADMIN;
CREATE USER <openflow_service_user> TYPE=SERVICE COMMENT='Service user for Openflow automation';

--Create a pair of secure keys (public and private). For more information, see
--key-pair authentication. Store the private key for the user in a file to supply
--to the connector’s configuration. Assign the public key to the Snowflake service user:
ALTER USER <openflow_service_user> SET RSA_PUBLIC_KEY = '<pubkey>';


--Create a role to manage the connector and the associated data and
--grant it to that user
USE ROLE SECURITYADMIN;
CREATE ROLE <openflow_connector_admin_role>;
GRANT ROLE <openflow_connector_admin_role> TO USER <openflow_service_user>;


--The following block is for USE CASE 2 (Cortex connect) ONLY
--Create a role for read access to the cortex search service created by this connector.
--This role should be granted to any role that will use the service
CREATE ROLE <cortex_search_service_read_only_role>;
GRANT ROLE <cortex_search_service_read_only_role> TO ROLE <whatever_roles_will_access_search_service>;

--Create the database the data will be stored in and grant usage to the roles created
USE ROLE ACCOUNTADMIN; --use whatever role you want to own your DB
CREATE DATABASE IF NOT EXISTS <destination_database>;
GRANT USAGE ON DATABASE <destination_database> TO ROLE <openflow_connector_admin_role>;

--Create the schema the data will be stored in and grant the necessary privileges
--on that schema to the connector admin role:
USE DATABASE <destination_database>;
CREATE SCHEMA IF NOT EXISTS <destination_schema>;
GRANT USAGE ON SCHEMA <destination_schema> TO ROLE <openflow_connector_admin_role>;
GRANT CREATE TABLE, CREATE DYNAMIC TABLE, CREATE STAGE, CREATE SEQUENCE, CREATE CORTEX
SEARCH SERVICE ON SCHEMA <destination_schema> TO ROLE <openflow_connector_admin_role>;

--The following block is for CASE 2 (Cortex connect) ONLY
--Grant the Cortex read-only role access to the database and schema
GRANT USAGE ON DATABASE <destination_database> TO ROLE <cortex_search_service_read_only_role>;
GRANT USAGE ON SCHEMA <destination_schema> TO ROLE <cortex_search_service_read_only_role>;

--Create the warehouse this connector will use if it doesn't already exist. Grant the
--appropriate privileges to the connector admin role. Adjust the size according to your needs.
CREATE WAREHOUSE <openflow_warehouse>
WITH
   WAREHOUSE_SIZE = 'MEDIUM'
   AUTO_SUSPEND = 300
   AUTO_RESUME = TRUE;
GRANT USAGE, OPERATE ON WAREHOUSE <openflow_warehouse> TO ROLE <openflow_connector_admin_role>;

Copy

设置连接器¶

作为数据工程师，执行以下任务以安装和配置连接器：

设置连接器¶

作为数据工程师，执行以下任务以安装和配置连接器：

设置连接器¶

作为数据工程师，执行以下任务以安装和配置连接器：

运行流¶

1. 运行流。

   1. 启动进程组。该流将在 Snowflake 中创建所有必需的对象。

   2. 右键点击导入的进程组并选择 Start。

2. 查询 Cortex Search 服务。

SELECT PARSE_JSON(
  SNOWFLAKE.CORTEX.SEARCH_PREVIEW(
    '<application_instance_name>.cortex.search_service',
      '{
        "query": "<your_question>",
         "columns": ["chunk", "web_url"],
         "filter": {"@contains": {"user_emails": "<user_emailID>"} },
         "limit": <number_of_results>
       }'
   )
)['results'] AS results

SELECT PARSE_JSON(
     SNOWFLAKE.CORTEX.SEARCH_PREVIEW(
          '<application_instance_name>.cortex.search_service',
          '{
             "query": "What is my vacation carry over policy?",
             "columns": ["chunk", "web_url"],
             "filter": {"@contains": {"user_emails": "<user_emailID>"} },
             "limit": 1
          }'
     )
 )['results'] AS results

import snowflake.snowpark as snowpark
from snowflake.snowpark import Session
from snowflake.core import Root

def main(session: snowpark.Session):

   root = Root(session)

   # fetch service
   my_service = (root
     .databases["<application_instance_name>"]
     .schemas["cortex"]
     .cortex_search_services["search_service"]
   )

   # query service
   resp = my_service.search(
     query="What is my vacation carry over policy?",
     columns = ["chunk", "web_url"],
     filter = {"@contains": {"user_emails": "<user_emailID>"} },
     limit=1
   )
   return (resp.to_json())

curl --location "https://<account_url>/api/v2/databases/<application_instance_name>/schemas/cortex/cortex-search-services/search_service" \
     --header 'Content-Type: application/json' \
     --header 'Accept: application/json' \
     --header "Authorization: Bearer <CORTEX_SEARCH_JWT>" \
     --data '{
         "query": "What is my vacation carry over policy?",
         "columns": ["chunk", "web_url"],
         "limit": 1
     }'