AWS VPC interface endpoints for internal stages¶

Benefits¶

Implementing VPC interface endpoints to access Snowflake internal stages provides the following advantages:

• Internal stage data does not traverse the public Internet.

• Client and SaaS applications, such as Microsoft PowerBI, that run outside of the AWS VPC can connect to Snowflake securely.

• Administrators are not required to modify firewall settings to access internal stage data.

• Administrators can implement consistent security and monitoring regarding how users connect to storage accounts.

Limitations¶

For limitations of AWS PrivateLink, see the AWS documentation (https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html#privatelink-limitations).

Prerequisites¶

• Set the ENABLE_INTERNAL_STAGES_PRIVATELINK parameter to enable support for connecting to an internal stage over AWS PrivateLink. For both implementation strategies discussed in this topic, the account administrator must execute:

  use role accountadmin;
  alter account set ENABLE_INTERNAL_STAGES_PRIVATELINK = true;
  

  Copy

• AWS PrivateLink for S3 (https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html).

  Important

  AWS PrivateLink for S3 is an AWS service that must be enabled in your cloud environment.

  For help with configuring and implementing this service, contact your internal AWS administrator.

• Update the firewall allow-listing as follows:

  • If using an outbound firewall, ensure that it allows all the URLs required by Snowflake. For details, see SnowCD (Connectivity Diagnostic Tool).

• For us-east-1 customers only: If using one of the following Snowflake clients to connect to Snowflake, please upgrade to the client version as follows:

  • JDBC driver: 3.13.3 (or higher)

  • ODBC driver: 2.23.2 (or higher)

  • Python Connector for Snowflake: 2.5.1 (or higher)

  • SnowSQL: 1.2.17 (or higher)

    • Upgrade SnowSQL before using this feature. For more information, see Installing SnowSQL.

    • Starting with version 1.3.0, SnowSQL disables automatic upgrades by default to avoid potential issues that can affect production environments when an automatic upgrade occurs. To upgrade, you should download and install new versions manually, preferably in a non-production environment. Snowflake recommends you leave this setting disabled, but you can manually enable the auto-upgrade behavior by configuring the SnowSQL noup option option.

Choosing an implementation strategy¶

Choosing the right implementation strategy depends on whether your organization is using AWS PrivateLink to access a single internal stage or multiple internal stages.

• If your organization is accessing the internal stage of a single account, see Accessing an internal stage with an interface endpoint.

• If your organization is accessing the internal stages of multiple accounts, see Accessing Internal stages with dedicated interface endpoints. This strategy uses multiple interface endpoints to connect, one for each internal stage.

Procedure¶

Complete the following steps to configure and implement secure access to a Snowflake internal stage through a VPC endpoint:

1. As a Snowflake account administrator, execute the following statements in your Snowflake account and record the value defined by the privatelink_internal_stage key. Note that the Amazon S3 bucket name is defined in the first segment of the URL when read from left to right. For more information, see ENABLE_INTERNAL_STAGES_PRIVATELINK and SYSTEM$GET_PRIVATELINK_CONFIG.

   use role accountadmin;
   alter account set ENABLE_INTERNAL_STAGES_PRIVATELINK = true;
   select key, value from table(flatten(input=>parse_json(system$get_privatelink_config())));
   

   Copy

2. As the AWS administrator, create a VPC endpoint for AWS PrivateLink for S3 using the AWS Console. Record the VPCE DNS Name for use in the next step; do not record any VPCE DNS zonal names.

   The VPCE DNS Name can be found by describing an interface endpoint (https://docs.aws.amazon.com/vpc/latest/privatelink/vpce-interface.html#describe-interface-endpoint) once the endpoint is created.

   In this example, a wildcard (i.e. *) is listed as the leading character in the VPCE DNS Name. Replace the leading wildcard with the Amazon S3 bucket name from the previous step. For example:

   Replace:

   *.vpce-000000000000a12-abc00ef0.s3.us-west-2.vpce.amazonaws.com

   With:

   <bucket_name>.vpce-000000000000a12-abc00ef0.s3.us-west-2.vpce.amazonaws.com

3. As the network administrator, update the DNS settings to resolve the following URL:

   <bucket_name>.s3.<region>.amazonaws.com to the VPCE DNS name after the leading wildcard is replaced with the Amazon S3 bucket name.

   In this example, resolve <bucket_name>.s3.<region>.amazonaws.com to <bucket_name>.vpce-000000000000a12-abc00ef0.s3.us-west-2.vpce.amazonaws.com.

   Tip

   • Do not use wildcard characters (i.e. *) with DNS mapping because of the possible impact of accessing other Amazon S3 buckets outside of Snowflake.

   • Use a separate Snowflake account for testing, and configure a private hosted DNS zone in a test VPC to test the feature so that the testing is isolated and does not impact your other workloads.

   • If using a separate Snowflake account is not possible, use a test user to access Snowflake from a test VPC where the DNS changes are made.

   • To test from on-premises applications, use DNS forwarding to forward requests to the AWS private hosted zone in the VPC where the DNS settings are made. If there are client applications in both the VPC and on-premises, use AWS Transit Gateway.

   • Execute the following command from the client machine to verify that the IP address returned is the private IP address for the storage account:

     dig <bucket_name>.s3.<region>.amazonaws.com
     

     Copy

4. For Snowflake accounts in us-east-1, verify your Snowflake clients are on their latest versions.

Benefits¶

The strategy in which an internal stage within an AWS deployment has a dedicated Amazon S3 interface endpoint has the following benefits:

Security:

Each account can have a different security strategy because individual interface endpoints can have different security configurations.

Chargeback models:

Companies can isolate network traffic based on the type of account (for example, production vs. development), and attribute costs associated with data flowing through an endpoint to the correct account.

DNS Management:

The DNS name of an Amazon S3 interface endpoint is a globally unique name that locates the specific endpoint within a specific region. AWS automatically registers this DNS name in its public DNS service, meaning it is publicly resolvable. For these reasons, an administrator does not need to do any additional DNS configuration to route traffic through an Amazon S3 interface endpoint to an internal stage. For example, the administrator does not need to set up a private hosted zone (PHZ) when configuring the Amazon Route 53 DNS service or register a DNS name to point to an endpoint.

Configuration¶

The network isolation strategy consists of the following:

1. In AWS, an administrator creates a new Amazon S3 interface endpoint for every Snowflake account in the organization. For example, if an organization has two accounts in the Snowflake deployment, the administrator creates two interface endpoints.

2. In Snowflake, an administrator uses the S3_STAGE_VPCE_DNS_NAME parameter to associate each Snowflake account with the DNS name of its dedicated interface endpoint. All traffic to the account’s internal stage goes through this interface endpoint.

alter account set S3_STAGE_VPCE_DNS_NAME='*.vpce-sd98fs0d9f8g.s3.us-west2.vpce.amazonaws.com';

Final DNS value¶

The final DNS name associated with an account has the form: <bucketname>.bucket.vpce-<vpceid>.s3.<region>.vpce.amazonaws.com

Where:

• <bucketname> is the name of the internal stage’s Amazon S3 bucket.

• <vpceid> is the unique identifier of the Amazon S3 interface endpoint associated with the account.

• <region> is the cloud region that hosts your Snowflake account.

The final DNS name appears in logs for each driver that connects to the internal stage.