Migrating Data from Amazon Redshift¶

This page covers Amazon Redshift-specific setup for Data migration. For workflow and Worker field definitions, see Data migration configuration reference.

Prerequisites¶

• ODBC driver on Worker hosts. Required for both regular and UNLOAD extraction.
• S3 bucket and IAM role when using UNLOAD extraction: configure unload_s3_bucket, unload_s3_prefix, and unload_iam_role_arn in Worker TOML so Redshift can write Parquet directly to S3. See Using UNLOAD extraction.

Connectivity and extraction strategies¶

Redshift supports two extraction strategies:

Standard authentication¶

[connections.source.redshift]
username = "myuser"
password = "mypassword"
database = "mydatabase"
host = "my-cluster.abcdef123456.us-west-2.redshift.amazonaws.com"
port = 5439
auth_method = "standard"

IAM authentication (provisioned cluster)¶

[connections.source.redshift]
username = "demo-user"
database = "demo_db"
auth_method = "iam-provisioned-cluster"
cluster_id = "my-aws-cluster"
region = "us-west-2"
access_key_id = "your-access-key-id"
secret_access_key = "your-secret-access-key"

Iceberg targets¶

Redshift as a source doesn’t prevent Apache Iceberg™ tables on Snowflake as targets. Set target.tableType to "iceberg" and supply target.icebergConfig. See Data migration configuration reference.

Using UNLOAD extraction¶

UNLOAD is the recommended strategy for large Redshift tables. Instead of streaming result sets through the Worker over ODBC, Redshift writes Parquet files directly to S3. The Worker then stages those files into Snowflake from the external stage, skipping re-upload.

When to use UNLOAD¶

Use UNLOAD when:

• Tables are large enough that ODBC throughput becomes a bottleneck.
• Redshift has an IAM role with write access to an S3 bucket.
• You can create a Snowflake external stage pointing at the same S3 bucket.

Setup¶

1. Grant Redshift permission to write to S3.

Your Redshift cluster’s IAM role must have s3:PutObject and s3:ListBucket permissions on the target bucket.

2. Create a Snowflake external stage pointing at the same S3 path:

CREATE STORAGE INTEGRATION my_s3_integration
  TYPE = EXTERNAL_STAGE
  STORAGE_PROVIDER = 'S3'
  ENABLED = TRUE
  STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::123456789012:role/MySnowflakeRole'
  STORAGE_ALLOWED_LOCATIONS = ('s3china://your-bucket/redshift-unload/');

CREATE OR REPLACE STAGE my_redshift_stage
  URL = 's3china://your-bucket/redshift-unload/'
  STORAGE_INTEGRATION = my_s3_integration
  FILE_FORMAT = (TYPE = 'PARQUET');

3. Add UNLOAD settings to Worker TOML:

[connections.source.redshift]
username = "myuser"
password = "mypassword"
database = "mydatabase"
host = "my-cluster.abcdef123456.us-west-2.redshift.amazonaws.com"
port = 5439
auth_method = "standard"
unload_s3_bucket = "your-bucket"
unload_s3_prefix = "redshift-unload/"
unload_iam_role_arn = "arn:aws:iam::123456789012:role/MyRole"

4. Reference the stage in your workflow YAML:

tables:
  - source:
      databaseName: dev
      schemaName: public
      tableName: large_events
    target:
      databaseName: TARGET_DB
      schemaName: public
      tableName: large_events
    columnNamesToPartitionBy:
      - event_date
    extraction:
      strategy: unload
      externalStage: TARGET_DB.PUBLIC.MY_REDSHIFT_STAGE

Data type mappings¶

Platform-specific suggestions¶

• Use UNLOAD + larger partitionSize targets when UNLOAD produces large Parquet files from wide tables. partitionSize: "auto" works well in most cases.
• When using IAM authentication, confirm the Redshift IAM role has both S3 write access and the correct trust relationship for your cluster.
• For tables migrated via UNLOAD, keep the source cluster accessible during validation. Cloud Data Validation runs SQL against live Redshift even when the original migration used UNLOAD.
• Anti-locking: No automatic hint is added on Redshift. Set queryModifiers only when you need custom source SQL hints. See Anti-locking and query modifiers.

Related content¶

• Data migration
• Validating Data from Amazon Redshift
• Manual Migration: Data migration