Snowflake Data Validation - Documentation Index¶
Welcome to the Snowflake Data Validation CLI documentation. The Snowflake Data Validation CLI (snowflake-data-validation or sdv) is a comprehensive command-line tool for validating data migrations between source databases (SQL Server, Teradata, Amazon Redshift, Snowflake) and Snowflake. It provides multi-level validation strategies to ensure data consistency and quality, including Snowflake-to-Snowflake validation for cross-account or cross-region migrations.
Documentation roadmap¶
1. Command References by Database Dialect¶
Choose your source database for dialect-specific commands:
- SQL Server Commands Reference - Complete SQL Server command documentation
- Teradata Commands Reference - Complete Teradata command documentation
- Amazon Redshift Commands Reference - Complete Redshift command documentation
- Snowflake Commands Reference - Complete Snowflake-to-Snowflake command documentation
Each command reference includes:
- Detailed syntax and options for all commands
- Connection configuration specifics
- Complete examples
- Troubleshooting tips
- Best practices for that platform
2. CLI Usage Guide - Comprehensive Reference¶
Start here for complete documentation.
A comprehensive, customer-facing guide covering all aspects of the CLI tool.
Contents:
- Complete installation instructions
- Detailed command reference for all source databases
- In-depth configuration file reference with all options explained
- Complete configuration examples
- Advanced usage patterns
- Troubleshooting guide
Best for:
- First-time users getting started
- Users needing detailed explanations of configuration options
- Troubleshooting issues
- Understanding all available features
3. Quick Reference Guide - Fast Lookup¶
Use this for quick lookups and reminders.
A concise reference guide with essential information in an easy-to-scan format.
Contents:
- Command syntax at a glance
- Quick configuration templates
- Table configuration patterns
- Common CLI options reference
- Performance tips
- Common issues and quick fixes
Best for:
- Experienced users who need quick reminders
- Looking up specific syntax
- Quick configuration templates
- Performance optimization tips
4. Configuration Examples - Ready-to-Use configurations¶
Copy and adapt these real-world examples.
A collection of ready-to-use configuration file examples for various scenarios.
Contents:
- 16+ complete configuration examples
- SQL Server configurations
- Teradata configurations
- Redshift configurations
- Snowflake-to-Snowflake configurations
- Scenario-based examples (dev, staging, production, PII-compliant, etc.)
- Tips for adapting examples
- Security best practices
Best for:
- Jump-starting your configuration
- Finding a configuration similar to your use case
- Learning by example
- Best practices for different scenarios
Quick Navigation by Task¶
The following sections provide quick references to the documentation for specific tasks.¶
Get Started¶
- Follow installation instructions in CLI Usage Guide
- Copy an example from Configuration Examples
- Run your first validation using the Quick Reference
Understand All Options¶
→ CLI Usage Guide - Configuration File Reference
Find a Command¶
→ Quick Reference - Common Commands
Create a Configuration File¶
→ Configuration Examples (pick the closest match to your scenario)
Troubleshoot an Issue¶
→ CLI Usage Guide - Troubleshooting
Optimize Performance¶
→ Quick Reference - Performance Tips
Validate Large Tables¶
→ CLI Usage Guide - Working with Large Tables
Understand Connection Options¶
→ CLI Usage Guide - Connection Configuration
Set Up Validation Levels¶
→ CLI Usage Guide - Validation Configuration
Configure Table-Specific Settings¶
→ CLI Usage Guide - Table Configuration
Configure View Validation¶
→ CLI Usage Guide - View Configuration
View Validation Examples¶
→ Configuration Examples - View Validation
Documentation by Source Database¶
The following sections provide quick references to the documentation for specific source databases.
SQL Server Users¶
Essential Reading:
- SQL Server Commands Reference - Complete command reference
- Quick Reference - SQL Server Connection
- CLI Usage Guide - SQL Server Commands
- Configuration Examples - SQL Server Examples
Key Examples:
- Example 1: Minimal SQL Server Configuration
- Example 2: Production SQL Server with SSL/TLS
- Example 3: SQL Server Incremental Validation
- Example 4: SQL Server with Column Mappings
Teradata Users¶
Essential Reading:
- Teradata Commands Reference - Complete command reference
- Quick Reference - Teradata Connection
- CLI Usage Guide - Teradata Commands
- Configuration Examples - Teradata Examples
Key Examples:
- Example 5: Basic Teradata Configuration
- Example 6: Teradata Large-Scale Migration
- Example 7: Teradata Multi-Schema Validation
Amazon Redshift Users¶
Essential Reading:
- Amazon Redshift Commands Reference - Complete command reference
- Quick Reference - Redshift Connection
- CLI Usage Guide - Redshift Commands
- Configuration Examples - Redshift Examples
Key Examples:
- Example 8: Basic Redshift Configuration
- Example 9: Redshift Data Lake Migration
- Example 10: Redshift with Complex Filtering
Documentation by Use Case¶
Development Environment¶
- Configuration Example 11: Development Environment - Fast Validation
- Quick Reference - Common Commands
Staging Environment¶
- Configuration Example 12: Staging Environment - Comprehensive Testing
- CLI Usage Guide - Advanced Usage
Production Environment¶
- Configuration Example 13: Production - Maximum Performance
- CLI Usage Guide - Working with Large Tables
- Quick Reference - Performance Tips
PII/Compliance Requirements¶
Migration Cutover¶
Continuous/Incremental Validation¶
- Configuration Example 16: Continuous Validation - Daily Incremental
- CLI Usage Guide - Advanced Usage
View Validation¶
Validate database views alongside or separately from tables. Views are materialized into temporary tables for comparison.
- CLI Usage Guide - View Configuration
- Configuration Examples - View Validation
- Quick Reference - View Configuration
Key Features:
- Validate views with the same options as tables (column selection, filtering, column mappings)
- Support for target database/schema/name overrides
- Views are automatically materialized for accurate comparison
Snowflake-to-Snowflake Validation¶
Validate data between different Snowflake accounts, regions, or databases.
Essential Reading:
- Snowflake Commands Reference - Complete command reference
- CLI Usage Guide - Snowflake Commands
- Configuration Examples - Snowflake Examples
Key Features:
- Cross-account validation with separate source and target credentials
- IPC mode for direct connection parameter specification
- Source-only validation with Parquet file export for offline comparison
- Same validation capabilities as other source platforms (schema, metrics, row-level)
Configuration Reference¶
The following sections provide quick references to the documentation for specific configuration scenarios.
Quick Config Template¶
→ Quick Reference - Configuration Template
Complete Field Reference¶
→ CLI Usage Guide - Configuration File Reference
Real-World Examples¶
Common Workflows¶
The following sections provide quick references to the documentation for common workflows.
First-Time Setup Workflow¶
-
Install the CLI
-
Generate configuration template
-
Copy and modify an example
-
Run validation
-
Review results
Troubleshooting Workflow¶
-
Check error message
-
Review configuration
-
Enable debug logging
-
Review logs
-
Adjust configuration
Performance Optimization Workflow¶
-
Review performance tips
-
Enable chunking
-
Adjust thread count
-
Add filters
-
Test with examples
Feature Matrix¶
| Feature | Command Refs | Quick Reference | Usage Guide | Examples |
|---|---|---|---|---|
| Installation | ✓ | ✓✓✓ | ||
| Command Syntax | ✓✓✓ | ✓✓✓ | ✓✓ | |
| Configuration | ✓ | ✓✓ | ✓✓✓ | ✓✓✓ |
| Connection Setup | ✓✓✓ | ✓ | ✓✓✓ | ✓✓✓ |
| Table Config | ✓✓ | ✓✓✓ | ✓✓✓ | |
| View Config | ✓✓ | ✓✓✓ | ✓✓✓ | |
| Validation Levels | ✓ | ✓✓✓ | ✓✓ | |
| Performance | ✓✓✓ | ✓✓ | ✓✓ | |
| Troubleshooting | ✓✓✓ | ✓✓ | ✓✓✓ | |
| Examples | ✓✓ | ✓ | ✓✓ | ✓✓✓ |
Legend: ✓ = Covered, ✓✓ = Good Coverage, ✓✓✓ = Comprehensive Coverage
Learning Path¶
Beginner Path¶
-
Day 1: Understanding the Tool
- Read the Main Project Repository (https://github.com/snowflake-eng/migrations-data-validation)
- Skim CLI Usage Guide - Overview
- Review Quick Reference
-
Day 2: First Validation
- Follow CLI Usage Guide - Quick Start
- Copy Configuration Example 1 or 5 or 8
- Run your first validation
-
Day 3: Configuration Mastery
- Read CLI Usage Guide - Configuration Reference
- Review multiple Configuration Examples
- Customize configuration for your needs
Intermediate Path¶
-
Optimize Performance
-
Advanced Features
-
CI/CD Integration
Expert Path¶
-
Custom Templates
-
Async Workflows
-
Production Deployment
Search Tips¶
Finding Information Quickly¶
For Commands:
- Look in Quick Reference first
- For details, see CLI Usage Guide - CLI Commands
For Configuration:
- Start with Quick Reference - Configuration Template
- For full details, see CLI Usage Guide - Configuration Reference
- For examples, see Configuration Examples
For Errors:
Additional Support¶
If you cannot find what you need in these documents:
Email us at snowconvert-support@snowflake.com