Microsoft SQL Server¶

Overview¶

Horizon Catalog ingests SQL Server metadata via a direct database connection using a dedicated service account with metadata-read privileges. For query logs (used for lineage and popularity) you pick one of three sources, each with a different AWS footprint:

By the time you finish this guide you’ll have:

1. A metadata-read SQL Server user dedicated to Horizon Catalog.
2. Network connectivity between your SQL Server instance and the Horizon Catalog service.
3. A working query-log source: system (default), s3, or none.
4. An active connector in the Horizon Catalog UI that uses all of the above.

Prerequisites¶

• Administrative access to the SQL Server instance so you can CREATE LOGIN, CREATE USER, and GRANT permissions.
• Open Horizon Catalog connector in another browser tab. To reach it, sign in to Snowflake, open Catalog > Connections, pick SQL Server from the list. There are two values you’ll need to paste into any IAM role you create for the s3 log source:
  • IAM Principal: The ARN of the Horizon Catalog service role that will assume your role. Copy it verbatim.
  • External ID: A per-connector secret starting with sf_horizon_ (for example, sf_horizon_Abc123…). Horizon Catalog rejects External IDs that don’t match this prefix. Don’t regenerate or rewrite it.
• Only if you’ll use the DAS (s3) log source: an AWS user with permission to create IAM roles/policies, S3 buckets, Kinesis Data Firehose delivery streams, and Lambda functions in the same AWS account that hosts the RDS for SQL Server instance; plus ability to modify that RDS instance to enable Database Activity Streams.

1. Create a SQL Server user for Horizon Catalog¶

Connect to the instance as an administrative user and create a login + user with metadata-read privileges. Replace <username> and <password> with real values.

CREATE LOGIN <username> WITH PASSWORD = '<password>';
CREATE USER <username> FOR LOGIN <username>;
GRANT VIEW DEFINITION TO <username>;

Then grant VIEW SERVER STATE. The exact syntax depends on the hosting variant:

Self-hosted or AWS RDS for SQL Server¶

USE master;
GRANT VIEW SERVER STATE TO <username>;

Azure SQL Database¶

GRANT VIEW DATABASE PERFORMANCE STATE TO <username>;

Google Cloud SQL for SQL Server¶

Log in as the sqlserver user:

GRANT VIEW SERVER STATE TO <username> AS CustomerDbRootRole;

2. Ensure network connectivity¶

Horizon Catalog connects to your SQL Server instance over the public internet only. The instance must be publicly reachable — either set Publicly accessible = Yes on the RDS instance, or place it behind an internet-facing load balancer or proxy that you control and that forwards to the SQL Server port (default 1433).

Horizon Catalog doesn’t guarantee stable egress IP addresses. The egress IPs it connects from can change at any time without notice. The simplest security-group rule is to allow inbound TCP on the SQL Server port from 0.0.0.0/0 and rely on SQL Server authentication (Step 1) plus TLS to protect the instance.

3. Pick a query-log source¶

Choose one of the following:

• system (default, no AWS work): Horizon Catalog uses the service account from Step 1 to query SQL Server DMVs and system views (for example, sys.dm_exec_query_stats, sys.dm_exec_sql_text) on each crawl. VIEW SERVER STATE granted in Step 1 is what unlocks those views. When you reach Step 4, pick Log source type: system.

• s3 via Database Activity Streams: Full DAS pipeline. See Query Logs for detailed instructions on setting up the Kinesis, Lambda, Firehose, and S3 infrastructure.

• none: Metadata only. Set Log source type: none in Step 4. Lineage and popularity are disabled; metadata sync still works.

4. Finish in the Horizon Catalog UI¶

Back in the Horizon Catalog connector wizard, fill in the connection details:

For s3, also paste the Authorization fields: Role ARN, AWS Region, S3 bucket name, and S3 object prefix processed/.

Select Test connection, then Save, then select the databases and schemas you want crawled.

For the s3 path the test confirms Horizon Catalog could (a) log in to SQL Server as the service user, (b) assume the IAM role, and (c) list and read the bucket. For system, the test only exercises (a).

Initial metadata typically appears within a few minutes; for the s3 path, lineage and popularity fill in as Firehose flushes batches into the bucket (up to 15 minutes per batch by default).

Connection security¶

Horizon Catalog connects to SQL Server with encryption enabled by default (Encrypt=yes). If your SQL Server instance uses a self-signed TLS certificate, the connection still succeeds because server certificate validation is not enforced by default.

For more details on the shared-responsibility model for Horizon Catalog connectors, see Security and data protection.

Troubleshooting¶