Snowflake Connector for Kafka release notes for 2023¶

Enabled Java Management Extensions (JMX) metrics for Snowpipe Streaming.

Enabled tombstone ingestion for Snowpipe Streaming.

Enabled Snowflake OAuth for Kafka connector with Snowpipe Streaming.

Enabled schematization columns with special or reserved keywords.

Fixed an issue that the one-client configuration option is not enabled by default. The one-client configuration option enable.streaming.client.optimization is now TRUE by default.

Fixed an issue with channel naming.

Improved performance for schematization permission checks when rebalancing.

Fixed a bug that caused missing data in tables due to issues with internal cache clearing during rebalancing.

Snowpipe Streaming with Kafka Connector is now Generally Available.

None.

One client configuration:

• Introduced the enable.streaming.client.optimization option, which is enabled by default.

• With this client optimization, only one client is created for multiple topic partitions per Kafka connector. This feature can reduce client runtime and lower migration cost by creating larger files.

• Note that in a high throughput scenario (for example, 50 MB/s per connector), we recommend that you disable this property if you see an increase in latency or costs.

Permissions and security:

• Unified Snowflake role and user for Snowpipe Streaming for table creation and insertion.

• Upgraded guava dependency to 32.0.1.

• Upgraded Snowpipe Streaming SDK dependency to 2.0.1.

Fixed a wrong result issue that offsets are skipped when schematization is enabled.

Snowpipe Streaming Channels are not closed on rebalance.

Added the ability to use one Streaming Ingest client (Default to false).

Started using the MDC context logger.

Upgraded to the following versions:

• Ingest SDK version 1.1.4

• JDBC version 3.13.30

Fixed an issue related to using the GET command when using the downscoped token on GCP.

Fixed Snowpipe-based KC’s commit offset behavior.