CaptureChangeSqlServer 2025.10.2.19¶
捆绑包¶
com.snowflake.openflow.runtime | runtime-database-cdc-processors-nar
描述¶
Reads CDC events from a SQL Server database. The processor periodically queries Change Tracking tables in the database, but only for the tables provided by the TableStateService. The processor maintains a state of the last processed event for each table. The processor moves the position after each processed table. The processor supports multi-threading. The number of threads and connection limit configured in the pool collectively define the upper bound of open connections to the source database. The processor outputs two types of FlowFiles: DDLs, containing the initial schema of a table, and then every time its schema changes, and DMLs, with records representing changes to data in the table. One FlowFile always represents data related to a single table. The DDL with the schema is written to the FlowFile content as a JSON object, in a form such as: { "columns": [ { "name": "<columnName>", "type": "<snowflakeType>", "nullable": <true|false>, "scale": <scale>, "precision": <precision> }, ... ], "primaryKeys": ["<primaryKey1>", "<primaryKey2>", ...] } The DML records are structured as: { "primaryKeys": { "<column>": <value>, ... }, "payload": { "<column>": <value>, ... }, "metadata": { "<column>": <value>, ... }
输入要求¶
FORBIDDEN
支持敏感的动态属性¶
false
属性¶
属性 |
描述 |
|---|---|
列筛选存储 |
存储每个表列筛选设置的服务。 |
连接池 |
连接池 |
提取大小 |
一次加载到内存中的最大行数 |
最大批处理大小 |
单个批次提取的最大行数 |
记录写入器 |
用于序列化 DML 事件的记录写入器 |
表变更查询间隔 |
在安排下一次表变更查询之前必须经过的最短时间间隔。这控制了数据库轮询的频率,以防止过度查询。 |
表状态存储 |
保存复制表状态的共享存储。 |
状态管理¶
范围 |
描述 |
|---|---|
CLUSTER |
该处理器会存储每个表上次处理的记录版本等信息,以便在重新启动时从同一位置继续运行。 |
关系¶
名称 |
描述 |
|---|---|
success |
通过 CDC 流活动成功创建 FlowFile |