Change data capture (CDC) provides efficient, distributed, row-level changefeeds into a configurable sink for downstream processing such as reporting, caching, or full-text indexing.
What is change data capture?
While CockroachDB is an excellent system of record, it also needs to coexist with other systems. For example, you might want to keep your data mirrored in full-text indexes, analytics engines, or big data pipelines.
The main feature of CDC is the changefeed, which targets an allowlist of tables, called the "watched rows". There are two implementations of changefeeds:
| Core changefeeds | Enterprise changefeeds |
|---|---|
| Useful for prototyping or quick testing. | Recommended for production use. |
| Available in all products. | Available in CockroachDB Dedicated or with an Enterprise license in CockroachDB. |
| Streams indefinitely until underlying SQL connection is closed. | Maintains connection to configured sink. |
Create with EXPERIMENTAL CHANGEFEED FOR. |
Create with CREATE CHANGEFEED. |
| Watches one or multiple tables in a comma-separated list. Emits every change to a "watched" row as a record. | Watches one or multiple tables in a comma-separated list. Emits every change to a "watched" row as a record in a configurable format ( JSON or Avro) to a configurable sink (Kafka). |
CREATE changefeed and cancel by closing the connection. |
Manage changefeed with CREATE, PAUSE, RESUME, and CANCEL, as well as monitor and debug. |
See Ordering Guarantees for detail on CockroachDB's at-least-once-delivery-guarantee as well as explanation on how rows are emitted.
Known limitations
- Changefeeds only work on tables with a single column family (which is the default for new tables). Tracking GitHub Issue
- Changefeeds cannot be backed up or restored. Tracking GitHub Issue
- Changefeeds cannot be altered. To alter, cancel the changefeed and create a new one with updated settings from where it left off. Tracking GitHub Issue
- Changefeed target options are limited to tables. Tracking GitHub Issue
- Using a cloud storage sink only works with
JSONand emits newline-delimited json files. Tracking GitHub Issue - Webhook sinks only support HTTPS. Use the
insecure_tls_skip_verifyparameter when testing to disable certificate verification; however, this still requires HTTPS and certificates. Tracking GitHub Issue - Currently, webhook sinks only have support for emitting
JSON. Tracking GitHub Issue - There is no concurrency configurability for webhook sinks. Tracking GitHub Issue
- Enterprise changefeeds are currently disabled for CockroachDB Serverless (beta) clusters. Core changefeeds are enabled. Tracking GitHub Issue
- Changefeeds will emit
NULLvalues forVIRTUALcomputed columns and not the column's computed value. Tracking GitHub Issue
Yes
No