What is Change Data Capture (CDC)?
What is Change Data Capture (CDC)?
About Remac: Remac is a high-performance, database-agnostic transaction log processor. It turns native database transaction logs into governed data streams, replicas, audit records, recovery data, and replayable events.
CDC is one of the six functions that Remac builds on this transaction-log foundation.
In the broader category, Change Data Capture detects database changes and delivers them downstream continuously or incrementally. Its latency depends on the capture method and the complete delivery path.
In Remac's product model, CDC converts selected committed changes into row-level events and delivers them to configured downstream sinks in real time.
A row is inserted, updated, or deleted in PostgreSQL. A CDC system detects the selected change and delivers a record to a downstream system under its configured latency and delivery policy.
Why CDC Exists
Data rarely stays in one place. The database that stores your orders also needs to feed your analytics dashboard, your search index, your recommendation engine, and maybe a second database in a different region. The question is how to keep all of these in sync.
The traditional answer was batch ETL: a scheduled job runs every hour or night, queries the source database for recent changes, transforms them, and loads them into the destination. Freshness can approach the full schedule interval plus job runtime. Broad scans add source work. Ordinary timestamp polling does not observe a deleted row, and table-oriented results do not preserve a source transaction boundary by themselves.
CDC can deliver selected changes after commit and can include deletes, positions, and transaction metadata. The available data depends on the capture method and source configuration. For the larger category, see What is Transaction Log Processing.
Three Ways to Capture Changes
Three common CDC approaches offer different data coverage, source costs, and operating requirements.
Timestamp-Based (Query Polling)
Timestamp polling is the simplest approach. Add an updated_at column to every table, update it on every write, then periodically query for rows where updated_at > last_poll_time.
SELECT * FROM orders WHERE updated_at > '2025-06-10 14:00:00';
This method uses ordinary query access and does not require replication privileges. It cannot see a hard-deleted row unless another mechanism records the deletion. Several updates between polls also collapse into the final visible row state.
Reliable polling usually requires a tracking column on every selected table and a stable cursor rule. Equal timestamps, late commits, clock sources, and a crash between reading and saving the cursor can create gaps or repeats when the query relies only on updated_at > last_poll_time.
Each poll adds source query load. The result also lacks source transaction context, so rows changed by one commit normally arrive as independent query results.
Timestamp polling can fit a bounded or append-oriented workload where the cursor, delete policy, and freshness target are explicit.
Trigger-Based
Database triggers fire custom procedures on every INSERT, UPDATE, or DELETE. In a trigger-based CDC setup, each trigger writes the change to a separate changelog table, which downstream consumers then read.
CREATE TRIGGER capture_order_changes
AFTER INSERT OR UPDATE OR DELETE ON orders
FOR EACH ROW
EXECUTE FUNCTION log_change_to_changelog();
This method can capture inserts, updates, and deletes on tables with the required triggers. End-to-end delay depends on how consumers read the changelog table. Every source change also writes and maintains additional data inside the source transaction, so the measured cost depends on the trigger, schema, indexes, transaction size, storage, and workload.
The trigger function and event format are coupled to the source schema and database engine. Schema changes can require maintenance across many trigger objects. The triggers record configured data changes, but a table rename, column type change, or index modification does not appear as an ordinary changelog row.
Trigger-based CDC can still fit a database that does not expose a suitable log interface. The team must accept its write-path and maintenance costs.
Log-Based
Log-based CDC uses a database change interface derived from the transaction log. It does not poll application tables or require a trigger for each captured write. The source must still be configured to expose enough logical information. For PostgreSQL, this includes wal_level = logical.
A correctly configured source can expose selected inserts, updates, deletes, and transaction markers. DDL coverage depends on the database and capture path. The reader uses the engine's log or change protocol instead of repeated table queries or a trigger write for every captured row.
The source can also identify changes that belong to one transaction. The processor and sink still determine whether the destination applies those changes atomically. This path continues to use source CPU, storage, network, retention, and replication resources. Its measured delivery delay depends on load, batching, network, sink behavior, and acknowledgment policy.
Log-based CDC requires source configuration and privileges. Examples include wal_level = logical in PostgreSQL, row-based binary logging in MySQL, and a replica set or sharded cluster for MongoDB change streams. Operators must also control log retention, source positions, delivery, and failure.
Comparison
| Timestamp-Based | Trigger-Based | Log-Based | |
|---|---|---|---|
| Captures deletes | Not hard deletes unless deletion is represented separately | Yes | Yes |
| Transaction context | Usually no | Requires custom design | Available when the source exposes it |
| Source database impact | Query work on each poll | Added transaction and changelog work | Log retention, decoding, network, and replication work |
| Latency | Polling interval | Trigger plus changelog consumption | Depends on pipeline and sink |
| Source changes required | Often a tracking column or comparable cursor | Trigger objects and an event table | Logical log or change-stream configuration |
| Handles DDL changes | No, not from row polling alone | No, not from DML triggers alone | Database- and decoder-specific |
| Typical operating work | Poll scheduling and cursor correctness | Trigger and event-table maintenance | Position, retention, decoding, and source configuration |
| Typical fit | Bounded or append-oriented polling | Systems where log access is unavailable | Continuous change capture from a supported log interface |
What a CDC Event Looks Like
The following examples show a normalized CDC event model when the capture path provides the required operation and row-image data. In log-based CDC, an output plugin or decoder can produce this information from the transaction log.
An INSERT:
{
"operation": "INSERT",
"table": "orders",
"timestamp": "2025-06-10T14:30:22Z",
"after": {
"id": 1001,
"customer_id": 42,
"total": 89.99,
"status": "pending"
}
}
An UPDATE (with before and after):
{
"operation": "UPDATE",
"table": "orders",
"timestamp": "2025-06-10T14:35:10Z",
"before": {
"id": 1001,
"status": "pending"
},
"after": {
"id": 1001,
"status": "shipped"
}
}
A DELETE:
{
"operation": "DELETE",
"table": "orders",
"timestamp": "2025-06-10T15:00:00Z",
"before": {
"id": 1001,
"customer_id": 42,
"total": 89.99,
"status": "shipped"
}
}
The timestamp field is illustrative. A production event must state whether its time represents the source change, source commit, or processing time.
The before image shows what the row looked like before the change. The after image shows what it looks like now. For deletes, there is no after because the row no longer exists. For inserts, there is no before because the row didn't exist yet.
How much detail the before image contains depends on the database and its configuration. MySQL row-based logging uses binlog_row_image = FULL by default. In PostgreSQL, the default replica identity uses the primary key when one exists. A DELETE can carry old key data, while an UPDATE carries old key data when the protocol needs it. A complete old row requires REPLICA IDENTITY FULL. The examples here assume complete before images are available.
When the source provides the required images, this structure makes CDC useful beyond simple replication. Downstream consumers can compute differences, react to selected field transitions, or provide change data to an audit system. The source image and metadata boundaries still determine what the consumer can prove.
Common Use Cases
Real-time analytics. Stream selected changes from a transactional database to an analytical system. Freshness depends on the full source, pipeline, and sink path.
Search index synchronization. Send product and content changes to an indexing path without waiting for a scheduled full scan.
Microservice data propagation. A service can consume committed changes without a synchronous read from the source service on each update. The teams still need an owned data contract, schema-change policy, and failure policy.
Audit input. CDC can provide selected row changes with available before-images, after-images, positions, timestamps, and transaction metadata. A compliant audit system must also define retention, access, integrity, identity, and tamper controls. CDC alone does not make a record immutable or compliant.
Cache invalidation. Use a committed database change as the input to a cache refresh or invalidation path. Delivery delay and failure policy still define how long stale entries can remain.
Where to Go from Here
The capture method determines which changes CDC can observe, what context it preserves, and what work it adds to the source.
For the broader picture of how CDC fits into transaction log processing, see What is Transaction Log Processing. For a deep dive into the logs that log-based CDC reads, see What is a Transaction Log.
Join the Remac waitlist for product updates.
External reference note: The external technical information and linked references in this article were current when we published it. External systems, documentation, and defaults can change after publication.
Further Reading:
- What is Change Data Capture?. Confluent's overview of CDC concepts and architecture.
- PostgreSQL Logical Decoding. Official documentation for PostgreSQL logical decoding, output plugins, and replica-identity boundaries.
- PostgreSQL Publications and Replica Identity. Official documentation for publication scope and
UPDATEorDELETEidentity requirements. - MySQL Binary Logging Options. Official MySQL 8.4 documentation for row-based binary logging and row-image settings.
- MongoDB Change Streams. Official documentation for change events and resumability.
- Debezium PostgreSQL Connector. Official connector documentation for event fields, processing timestamps, and PostgreSQL replica identity.