Outbox Pattern with Debezium www.thoughts-on-java.org Outbox Event - PowerPoint PPT Presentation

Outbox Pattern with Debezium www.thoughts-on-java.org

Outbox Event Router • Goals • Monitor only outboxevent table • Transform events • Remove before state • Define message key • Change routing www.thoughts-on-java.org

Outbox Table www.thoughts-on-java.org

Outbox Event Router curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ "name": "order-outbox-connector", "config": { "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "tasks.max": "1", "database.hostname": "postgres", "database.port": "5432", "database.user": "postgres", "database.password": "postgres", "database.dbname" : "order", "database.server.name": "localhost", "tombstones.on.delete" : "false", "table.whitelist" : "public.outboxevent", "transforms" : "outbox", "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter"} }' www.thoughts-on-java.org

Code Samples www.thoughts-on-java.org

• table.field.event.id Configuration • Default: id • Id of event in outboxevent table • table.field.event.key • Default: aggregateid • Key of event in outboxevent table www.thoughts-on-java.org

• table.field.event.type Configuration • Default: type • Event type in outboxevent table • table.field.event.timestamp • Default: Kafka message timestamp • Event timestamp in outbox event table www.thoughts-on-java.org

• table.field.event.payload Configuration • Default: payload • Payload of event in outboxevent table • table.field.event.payload.id • Default: aggregateid • Payload id in outboxevent table www.thoughts-on-java.org

• table.fields.additional.placement Configuration • Adding extra fields to event format: <column_name>:<header or envelope>:<alias> e.g.: aggregateid:envelope:id • table.field.event.schema.version • Kafka Connect schema version www.thoughts-on-java.org

• route.by.field Configuration • Default: aggregatetype • Column used for routing • route.topic.regex • Default: (?<routedByValue>.*) • Regex to replace topic name www.thoughts-on-java.org

• route.topic.replacement Configuration • Default: outbox.event.${routedByValue} • Name of the topic • route.tombstone.on.empty.payload • Default: false • Shall empty event cause a tombstone event www.thoughts-on-java.org

• debezium.op.invalid.behavior Configuration • Options: warn, error, fatal • Default: warn • If update event occurs, log warning/error or stop process www.thoughts-on-java.org

Customized Event Router curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ "name": "order-outbox-connector", "config": { "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "tasks.max": "1", "database.hostname": "postgres", "database.port": "5432", "database.user": "postgres", "database.password": "postgres", "database.dbname" : "order", "database.server.name": "localhost", "tombstones.on.delete" : "false", "table.whitelist" : "public.outboxevent", "transforms" : "outbox", "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter", "transforms.outbox.route.topic.replacement" : "bookstore.events", "transforms.outbox.table.fields.additional.placement" : "aggregateid:envelope:id"} }' www.thoughts-on-java.org

Code Samples www.thoughts-on-java.org

• Connect outbox table with Apache Kafka • Configurable event transformation Summary • No code necessary • Requires complex infrastructure www.thoughts-on-java.org