Complex event flows in distributed systems @berndruecker - PowerPoint PPT Presentation

Complex event flows in distributed systems @berndruecker

@berndruecker 3 common hypotheses I check today: # Events decrease coupling # Orchestration needs to be avoided # Workflow engines are painful

Warning: Contains Opinion

Bernd Ruecker Co-founder and Chief T echnologist of Camunda Berlin, Germany mail@berndruecker.io @berndruecker

@berndruecker Simplified example: dash button Photo by 0xF2, available under Creative Commons BY-ND 2.0 license. https://www.flickr.com/photos/0xf2/29873149904/

@berndruecker Three steps …

@berndruecker Who is involved? Some bounded contexts … Checkout Shipment Payment Inventory

@berndruecker (Micro-)services Checkout Shipment Payment Inventory

@berndruecker Autonomous (micro-)services Checkout Dedicated Application Processes Dedicated infrastructure Shipment Payment Dedicated Development Teams Inventory

Events decrease coupling @berndruecker

@berndruecker Example The button blinks if we can ship within 24 hours Checkout Shipment Payment Inventory

@berndruecker Request/response: temporal coupling The button blinks if we can ship within 24 hours Checkout Response Request Shipment Payment Inventory

@berndruecker T emporal decoupling with events and read models The button blinks if we can ship within 24 hours Checkout Read Model Good Good Fetched Stored Shipment Payment Inventory *Events are facts about what happened (in the past)

@berndruecker Event-driven architecture Order Checkout Placed Payment Notification Received Goods Fetched Shipment Payment Inventory

@berndruecker Events can decrease coupling* *e.g. decentral data-management, read models, extract cross-cutting aspects

@berndruecker Peer-to-peer event chains Order placed Checkout Shipment Payment Inventory Goods shipped Payment Goods received fetched

@berndruecker Peer-to-peer event chains Order placed Checkout Shipment Payment Inventory Goods shipped Payment Goods received fetched

@berndruecker The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html

@berndruecker The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html

@berndruecker The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html

@berndruecker Monitoring Workflows Across Microservices https://www.infoq.com/articles/monitor-workflow-collaborating-microservices

T ypical approaches Distributed T racing Data Lake / Event Monitoring Process Mining Process T racking @berndruecker

@berndruecker Stefan Tilkov: Microservice Patterns & Antipatterns - MicroXchg 2018

@berndruecker Peer-to-peer event chains Fetch the goods before the Order payment placed Checkout Shipment Payment Inventory Goods shipped Payment Goods received fetched

@berndruecker Peer-to-peer event chains Fetch the goods before the Order payment placed Checkout Shipment Payment Inventory Goods shipped Payment Goods received fetched

@berndruecker Photo by born1945, available under Creative Commons BY 2.0 license.

@berndruecker What we wanted Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0

@berndruecker https://camunda.com/microservices-orchestration-survey-results-2018/ „ Challenges ?“ Source: Microservices orchestration survey, July 2018, 354 responses

@berndruecker Extract the end-to-end responsibility Order placed Checkout Order Retrieve payment Payment Shipment Payment received Inventory *Commands have an intent about what needs to happen in the future

@berndruecker It is about where to decide about the coupling! Order decides Order placed . to listen to the event Checkout . to issue the command Order Retrieve payment Shipment Payment Inventory

@berndruecker It is about where to decide about the coupling! It can still be messaging! Order placed Checkout Order Retrieve payment Shipment Payment Inventory

@berndruecker Commands help to avoid (complex) peer-to-peer event chains

Orchestration needs to be avoided @berndruecker

@berndruecker Smart ESB-like middleware Order Checkout Order placed Payment received Good Shipment fetched Payment Inventory Good shipped

@berndruecker Dumb pipes Checkout Order Martin Fowler Smart endpoints Shipment Payment and dumb pipes Inventory

@berndruecker Danger of god services? Checkout Order Sam Newmann A few Shipment smart god services Payment tell Inventory anemic CRUD services what to do

@berndruecker Danger of god services? Checkout Order Sam Newmann A few Shipment Payment smart god services Inventory tell anemic CRUD services what to do

@berndruecker A god service is only created by bad API design!

@berndruecker Example Retrieve Payment Order Payment

@berndruecker Example Retrieve Payment Credit Order Payment Card

@berndruecker Example Retrieve Payment Credit Order Payment Card Rejected

@berndruecker Example Retrieve Payment Credit Order Payment Card Rejected If the credit Rejected card was rejected, the customer can provide new details

@berndruecker Example Retrieve Payment Credit Order Payment Card Rejected If the credit Rejected card was rejected, the customer can provide new details Client of dumb endpoints easily become a god services.

@berndruecker Who is responsible to deal with problems? Retrieve Payment Credit Order Payment Card Payment If the credit Rejected card was received Payment rejected, the failed customer can provide new details

@berndruecker Long running services Retrieve Payment Credit Order Payment Card Payment Rejected received Payment failed Smart endpoints are potentially long-running

@berndruecker Handling State Persist thing State machine or (Entity, Actor, …) workflow engine DIY = effort, T ypical Scheduling, Versioning, accidental concerns operating, visibility, complexity scalability , …

@berndruecker Workflow engines are painful Complex, proprietary, heavyweight, central, developer adverse, …

@berndruecker Avoid the wrong tools! Low-code is great! (You can get rid of your developers!) Death by properties panel Complex, proprietary, heavyweight, central, developer adverse, …

@berndruecker Workflow engines, state machines It is relevant in modern architectures

@berndruecker Workflow engines, CADENCE state machines Silicon valley has recognized

@berndruecker Workflow engines, CADENCE state machines

@berndruecker public static void main(String[] args) { ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() .buildProcessEngine(); engine.getRepositoryService().createDeployment() // .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") // .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) What do I mean by .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() „ leightweight ?“ ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }

@berndruecker public static void main(String[] args) { Build engine ProcessEngine engine = new StandaloneInMemProcessEngineConfiguration() in one line of .buildProcessEngine(); code engine.getRepositoryService().createDeployment() // (using in- .addModelInstance("flow.bpmn", Bpmn.createExecutableProcess("flow") // memory H2) .startEvent() .serviceTask("Step1").camundaClass(SysoutDelegate.class) .serviceTask("Step2").camundaClass(SysoutDelegate.class) .endEvent() .done() ).deploy(); engine.getRuntimeService().startProcessInstanceByKey( "flow", Variables.putValue("city", "New York")); } public class SysoutDelegate implements JavaDelegate { public void execute(DelegateExecution execution) throws Exception { System.out.println("Hello " + execution.getVariable("city")); } }