ON THE ROAD TO BENCHMARKING BPMN 2.0 WORKFLOW ENGINES Marigianna - PowerPoint PPT Presentation

Architecture, Design and Web Information Systems Engineering Group ON THE ROAD TO BENCHMARKING BPMN 2.0 WORKFLOW ENGINES Marigianna ¡Skouradaki, ¡Dieter ¡H. ¡Roller, ¡Frank ¡Leymann ¡ Vincenzo ¡Ferme , ¡Cesare ¡Pautasso ¡ Institute ¡of ¡Architecture ¡and ¡Application ¡Systems ¡ Faculty ¡of ¡Informatics ¡ University ¡of ¡Stuttgart ¡ University ¡of ¡Lugano ¡(USI) ¡ Germany ¡ Switzerland

Architecture, Design and Web Information Systems Engineering Group What is a Workflow Engine? Workflow Engine Task Dispatcher C Users D B Process Navigator A Core Job Executor Engine Web Service Service Invoker Transaction Manager Instance Persistent Manager Database … DBMS Application Server 2 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group Many Business Process Modeling/Execution Languages YAWL EPC XPDL BPEL PNML BPMN 2002 2004 2008 1992 1998 … 3 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group BPMN 2.0: A Widely Adopted Standard ISO/IEC 19510 BPMN 2.0 BPMN 2.0.2 Jan 2014 Jan 2011 https://en.wikipedia.org/wiki/List_of_BPMN_2.0_engines 4 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group Main Challenges in Benchmarking BPMN 2.0 Workflow Engines 5 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group Main Challenges in Benchmarking BPMN 2.0 Workflow Engines WORKLOAD CHARACTERIZATION BENCHMARK EXECUTION 5 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group Main Challenges in Benchmarking BPMN 2.0 Workflow Engines 20% WORKLOAD CHARACTERIZATION C B 1. Define the Workload Mix 80% D A 2. Define the Load Functions BENCHMARK EXECUTION 5 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group Main Challenges in Benchmarking BPMN 2.0 Workflow Engines 20% WORKLOAD CHARACTERIZATION C B 1. Define the Workload Mix 80% D A 2. Define the Load Functions BENCHMARK EXECUTION Engine ↔ Web Services Client → Engine Engine ↔ Users 3. Deal with engine-specific interfaces x and BPMN 2.0 customizations Instance 4. Asynchronous execution of business processes Database 5. Define meaningful and reliable KPIs 5 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix 6 6 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix Data Flow Control Flow A C B A B Activities Events Execution Behavior Task Types G A B H C D I E F 6 6 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix NUMBER OF REAL-WORLD MODELS NUMBER OF ENGINES SUPPORTING THE FEATURE 7 7 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix NUMBER OF REAL-WORLD MODELS 200 0 NUMBER OF ENGINES SUPPORTING THE FEATURE 7 7 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix NUMBER OF REAL-WORLD MODELS 950 200 0 12 NUMBER OF ENGINES SUPPORTING THE FEATURE 7 7 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix NUMBER OF REAL-WORLD MODELS 950 800 700 600 F 400 200 0 2 4 5 8 10 12 NUMBER OF ENGINES SUPPORTING THE FEATURE 7 7 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 2. Define the Load Functions Events Start C Workflow Users D B Users Engine A Web Start Service Application Server Web Services Instance Database DBMS 8 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 3. Deal with engine-specific interfaces and BPMN 2.0 customizations A C D Workflow B Users Engine Loading Driver Web Service Application Server Instance Database DBMS 9 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 4. Asynchronous execution of processes Start A C D Workflow B Users Engine Loading Driver Web Service Application Server Instance Database DBMS 10 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 4. Asynchronous execution of processes Start A C D Workflow B Users Engine End Loading Driver Web Service Application Server Instance Database DBMS 10 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group The BenchFlow Project “ Design the first benchmark to assess and compare the performance of Workflow Engines that are compliant with Business Process Model and Notation 2.0 (BPMN 2.0) standard ” Engine ↔ Web Services 20% C Client → Engine Engine ↔ Users Instance B Database 80% D A 11 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix C D B A E G F H REAL-WORLD PROCESSES What we need: even more (anonymized) real-world BPMN 2.0 process models 12 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix Skouradaki et al. [SOSE2015] Graph Matching C D a 1 B a 2 A a 3 E G a 5 F a 6 a 4 H REAL-WORLD REOCCURRING STRUCTURES PROCESSES What we need: even more (anonymized) real-world BPMN 2.0 process models 12 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix Skouradaki et al. Selection [SOSE2015] Criteria Graph Matching C D a 1 B a 2 A a 3 E G a 5 F a 6 a 4 H REAL-WORLD REOCCURRING STRUCTURES PROCESSES What we need: even more (anonymized) real-world BPMN 2.0 process models 12 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group 1. Define the Workload Mix Skouradaki et al. Selection [SOSE2015] Criteria Composition Graph Criteria Matching C D a 1 B a 1 a 1 50% a 2 A a 2 a 3 E 50% a 3 a 5 G a 5 F a 6 a 4 H WORKLOAD MIX REAL-WORLD REOCCURRING STRUCTURES PROCESSES What we need: even more (anonymized) real-world BPMN 2.0 process models 12 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group Enabling the Benchmark Execution A C D Workflow B Users Engine Loading Driver Web Service Application Server Instance Database DBMS 13 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme

Architecture, Design and Web Information Systems Engineering Group Enabling the Benchmark Execution harness Workflow Faban Users Engine Faban Drivers Web C Service D B Application Server A Loading Functions Instance Database DBMS 14 Context » BPMN 2.0 Adoption » Why a Benchmark? » Challenges » BenchFlow » Next Steps » Conclusions Vincenzo Ferme