Your Program as a Transpiler Applying Compiler Design to Everyday - PowerPoint PPT Presentation

Your Program as a Transpiler Applying Compiler Design to Everyday Programming

About Me • Edoardo Vacchi @evacchi • Research @ University of Milan • Research @ UniCredit R&D • Drools and jBPM Team @ Red Hat

Motivation

Motivation • My first task in Red Hat: marshalling backend for jBPM • Data model mapping • From XML tree model to graph representation • Apparently boring, but challenging in a way

Motivation • Language implementation is often seen as a dark art • But some design patterns are simple at their core • Best practices can be applied to everyday programming

Motivation (cont'd) • Learning about language implementation will give you a different angle to deal with many problems • It will lead you to a better understanding of how GraalVM and Quarkus do their magic

Goals • Programs have often a pre-processing phase where you prepare for execution • Then, there's actual process execution phase • Learn to recognize and structure the pre-processing phase

Transpilers

Transpilers vs. Compilers • Compiler : translates code written in a language ( source code ) into code written in a target language ( object code ). The target language may be at a lower level of abstraction • Transpiler : translates code written in a language into code written in another language at the same level of abstraction ( Source-to-Source Translator ).

Are transpilers simpler than compilers? • Lower-level languages are complex • They are not: if anything, they're simple • Syntactic sugar is not a higher-level of abstraction • It is: a concise construct is expanded at compile-time • Proper compilers do low-level optimizations • You are thinking of optimizing compilers .

The distinction is moot • It is pretty easy to write a crappy compiler, call it a transpiler and feel at peace with yourself • Writing a good transpiler is no different or harder than writing a good compiler • So, how do you write a good compiler?

Your Program as a Compiler Applying Compiler Design to Everyday Programming

Compiler-like workflows • At least two classes of problems can be solved with compiler-like workflows • Boot time optimization problems • Data transformation problems

Compiler-like workflows • At least two classes of problems can be solved with compiler-like workflows • Boot time optimization problems • Data transformation problems

Running Example Function Orchestration

Function Orchestration • You are building an immutable Dockerized serverless function g f

Function Orchestration • Problem • No standard * way to describe function orchestration yet g f * Yes, I know about https://github.com/cncf/wg-serverless

process : - edge : target: *_3 source: *_2 - edge : elements : source: *_1 target: *_2 name: End - end : &_3 name: Hello - function : &_2: name: Start - start : &_1 Solution: Roll your own YAML format Hello Start End Congratulations ! Enjoy attending conferences worldwide

Alternate Solution • You are describing a workflow Task 1 Task 2 • There is a perfectly fine standard: BPMN • Business Process Model and Notation

< process id="Minimal" name="Minimal Process"> < startEvent id="_1" name=" Start "/> < scriptTask id="_2" name=" Hello "> <script>System.out.println("Hello World");</script> </scriptTask> < endEvent id="_3" name=" End "> <terminateEventDefinition/> </endEvent> < sequenceFlow id="_1-_2" sourceRef="_1" targetRef="_2"/> < sequenceFlow id="_2-_3" sourceRef="_2" targetRef="_3"/> </process> https://github.com/evacchi/ypaat Hello Start End

Hello Start End Downside: Nobody will invite you at their conference to talk about BPM.

Hello Start End Downside: Nobody will invite you at their conference to talk about BPM. Unless you trick them.

Bonuses for choosing BPMN Start • Standard XML-based serialization format • that's not the bonus • There is standard tooling to validate and parse Hello • that is a bonus • Moreover: • Different types of nodes included in the main spec End • Optional spec for laying out nodes on a diagram

Goals Start • Read a BPMN workflow • Execute that workflow Hello • Visualize that workflow End

Step 1 Recognize your compilation phase

What's a compilation phase? • It's your setup phase. • You do it only once before the actual processing begins

Configuring the application • Problem. Use config values from a file/env vars/etc • Do you validate config values each time you read them ? • Compile-time: • Read config values into a validated data structure • Run-time: • Use validated config values

Data Transformation Pipeline • Problem. Manipulate data to produce analytics • Compile-time: • Define transformations (e.g. map , filter , etc. operations) • Decide the execution plan (local, distributed, etc.) • Run-time: • Evaluate the execution plan

Example: BPMN Execution Start • Problem. Execute a workflow description. • Compile-time: Hello • Read BPMN into a visitable structure ( StartEvent ) • Run-time: • Visit the structure End • For each node, execute tasks

Example: BPMN Visualization Start • Problem. Visualize a workflow diagram. • Compile-time: Hello • Read BPMN into a graph • Run-time: • For each node and edge, draw on a canvas End

TDefinitions tdefs = JAXB.unmarshal( resource, TDefinitions.class); Read BPMN into a Data Structure • Full XML Schema Definition* is automatically mapped onto Java classes, validated against schema constraints * Yes kids, we have working schemas

< process id="Minimal" name="Minimal Process"> < sequenceFlow id="_1-_2" sourceRef=" _1 " targetRef=" _2 "/> < endEvent id="_3" name="End"> <terminateEventDefinition/> </endEvent> < sequenceFlow id="_2-_3" sourceRef="_2" targetRef="_3"/> < scriptTask id=" _2 " name="Hello"> <script>System.out.println("Hello World");</script> </scriptTask> < startEvent id=" _1 " name="Start"/> </process> BPMN: From Tree to Graph • No ordering imposed on the description Forward References

<definitions> </bpmndi:BPMNEdge> <bpmndi:BPMNShape bpmnElement=" _3 "> <dc:Bounds x="396" y="30" width="48" height="48"/> </bpmndi:BPMNShape> <bpmndi:BPMNEdge bpmnElement=" _1-_2 "> <di:waypoint x="35" y="50"/> <di:waypoint x="229" y="50"/> <bpmndi:BPMNEdge bpmnElement=" _2-_3 "> <dc:Bounds x="193" y="30" width="80" height="48"/> <di:waypoint x="229" y="50"/> <di:waypoint x="441" y="50"/> </bpmndi:BPMNEdge> </bpmndi:BPMNPlane> </bpmndi:BPMNDiagram> </definitions> </bpmndi:BPMNShape> <bpmndi:BPMNShape bpmnElement=" _2 "> < process id="Minimal" name="Minimal Process"> </endEvent> < startEvent id=" _1 " name="Start"/> < scriptTask id=" _2 " name="Hello"> <script>System.out.println("Hello World");</script> </scriptTask> < endEvent id=" _3 " name="End"> <terminateEventDefinition/> < sequenceFlow id=" _1-_2 " sourceRef="_1" targetRef="_2"/> </bpmndi:BPMNShape> < sequenceFlow id=" _2-_3 " sourceRef="_2" targetRef="_3"/> </process> <bpmndi:BPMNDiagram> <bpmndi:BPMNPlane bpmnElement="SubProcess"> <bpmndi:BPMNShape bpmnElement=" _1 "> <dc:Bounds x="11" y="30" width="48" height="48"/> https://github.com/evacchi/ypaat Separate Layout Definition