Fine-grained language composition Edd Barrett Carl Lukas Laurence - PowerPoint PPT Presentation

Fine-grained language composition Edd Barrett Carl Lukas Laurence Naveneetha Friedrich Diekmann Tratt Krishnan Bolz Vasudevan Software Development Team 2015-11-07 1 / 24 http://soft-dev.org/

What to expect from this talk What to expect from this talk A B 2 / 24 http://soft-dev.org/

What to expect from this talk What to expect from this talk gA ∪ Bg 2 / 24 http://soft-dev.org/

What to expect from this talk What to expect from this talk Python ∪ Prolog 2 / 24 http://soft-dev.org/

What to expect from this talk What to expect from this talk Python ∪ PHP 2 / 24 http://soft-dev.org/

Two levels of challenge Two levels of challenge Tooling 3 / 24 http://soft-dev.org/

Two levels of challenge Two levels of challenge Tooling Language friction 3 / 24 http://soft-dev.org/

Tooling challenges Tooling challenges 4 / 24 http://soft-dev.org/

Tooling challenges Tooling challenges Python PyHyp PHP 4 / 24 http://soft-dev.org/

Tooling challenges Tooling challenges syntax Python runtime syntax PyHyp runtime syntax PHP runtime 4 / 24 http://soft-dev.org/

Tooling challenges Tooling challenges syntax Language boxes Python runtime syntax PyHyp runtime syntax PHP runtime 4 / 24 http://soft-dev.org/

Tooling challenges Tooling challenges syntax Language boxes Python runtime syntax PyHyp runtime syntax PHP Composed runtime meta-tracing VMs 4 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... term::= ... | ... | ... func ::= ... PL Y <program> for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... term::= ... | ... | ... func ::= ... Parser PL Y <program> for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... term::= ... | ... | ... func ::= ... Parser PL Y <program> Parse Tree for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... term::= ... | ... | ... func ::= ... LR PL Y <program> Parse Tree for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... term::= ... U | ... n | ... d e func ::= ... f LR i n e PL Y d <program> Parse Tree for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... term::= ... | ... | ... func ::= ... Generalised PL Y <program> Parse Tree for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... A term::= ... m | ... b | ... i g func ::= ... u Generalised o u s PL Y <program> Parse Tree for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... term::= ... | ... | ... func ::= ... PEG PL Y <program> Parse Tree for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

Syntax composition Syntax composition PL X <grammar> expr::= ... term::= ... | ... S | ... h func ::= ... a d PEG o w s PL Y <program> Parse Tree for (j : js) { doStuff(); } . . . 5 / 24 http://soft-dev.org/

The only choice? The only choice? 6 / 24 http://soft-dev.org/

The only choice? The only choice? SDE 6 / 24 http://soft-dev.org/

The challenge The challenge Challenge: SDE’s power + a text editor feel? 7 / 24 http://soft-dev.org/

Eco demo Eco demo 8 / 24 http://soft-dev.org/

Runtime composition Runtime composition 9 / 24 http://soft-dev.org/

Runtime composition Runtime composition PL X PL Y Interpreter Interpreter C/C++ 9 / 24 http://soft-dev.org/

Runtime composition Runtime composition PL X PL Y Too slow Interpreter Interpreter C/C++ 9 / 24 http://soft-dev.org/

Runtime composition Runtime composition JIT Compiler JIT Compiler PL X PL Y Interpreter Interpreter C/C++ 9 / 24 http://soft-dev.org/

Runtime composition Runtime composition JIT Compiler JIT Compiler PL X PL Y Too much engineering Interpreter Interpreter C/C++ 9 / 24 http://soft-dev.org/

Runtime composition Runtime composition PL X PL Y Interpreter Interpreter JVM/CLR JIT Compiler 9 / 24 http://soft-dev.org/

Runtime composition Runtime composition PL X PL Y Semantic mismatch Interpreter Interpreter JVM/CLR JIT Compiler 9 / 24 http://soft-dev.org/

Runtime composition Runtime composition 9 / 24 http://soft-dev.org/

Runtime composition Runtime composition Meta-tracing PL X PL Y PL Z RPython Interpreters Interpreter Tracing JIT Glue 9 / 24 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter 10 / 24 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter translator RPython 10 / 24 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter translator RPython Optimised JIT Interpreter 10 / 24 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter translator RPython Optimised JIT Interpreter 10 / 24 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter translator RPython Optimised JIT Interpreter 10 / 24 http://soft-dev.org/

Adding a JIT to an RPython interpreter Adding a JIT to an RPython interpreter ... pc := 0 while 1: instr := load_next_instruction(pc) if instr == POP: stack.pop() pc += 1 elif instr == BRANCH: off = load_branch_jump(pc) pc += off elif ...: ... Observation: interpreters are big loops. 11 / 24 http://soft-dev.org/

Adding a JIT to an RPython interpreter Adding a JIT to an RPython interpreter ... pc := 0 while 1: jit_merge_point(pc) instr := load_next_instruction(pc) if instr == POP: stack.pop() pc += 1 elif instr == BRANCH: off = load_branch_jump(pc) if off < 0: can_enter_jit(pc) pc += off elif ...: ... Observation: interpreters are big loops. 11 / 24 http://soft-dev.org/

Runtime composition recap Runtime composition recap 12 / 24 http://soft-dev.org/

Runtime composition recap Runtime composition recap Meta-tracing PL X PL Y PL Z RPython Interpreters Interpreter Tracing JIT Glue 12 / 24 http://soft-dev.org/

Runtime composition recap Runtime composition recap Meta-tracing PyPy Hippy RPython PyHyp Interpreters Interpreter Tracing JIT Glue 12 / 24 http://soft-dev.org/

Composed Richards vs. other VMs Composed Richards vs. other VMs Type VM 9.475 ± 0.0127 CPython 2.7.7 4.264 ± 0.0386 HHVM 3.4.0 0.250 ± 0.0008 Mono HippyVM 0.178 ± 0.0006 PyPy 2.4.0 9.070 ± 0.0361 Composed Zend 5.5.13 13 / 24 http://soft-dev.org/

Composed Richards vs. other VMs Composed Richards vs. other VMs Type VM 9.475 ± 0.0127 CPython 2.7.7 4.264 ± 0.0386 HHVM 3.4.0 0.250 ± 0.0008 Mono HippyVM 0.178 ± 0.0006 PyPy 2.4.0 9.070 ± 0.0361 Composed Zend 5.5.13 0.335 ± 0.0012 Composed PyHyp 13 / 24 http://soft-dev.org/

Datatype conversion Datatype conversion PHPRoot PHPObject PHPInt PHPFunc 14 / 24 http://soft-dev.org/

Datatype conversion Datatype conversion PHPRoot PHPObject PHPInt PHPFunc PyRoot PyObject PyInt PyFunc 14 / 24 http://soft-dev.org/

Datatype conversion: primitive types Datatype conversion: primitive types Python PHP 15 / 24 http://soft-dev.org/

Datatype conversion: primitive types Datatype conversion: primitive types Python PHP 2 : PHPInt 15 / 24 http://soft-dev.org/

Datatype conversion: primitive types Datatype conversion: primitive types Python PHP 2 : PyInt 2 : PHPInt 15 / 24 http://soft-dev.org/

Datatype conversion: user types Datatype conversion: user types Python PHP 16 / 24 http://soft-dev.org/

Datatype conversion: user types Datatype conversion: user types Python PHP o : PHPObject 16 / 24 http://soft-dev.org/

Datatype conversion: user types Datatype conversion: user types PyRoot PyObject PyInt PyFunc 16 / 24 http://soft-dev.org/

Datatype conversion: user types Datatype conversion: user types PyRoot PyObject PyInt PyFunc PyPHPAdapter 16 / 24 http://soft-dev.org/

Datatype conversion: user types Datatype conversion: user types PyRoot PyObject PyInt PyFunc PyPHPAdapter php_obj : PHPObject 16 / 24 http://soft-dev.org/

Datatype conversion: user types Datatype conversion: user types Python PHP o : PHPObject 16 / 24 http://soft-dev.org/

Datatype conversion: user types Datatype conversion: user types Python PHP o : PHPObject :PyPHPAdapter 16 / 24 http://soft-dev.org/