Fast enough VMs in fast enough time Laurence Tratt Software - PowerPoint PPT Presentation

Fast enough VMs in fast enough time Laurence Tratt Software Development Team 2016-02-04 1 / 26 http://soft-dev.org/

Language designers dilemma Language designers dilemma 2 / 26 http://soft-dev.org/

Language designers dilemma Language designers dilemma Implementation E ff ort 2 / 26 http://soft-dev.org/

Language designers dilemma Language designers dilemma Less Implementation E ff ort T oo slow 2 / 26 http://soft-dev.org/

Language designers dilemma Language designers dilemma Less More Implementation E ff ort T oo slow Design su ff ers 2 / 26 http://soft-dev.org/

The traditional routes The traditional routes E ffi ciency Di ffi culty 3 / 26 http://soft-dev.org/

The traditional routes The traditional routes E ffi ciency AST interpreter Di ffi culty 3 / 26 http://soft-dev.org/

The traditional routes The traditional routes Translation E ffi ciency AST interpreter Di ffi culty 3 / 26 http://soft-dev.org/

The traditional routes The traditional routes Translation (static content) E ffi ciency Translation (dynamic content) AST interpreter Di ffi culty 3 / 26 http://soft-dev.org/

The traditional routes The traditional routes Translation (static content) O ff the shelf VM E ffi ciency Translation (dynamic content) AST interpreter Di ffi culty 3 / 26 http://soft-dev.org/

The traditional routes The traditional routes O ff the shelf VM E ffi ciency Di ffi culty 3 / 26 http://soft-dev.org/

The traditional routes The traditional routes O ff the shelf VM (semantic match) E ffi ciency O ff the shelf VM (semantic mismatch) Di ffi culty 3 / 26 http://soft-dev.org/

The traditional routes The traditional routes O ff the shelf VM (semantic match) E ffi ciency Home brew VM O ff the shelf VM (semantic mismatch) Di ffi culty 3 / 26 http://soft-dev.org/

JIT compiled VM components JIT compiled VM components Interpreter JIT 1. Start 2. Detect hot method 3. Convert to machine code 4 / 26 http://soft-dev.org/

ffi ffi JIT compiled VM issues JIT compiled VM issues Interpreter JIT 5 / 26 http://soft-dev.org/

ffi ffi JIT compiled VM issues JIT compiled VM issues Interpreter Interpreter JIT JIT + Easy to write - Slow 5 / 26 http://soft-dev.org/

ffi JIT compiled VM issues JIT compiled VM issues Interpreter Interpreter Interpreter JIT JIT JIT + + Easy to write Easy to write + Fast - - Slow Slow - Di ffi cult to write 5 / 26 http://soft-dev.org/

JIT compiled VM issues JIT compiled VM issues Interpreter Interpreter Interpreter Interpreter JIT JIT JIT JIT + + + Easy to write Easy to write Easy to write + + Fast Fast - - - Slow Slow Slow - - Di ffi cult to write Di ffi cult to write } Interpreter and JIT must be kept in sync 5 / 26 http://soft-dev.org/

Build your own JIT compiler? Build your own JIT compiler? Building JIT compiled VMs is hard 6 / 26 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter 7 / 26 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter translator RPython 7 / 26 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter translator RPython Optimised JIT Interpreter 7 / 26 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter translator RPython Optimised JIT Interpreter 7 / 26 http://soft-dev.org/

Meta-tracing translation with RPython Meta-tracing translation with RPython Interpreter translator RPython Optimised JIT Interpreter 7 / 26 http://soft-dev.org/

Adding a JIT to an RPython interpreter Adding a JIT to an RPython interpreter ... pc := 0 while 1: instr := load_next_instruction(pc) if instr == POP: stack.pop() pc += 1 elif instr == BRANCH: off = load_branch_jump(pc) pc += off elif ...: ... Observation: interpreters are big loops. 8 / 26 http://soft-dev.org/

Adding a JIT to an RPython interpreter Adding a JIT to an RPython interpreter ... pc := 0 while 1: jit_merge_point(pc) instr := load_next_instruction(pc) if instr == POP: stack.pop() pc += 1 elif instr == BRANCH: off = load_branch_jump(pc) if off < 0: can_enter_jit(pc) pc += off elif ...: ... Observation: interpreters are big loops. 8 / 26 http://soft-dev.org/

RPython translation RPython translation Interpreter translator RPython Optimised JIT Interpreter 9 / 26 http://soft-dev.org/

RPython translation RPython translation Interpreter translator RPython Language Trace Interpreter Interpreter 9 / 26 http://soft-dev.org/

Tracing JIT compilers Tracing JIT compilers User program (lang FL ) if x < 0: x = x + 1 else: x = x + 2 x = x + 3 10 / 26 http://soft-dev.org/

Tracing JIT compilers Tracing JIT compilers User program (lang FL ) Trace when x is set to 6 if x < 0: guard_type(x, int) x = x + 1 guard_not_less_than(x, 0) else: guard_type(x, int) x = x + 2 x = int_add(x, 2) x = x + 3 guard_type(x, int) x = int_add(x, 3) 10 / 26 http://soft-dev.org/

A tracing system’s states A tracing system’s states 11 / 26 http://soft-dev.org/

A tracing system’s states A tracing system’s states New hot loop Trace 11 / 26 http://soft-dev.org/

A tracing system’s states A tracing system’s states New hot loop New hot loop Trace Trace Optimise 11 / 26 http://soft-dev.org/

A tracing system’s states A tracing system’s states New hot loop New hot loop New hot loop Trace Trace Trace Run trace Optimise Optimise 11 / 26 http://soft-dev.org/

A tracing system’s states A tracing system’s states New hot loop New hot loop New hot loop New hot loop Old hot loop Trace Trace Trace Trace Run trace Run trace Optimise Optimise Optimise 11 / 26 http://soft-dev.org/

A tracing system’s states A tracing system’s states New hot loop New hot loop New hot loop New hot loop New hot loop Old hot loop Old hot loop Guard fails Trace Trace Trace Trace Trace Run trace Run trace Run trace Optimise Optimise Optimise Optimise 11 / 26 http://soft-dev.org/

A tracing system’s states A tracing system’s states New hot loop New hot loop New hot loop New hot loop New hot loop New hot loop Old hot loop Old hot loop Old hot loop Guard fails Guard fails Trace Trace Trace Trace Trace Trace Run trace Run trace Run trace Run trace Repeated guard fail Optimise Optimise Optimise Optimise Optimise 11 / 26 http://soft-dev.org/

Trace optimisations (1) Trace optimisations (1) 1 Reuse typical compiler optimisations. 12 / 26 http://soft-dev.org/

Tracing JIT compilers Tracing JIT compilers User program (lang FL ) Trace when x is set to 6 if x < 0: guard_type(x, int) x = x + 1 guard_not_less_than(x, 0) else: guard_type(x, int) x = x + 2 x = int_add(x, 2) x = x + 3 guard_type(x, int) x = int_add(x, 3) 13 / 26 http://soft-dev.org/

Tracing JIT compilers Tracing JIT compilers User program (lang FL ) Optimised trace if x < 0: guard_type(x, int) x = x + 1 guard_not_less_than(x, 0) else: x = int_add(x, 5) x = x + 2 x = x + 3 13 / 26 http://soft-dev.org/

Meta-tracing VM components Meta-tracing VM components Language Trace Interpreter Interpreter 1. Start 2. Detect hot loop 3. Execute and trace 4. Convert trace to machine code 14 / 26 http://soft-dev.org/

Meta-tracing JITs Meta-tracing JITs FL Interpreter program_counter = 0; stack = [] elif instr == INSTR_IF: vars = {...} result = stack.pop() while True: if result == True: jit_merge_point(program_counter) program_counter += 1 instr = load_instruction(program_counter) else: if instr == INSTR_VAR_GET: program_counter += stack.push( read_jump_if_instruction() vars[read_var_name_from_instruction()]) elif instr == INSTR_ADD: program_counter += 1 lhs = stack.pop() elif instr == INSTR_VAR_SET: rhs = stack.pop() vars[read_var_name_from_instruction()] if isinstance(lhs, int) = stack.pop() and isinstance(rhs, int): program_counter += 1 stack.push(lhs + rhs) elif instr == INSTR_INT: else: ... stack.push(read_int_from_instruction()) program_counter += 1 program_counter += 1 elif instr == INSTR_LESS_THAN: rhs = stack.pop() lhs = stack.pop() if isinstance(lhs, int) and isinstance(rhs, int): if lhs < rhs: stack.push(True) else: stack.push(False) else: ... program_counter += 1 15 / 26 http://soft-dev.org/

Meta-tracing JITs Meta-tracing JITs FL Interpreter program_counter = 0; stack = [] vars = {...} while True: jit_merge_point(program_counter) instr = load_instruction(program_counter) if instr == INSTR_VAR_GET: stack.push( vars[read_var_name_from_instruction()]) program_counter += 1 elif instr == INSTR_VAR_SET: vars[read_var_name_from_instruction()] = stack.pop() program_counter += 1 elif instr == INSTR_INT: stack.push(read_int_from_instruction()) program_counter += 1 elif instr == INSTR_LESS_THAN: rhs = stack.pop() lhs = stack.pop() if isinstance(lhs, int) and isinstance(rhs, int): if lhs < rhs: stack.push(True) else: stack.push(False) else: ... program_counter += 1 15 / 26 http://soft-dev.org/