WEBPACK + WEBASSEMBLY W E B A S S E M B L Y W E B P A C K A N D T H E C H A L L E N G E O F
INTRODUCTION WEBASSEMBLY AND ESM
WEBASSEMBLY? • L OW -L EVEL B INARY F ORMAT FOR C ODE • T YPED ( I 8 – I 64, F 32, F 64) • M EMORY • U SUALLY COMPILED FROM NATIVE LANGUAGES (C/C++, R UST , …) • A SSEMBLY - LIKE T EXTUAL R EPRESENTATION (WAT) • WASM = W EB A SSEMBLY
STRUCTURE OF A WASM-FILE • M ULTIPLE S ECTIONS : • D ATA • I MPORT • E XPORT • G LOBALS • S TART • F UNCTION - TYPES • O THER AND ALSO C USTOM SECTIONS • C ODE
EXAMPLE WAT FILE (module (type $addType (func (param i32) (result i32))) (type $getNumberType (func (result i32))) (import "./module" "getValue" (func $getValue (type $getNumber_type))) (func $add (export "add") (type $add_type) (param $p0 i32) (result i32) (i32.add (call $getValue) (get_local $p1) ) ) (func $getNumber (export "getNumber") (type $getNumber_type) (result i32) (i32.const 40) ) )
WEBASSEMBLY API • 1. FETCH THE BINARY • 2. COMPILE A WebAssembly.Module • 3. INSTANCIATE A WebAssembly.Instance A N importsObject CAN BE PASSED • • 4. U SE instance.exports • 2. AND 3. ARE AVAILABLE IN TWO VARIANTS : A SYNC AND S YNC
ECMASCRIPT MODULES? • (N EW ) SOURCE TYPE “ MODULE ” NEXT TO “ SCRIPT ” • A LLOWS NEW S YNTAX : I MPORT • E XPORT • • A LWAYS STRICT • ESM = E CMA S CRIPT M ODULE
EXPORTS, IMPORTS AND LIVE-BINDINGS • export { x, y }; D OES NOT EXPORT AN OBJECT • D OES NOT EXPORT THE VALUES OF x AND y • I T EXPORTS BINDINGS TO THE VARIABLES x AND y IN THIS SCOPE ! • • import { x, y } from "module"; B INDS x AND y IN THE NEW SCOPE ( READ - ONLY ) • A NY CHANGE IN THE ORIGIN MODULE IS REFLECTED HERE •
ESM LIFECYCLE • P HASE 1: R ECURSIVELY : L OAD AND P ARSE FILES • R ESOLVE I MPORTS • • P HASE 2: “C ONNECT ” / B IND EXPORTS AND IMPORTS • P HASE 3: E VALUATE ALL CODE AT ONCE ( IN THE SAME MICROTASK / TICK ) D EPENDENCIES ARE EVALUATED BEFORE P ARENT ( EXCEPT IN CIRCLES ) • D EPENDENCIES ARE EVALUATED IN ORDER OF O CCURRENCE ( IMPORTS ) •
ADDING WASM TO WEBPACK
THE STORY • S O WE WANT TO ADD W EB A SSEMBLY SUPPORT TO WEBPACK . • H OW TO DO THAT ? L ET ’ S START WITH AN PROTOTYPE AND FIND PROBLEMS • I TERATE ON THAT UNTIL ALL G OALS AND M ILESTONES ARE COMPLETED •
MOZILLA SPONSORSHIP • W E GOT A SPONSORSHIP FROM M OZILLA TO WORK ON THAT S EEMS LIKE THEY WANT TO PUSH WASM AND R UST FORWARD • • T HIS CAN COVER A PART OF THE IMPLEMENTATION COST
GOALS • EASY TO USE • H IDE T ECHNICAL DETAILS • W EBPACK DOESN ’ T CARE ABOUT H IGH - LEVEL LANGUAGE (C/C++, R UST ) • P ERFORMANT ( PRIMARY RUNTIME , BUT ALSO BUILD ) • WASM FILE IS A MODULE • I NTEGRATE WELL WITH ESM • C ODE S PLITTING SUPPORT
GOAL AS SYNTAX • import { func } from "./module.wasm"; • import { fn1 } from "./module.wat"; • import { fn2 } from "./module.rs"; • import { fn3 } from "./module.cpp"; • (import "./memory.wasm" "memory" (memory $mem 1))
IDEA • A DD WASM AS MODULE TYPE • R UN ASYNC OPERATIONS ON S PLIT P OINTS ( I . E . import() ) • T REAT IMPORT AND EXPORTS SECTION LIKE IN ESM • E NFORCE SINGLE INSTANCE SIMILAR TO ESM • R UN S TART SECTION ON “E VALUATE ”
CURRENT STATE (WEBPACK 3) WEBPACK ONLY SUPPORT A SINGLE MODULE TYPE : J AVA S CRIPT (JS) • E VERYTHING IS COMPILED TO JS • WEBPACK DOESN ’ T SUPPORT ADDITIONAL OPERATIONS ON S PLIT P OINTS •
WEBPACK CHANGE PLAN • A DD SUPPORT FOR MULTIPLE MODULE TYPES JS/ AUTO • JS/ ESM • WASM • JSON • CSS ( SEPARATE PLUGIN ) • • A DD SUPPORT FOR EMITTING MULTIPLE ASSETS PER CHUNK + R UNTIME CODE
CHALLENGE 1: ESM SPEC VS WEBASSEMBLY API E CMA S CRIPT M ODULE W EB A SSEMBLY M ODULE L IFE - CYCLE W ASM API • • 1. L OAD 1. FETCH • • 2. P ARSE 2. COMPILE • • 3. B IND E XPORTS AND I MPORTS 3. P ROVIDE IMPORTS • • 4. E VALUATE ( SINGLE M ICROTASK ) 4. INSTANTIATE , E VALUATE S TART • • 5. R ECEIVE EXPORTS •
WEBPACK CHANGE PLAN • A DD P ARSER FOR W EB A SSEMBLY TO EXTRACT IMPORT AND EXPORT SECTION • U SE IMPORTS IN SECTION AS DEPENDENCIES • R UNTIME CODE : FETCH + COMPILE AT THE S PLIT P OINT • R UNTIME CODE : • LOAD DEPENDENCIES AND CREATE importObject • INSTANTIATE SYNC • SETUP EXPORTS
SIMPLIFIED GENERATED CODE // at split point res = await fetch(url); wasmModule = await WebAssembly.compileStreaming(res); // at evaluate instance = new WebAssembly.Instance(wasmModule, { importedModule: __webpack_require__(123) }); module.exports = instance.exports;
PROBLEMS • C HROME THROWS E RROR WHEN USING SYNC instantiate WITH BINARY > 4 KB T HIS IS AGAINST THE S PEC ! • • S AFARI DOESN ’ T COMPILE IN “ compile ”, BUT IN instantiate F OR “ TECHNICAL ” REASONS … • • W E MUST USE instantiateStreaming INSTEAD OF compileStreaming + SYNC instantiate !
CHALLENGE 2: ASYNC INSTANTIATE • W E NEED TO MOVE instantiate TO THE S PLIT P OINT • T HIS MEANS instantiate COULD BE CALLED BEFORE DEPENDENCIES ARE EVALUATED S TART SECTION WOULD RUN IN WRONG P HASE • I MPORT HANDLES VALUES AND NOT BINDINGS • V ALUES ARE COPIED / SNAPSHOT ON instantiate • V ALUES ARE NOT AVAILABLE BEFORE DEPENDENCIES ARE EVALUATED • • I T TOOK A BIT UNTIL WE HAD A “ SOLUTION ” FOR THIS PROBLEM …
NEW PLAN • R EWRITE THE WASM BINARY R EMOVE S TART SECTION • R EWRITE IMPORTED G LOBALS TO MUTABLE G LOBALS • C REATE A NEW EXPORTED _webpack_init_ FUNCTION • S ET IMPORTED GLOBALS ( PASSED AS ARGUMENTS ) • C ALL OLD S TART FUNCTION • • I MPORTED FUNCTIONS GET WRAPPED IN TRAMPOLINE FUNCTION E VEN ENABLED LIVE - BINDINGS FOR FUNCTIONS •
REWRITING WASM (func $something (import "./module" “something" ) (func $something (param i32)) (import "./module" “something" ) (param i32)) (global $magicNumber (mut i32) (i32.const 66)) (global $magicNumber (func $start (import "./a.js" "magicNumber") get_global $magicNumber i32) call $something ) (func $start get_global $magicNumber (func $__webpack_init__ call $something (export "__webpack_init__") (param $p0 i32) ) get_local $p0 set_global $g0 (start $start) call $start )
INSTANTIATE AT SPLIT POINT & TRAMPOLINE // at split point instance = new WebAssembly.Instance(wasmModule, { "./module": { "something": function(p0i32) { return installedModules[123].exports["something"](p0i32); } } }); // at evaluate module.exports = instance.exports; __webpack_require__(123); a = __webpack_require__(321); instance.exports.__webpack_init__( a.magicNumber )
CHALLENGE 3: MEMORY AND TABLES • I MPORTING M EMORY AND T ABLES DOESN ’ T WORK C AN ’ T BE PASSED AS ARGUMENTS TO THE INIT FUNCTION • C AN ’ T USE TRAMPOLINES ( NO FUNCTIONS ) • MUST BE PROVIDED ON INSTANTIATE •
EXTRA PLAN • F ORBIT IMPORTING M EMORY OR T ABLE FROM NON -WASM INTO WASM • F OR W ASM - TO -W ASM IMPORTS : H ANDLE M EMORY AND T ABLE DIRECTLY IN RUNTIME CODE • M AY SEQUENTIALIZE INSTANTIATE (D OWNLOAD AND C OMPILE IS STILL PARALLEL ) •
CHALLENGE 4: REEXPORTED GLOBALS AND INIT ;; importing a global • M UTABLE G LOBALS CAN ’ T BE EXPORTED ( AT THIS TIME ) (import "module" "abc" (global i32)) P ROBLEM WHEN EXPORTING AND IMPORTED GLOBAL • ;; re-exporting this global • O NE CAN PROVIDE AN EXPRESSION FOR INITIALIZING THE (export "def" (global 0)) GLOBAL ;; global with init expression T HIS CAN BE “ GET OTHER GLOBAL ” • (global $g i32 (get_global 0)) P ROBLEM WHEN OTHER GLOBAL IS IMPORTED • ;; export this global E XTRA PROBLEM WHEN THIS GLOBAL IS EXPORTED • (export "ghi" (global $g))
EXTRA PLAN (THE SECOND) • D ETECT THESE SPECIAL CASES M AY MOVE INIT EXPRESSION TO OUR _webpack_init_ FUNCTION • H ANDLE REEXPORTING IN THE RUNTIME CODE • P ROBLEMATIC EXPORTS ARE HANDLED IN JS INSTEAD OF WASM •
CHALLENGE 5: I64 • W EB A SSEMBLY HAS THIS NICE 64 BIT INTEGER DATA TYPE • J AVA S CRIPT CAN ’ T HANDLE IT ( CURRENTLY )* • A NY FUNCTION WITH i64 IN SIGNATURE WILL CRASH IN JS ( RuntimeError ) B UT IT CAN BE PASSED TO OTHER W EB A SSEMBLY MODULES • O UR T RAMPOLINE WILL ALSO CRASH • * BigInt COULD
EXTRA PLAN (3 RD ) • HANDLE THESE FUNCTIONS IN RUNTIME E NFORCE DIRECT CONNECTING THESE MODULES • H ANDLED LIKE M EMORY AND T ABLE TYPES • • E MIT A BUILD WARNING WHEN IMPORTING THESE I 64- FUNCTIONS INTO ESM C AN ’ T ALWAYS DETECT IT : W ILL RuntimeError ANYWAY •
IT WORKS • U NSOLVED L IMITATIONS : C AN ’ T IMPORT MEMORY / TABLE FROM JS • W ATERFALL - LIKE INSTANTIATE WHEN DIRECT CONNECTION ARE REQUIRED • T RAMPOLINE FUNCTIONS HAVE EXTRA OVERHEAD • W E TRY TO OMIT THEM WHEN POSSIBLE • • S O FAR THESE LIMITATIONS HAVE NOT BECOME PROBLEMS A LL COMPILE - TO - WASM LANGUAGES EMBED AND EXPORT THE MEMORY • M ULTIPLE W EB A SSEMBLY MODULES IN A SINGLE CHUNK ARE RARE ( CURRENTLY ) •
Recommend
More recommend
