JSMeter: Characterizing the Behavior of JavaScript Web Applications - - PowerPoint PPT Presentation

▶
jsmeter characterizing the behavior of javascript web
SMART_READER_LITE
LIVE PREVIEW

JSMeter: Characterizing the Behavior of JavaScript Web Applications - - PowerPoint PPT Presentation

Dec 14, 2023 326 likes •679 views

JSMeter: Characterizing the Behavior of JavaScript Web Applications Paruj Ratanaworabhan Kasetsart University, Thailand Ben Livshits and Ben Zorn Microsoft Research, Redmond in collaboration with David Simmons, Corneliu Barsan, and Allen

slide-1
SLIDE 1

Paruj Ratanaworabhan

Kasetsart University, Thailand

Ben Livshits and Ben Zorn

Microsoft Research, Redmond

JSMeter: Characterizing the Behavior of JavaScript Web Applications

1

in collaboration with David Simmons, Corneliu Barsan, and Allen Wirfs-Brock

slide-2
SLIDE 2

Why Measure JavaScript?

  • Standardized, de facto language for the web

– Support in every browser, much existing code

  • Browser and JavaScript performance is important

– Are current JavaScript benchmarks representative? – Limited understanding of JavaScript behavior in real sites

  • Who cares?

– Users, web application developers, JavaScript engine developers

3

slide-3
SLIDE 3

4

ZDNet 29 May 2008 ghacks.net Dec. 2008

Browser Wars!

slide-4
SLIDE 4

Artificial Benchmarks versus Real World Sites

5

Maps Maps

7 V8 programs:

  • richards
  • deltablue
  • crypto
  • raytrace
  • earley-

boyer

  • regexp
  • splay

8 SunSpider programs:

  • 3-draytrace
  • access-nbody
  • bitops-nsieve
  • controlflow
  • crypto-aes
  • date-xparb
  • math-cordic
  • string-tagcloud

JSMeter

11 real sites:

Goals of JSMeter Project

  • Instrument JavaScript execution and measure behavior
  • Compare behavior of JavaScript benchmarks against real sites
  • Consider how benchmarks can mislead design decisions
slide-5
SLIDE 5

6

How We Measured JavaScript

\ie\jscript\*.cpp

Source-level instrumentation

custom jscript.dll custom trace files website visits

Offline analyzers

custom trace files

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Constant Other Str Ops Concat Op

slide-6
SLIDE 6

Visiting the Real Sites

  • Getting past page load performance
  • Attempted to use each site in “normal” way:

7

amazon Search a book, add to shopping cart, sign in, and sign out bing Type in a search query and also look for images and news bingmap Search for a direction from one city to another cnn Read front page news ebay Search for a notebook, bid, sing in, and sign out economist Read front page news, view comments facebook Log in, visit a friend pages, browse through photos and comments gmail Sign in, check inbox, delete a mail, and sign out google Type in a search query and also look for images and news googlemap Search for a direction from one city to another hotmail Sign in, check inbox, delete a mail, and sign out

slide-7
SLIDE 7

Understanding JavaScript Behavior

Code Events

JavaScript

Objects

8

slide-8
SLIDE 8

Code Behavior

Code Events

JavaScript

Objects

9

  • Function size
  • Instructions/call
  • Code locality
  • Instruction mix
slide-9
SLIDE 9

Total Bytes of JavaScript Source

10

500 1000 1500 2000 2500

amazon bing bingmap cnn ebay economist facebook gmail google googlemap hotmail richards deltablue crypto raytrace earley regexp splay 3d-raytrace access-nbody bitops-nsieve controlflow crypto-aes date-xparb math-cordic regexp-dna string-tagcloud

Sour urce ce size ze (kilo lobytes ytes)

Real Sites V8 SunSpider

Code|Objects|Events

slide-10
SLIDE 10

Static Unique Functions Executed

12

500 1,000 1,500 2,000 2,500 3,000 3,500 4,000

amazon bing bingmap cnn ebay economist facebook gmail google googlemap hotmail richards deltablue crypto raytrace earley regexp splay 3d-raytrace access-nbody bitops-nsieve controlflow crypto-aes date-xparb math-cordic regexp-dna string-tagcloud Unique que Funct ctio ions ns Execut uted

Real Sites V8 SunSpider

Code|Objects|Events

slide-11
SLIDE 11

Bytecodes / Call

15

0.00 50.00 100.00 150.00 200.00 250.00 300.00

amazon bing bingmap cnn ebay economist facebook gmail google googlemap hotmail richards deltablue crypto raytrace earley regexp splay 3d-raytrace access-nbody bitops-nsieve controlflow crypto-aes date-xparb math-cordic regexp-dna string-tagcloud

Byteco ecodes es / C Call ll

function(a,b) { var i=0,elem,pos=a.length; if(D.browser.msie) { while(elem=b[i++]) if(elem.nodeType!=8) a[pos++]=elem; } else while(elem=b[i++]) a[pos++]=elem; return a }

Real Sites V8 SunSpider

Code|Objects|Events

slide-12
SLIDE 12

Fraction of Code Executed

16

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

amazon bing bingmap cnn ebay economist facebook gmail google googlemap hotmail richards deltablue crypto raytrace earley regexp splay 3d-raytrace access-nbody bitops-nsieve controlflow crypto-aes date-xparb math-cordic regexp-dna string-tagcloud

Fract ction ion of Code e Execu cuted ted

Most code not executed Real Sites V8 SunSpider

Code|Objects|Events

slide-13
SLIDE 13

Object Allocation Behavior

Code Events

JavaScript

Objects

17

  • Allocation by types
  • Live heap composition
  • Lifetime distribution
slide-14
SLIDE 14

Total Bytes Allocated

18

5000 10000 15000 20000 25000 30000 35000 40000 45000

amazon bing bingmap cnn ebay economist facebook gmail google googlemap hotmail richards deltablue crypto raytrace earley regexp splay 3d-raytrace access-nbody bitops-nsieve controlflow crypto-aes date-xparb math-cordic regexp-dna string-tagcloud

Total heap data (kilobytes tes)

Real Sites V8 SunSpider

Code|Objects|Events

slide-15
SLIDE 15

Heap Data by Type

19

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

amazon bing bingmap cnn ebay economist facebook gmail google googlemap hotmail richards deltablue crypto raytrace earley regexp splay V8 aggragate 3d-raytrace access-nbody bitops-nsieve controlflow crypto-aes date-xparb math-cordic regexp-dna string-tagcloud aggregate

Heap Data by Type

Objects Enumarator RegExp Math Error Date Native Func String Arrays Script Func

Real Sites V8 SunSpider Many functions Rest are strings

Few benchmarks allocate much data

Code|Objects|Events

slide-16
SLIDE 16

Live Heap Over Time (gmail)

20

Functions grow steadily

Code|Objects|Events

GC reduces size of heap Objects grow steadily too

slide-17
SLIDE 17

Live Heap over Time (ebay)

21

Heaps repeatedly created, discarded Heap contains mostly functions

Code|Objects|Events

Heap drops to 0

  • n page load
slide-18
SLIDE 18

2 Search Websites, 2 Architectures

Bing Google

Code|Objects|Events

22

You stay on the same page during your entire visit Code loaded once Heap is bigger Every transition loads a new page Code loaded repeatedly Heap is smaller

slide-19
SLIDE 19

Event Handlers in JavaScript

Code Events

JavaScript

Objects

26

  • Number of events
  • Sizes of handlers
slide-20
SLIDE 20

Event-driven Programming Model

  • Single-threaded, non-preemptive event

handlers

  • Example handlers: onabort, onclick, etc.
  • Very different from batch processing of

benchmarks

  • Handler responsiveness critical to user

experience

27

Code|Objects|Events

slide-21
SLIDE 21

Total Events Handled

28

1,000 2,000 3,000 4,000 5,000 6,000 7,000

Tota tal l Event nts s Hand ndled led

Real Sites V8 Almost no events

Code|Objects|Events

slide-22
SLIDE 22

Median Bytecodes / Event Handled

30

50 100 150 200 250 300 350 400 450 500

Medi dian n Byteco ecodes des / E Event ent Hand ndled ed

Code|Objects|Events

506 2137

slide-23
SLIDE 23

Sure, this is all good, but…

  • Everyone knows benchmarks are

unrepresentative

  • How much difference does it make, anyway?
  • Wouldn’t any benchmarks have similar issues?

33

slide-24
SLIDE 24

Cold-code Experiment

  • Observation

– Real web apps have lots of code (much of it cold) – Benchmarks do not

  • Question: What happens if the benchmarks

have more code?

– We added extra, unused to code to 7 SunSpider benchmarks – We measured the impact on the benchmark performance

34

slide-25
SLIDE 25

Performance Impact of Cold Code

35

100 200 300 400 500 600 700 800 900 Time (msec) 0K 200K 400K 800K 1M 2M 100 200 300 400 500 600 700 800 900 Time (msec) 0K 200K 400K 800K 1M 2M

Chrome 3.0.195.38 IE 8 8.0.601.18865

Cold code has non-uniform impact

  • n execution time

Cold code makes SunSpider on Chrome up to 4.5x slower

slide-26
SLIDE 26

Impact of Benchmarks

  • What gets emphasis

– Making tight loops fast – Optimizing small amounts of code

  • Important issues ignored

– Garbage collection (especially of strings) – Managing large amounts of code – Optimizing event handling – Considering JavaScript context between page loads

36

slide-27
SLIDE 27

Conclusions

  • JSMeter is an instrumentation framework

– Used to measure and compare JavaScript applications – High-level views of behavior promote understanding

  • Benchmarks differ significantly from real sites

– Misleads designers, skews implementations

  • Next steps

– Develop and promote better benchmarks – Design and evaluate better JavaScript runtimes – Promote better performance tools for JavaScript developers

37

slide-28
SLIDE 28

Additional Resources

  • Project: http://research.microsoft.com/en-us/projects/jsmeter/
  • Video: Project JSMeter: JavaScript Performance Analysis in the Real

World" - MSDN Channel 9 interview with Erik Meier, Ben Livshits, and Ben Zorn

  • Paper:

– “JSMeter: Comparing the Behavior of JavaScript Benchmarks with Real Web Applications”, Paruj Ratanaworabhan, Benjamin Livshits and Benjamin G. Zorn, USENIX 2010 Conference on Web Application Development (WebApps’10), June 2010.

  • Additional measurements:

– "JSMeter: Characterizing Real-World Behavior of JavaScript Programs", Paruj Ratanaworabhan, Benjamin Livshits, David Simmons, and Benjamin Zorn, MSR-TR-2009-173, December 2009 (49 pages), November 2009.

38

slide-29
SLIDE 29

Additional Slides

39

slide-30
SLIDE 30

Hot Function Distribution

40

Real Sites V8 Benchmarks 80% of time in < 10 functions 80% of time in 100+ functions

80% 80%

Code|Objects|Events

slide-31
SLIDE 31

Opcode Distribution

41

Green = SunSpider | Blue= Real Web Apps | Red = V8

Real Apps Outliers

Code|Objects|Events

slide-32
SLIDE 32

Object Type Distribution

42

Real Apps

Code|Objects|Events

economist is an outlier (arrays)

Green = SunSpider | Blue= Real Web Apps | Red = V8

slide-33
SLIDE 33

Distribution of Time in Handlers

43

Code|Objects|Events

slide-34
SLIDE 34

Related Work

  • JavaScript

– Richards, Lebresne, Burg, and Vitek (PLDI’10) – Draw similar conclusions

  • Java

– Doufour et al. (OOPSLA’03), Dieckmann and U. Hölzle (ECOOP’99)

  • Other languages

– C++: Calder et al. (JPL’95) – Interpreted languages: Romer et al. (ASPLOS’96)

44