T22 Performance Testing Thursday, May 3rd, 2018 3:00 PM Open Source Tools for Performance Testing and Monitoring Presented by: Kaushal Dalvi Ultimate Software Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888 --- 268 --- 8770 ·· 904 --- 278 --- 0524 - info@techwell.com - http://www.stareast.techwell.com/
Kaushal Dalvi Ultimate Software Since he got his first computer at age thirteen, Kaushal Dalvi has been interested in systems and software performance. He spent days researching performance characteristics of different motherboards, CPUs, GPUs, RAM, and disks to configure/overclock them to squeeze the maximum frames/second out of the games he played. Kaushal built and maintained websites for local businesses, where he started learning about performance and reliability. He developed a taste for it and now nine years later continues working in the same field with workload modeling, test scripting and execution, result and bottleneck analysis, and much more.
- OPEN SOURCE TOOLS FOR PERFORMANCE TESTING AND MONITORING -KAUSHAL DALVI WHY PERFORMANCE? BUILD CONSENSUS AROUND THE VALUE 03 05 STAPLES 01 WALMART PINTREST Increase in conversions 04 Increase revenue Increase sign-ups NETFLIX Decrease bandwidth bill 0 2 B C B e r l o s s e n t u s P r e v Pintrest - https://medium.com/@Pinterest_Engineering/driving-user-growth-with-performance-improvements-cfc50dafadd7 BBC - www.creativebloq.com/features/how-the-bbc-builds-websites-that-scale Staples - https://www.slideshare.net/cliffcrocker/velocity-ny-how-to-measure-revenue-in-milliseconds Walmart - https://www.slideshare.net/devonauerswald/walmart-pagespeedslide Netflix - http://cdn.oreillystatic.com/en/assets/1/event/7/Improving%20Netflix%20Performance%20Presentation.pdf 1
- WHERE DO YOU MEASURE? PROS AND CONS 30 LAB 25 20 TESTING TOOLS 15 Lab data is performance data collected within a controlled environment with predefined 10 device and network settings. This offers reproducible results and debugging capabilities to help identify, isolate, and fix performance issues. 5 0 0 2 4 6 8 10 12 25 PROD 20 MONITORING TOOLS 15 10 (Also Real User Monitoring or RUM) Field data is performance data collected from real page loads your users are 5 experiencing in the wild. 0 0 2 4 6 8 10 12 THE RIGHT APPROACH? FIND THE BALANCE 2
- THE WEB EXPERIENCE END USER PERSPECTIVE THE WEB REQUEST LIFECYCLE THE COMPONENTS THE MAKE UP END USER PERCIEVED RESPONSE TIMES • BROWSER • OS DNS CACHE CHECKS • ROUTER & CONNECTION • ISP • REQUEST SENT OVER TCP 3
- SERVER RENDERS • APP SHELL HTML • BASIC HTML PAGE DOM CSSOM RENDER TREE TREE TREE BUILDING THE TREES How the browser begins the process of showing the web page 4
- RENDER BLOCKS DOWNLOAD ADDITIONAL RESOURCES CSS, JS, WOFF files may block the parse and/or render trees from being generated AJAX REQUESTS Data is increasingly being populated after the app shell has bootstrapped 5
- VISUALLY COMPLETE AND INTERACTIVE Page is fully loaded and is usable. WHAT CONSTITUTES PERFORMANCE? WHAT IS PERFORMANCE FROM THE END USER PERSPECTIVE? BACKEND NW SERVERS CODE + The parts of the system that determine what needs to be served to the user. The design and architecture choices here greatly influence the Performance, Scalability and Reliability characteristics 100% of the system. FOR THE MANY FRONTEND NW HTML CSS JS + The parts of the system that determine how the information is presented to the user. The design and architecture choices here greatly influence the Performance characteristics of the system for the users based on their device and network conditions. FOR THE ONE 6
- THE IDEAL SETUP NON FUNCTIONAL REQUIREMENTS At a product/feature/story level. FEEDBACK PERFORMANCE Continuously feed back into the AS A FEATURE system to keep improving it Team treats SLOs as ENSURING features to strive for and designs for them. PERFORMANCE REAL WORLD LAB PERFORMANCE MONITORING TESTS Validating expectations in production Providing input and confidence before and re-enforcing base data and shipping to production. assumptions. STATISTICS TERMS TO KNOW TERMS GOOD TO BE AWARE OF WHEN ANALYZING/INTERPRETING TEST/MONITORING RESULTS AVERAGES OUTLIERS AVG OUT Mean | median | mode Min | max PERCENTILES DISTRIBUTIONS PER DST Quantiles, histograms Normal, uniform, long-tail 7
- FRONTEND PERFORMANCE FRONTEND PERFORMANCE METRICS Each of these have an impact on the perception of performance by the user. FIRST PAINT HERO ELEMENT When the app shows some sign of When the most important part of the page life/activty upon submitting request. that the user is interested in has loaded. FIRST CONTENTFUL PAINT TIME TO INTERACTIVE (FIRST) When the canvas of the app has taken the When most of the elements on the page shape and structure of the final form. respond to input from the user. FIRST MEANINGFUL PAINT CONSISTENTLY INTERACTIVE When the user can start to get meaningful When the app is no longer issuing network cues from the app. requests (except websockets) 8
- - FRONTEND PERFORMANCE METRICS In build pipelines. wi window.perf rform rmance ____ligh ghthou ouse function showPaintTimings() { module.exports = { if (window.performance) { passes: [{ let performance = window.performance; recordTrace: true, let performanceEntries = pauseAfterLoadMs: 5000, performance.getEntriesByType('paint'); useThrottling: true, performanceEntries.forEach( }], (performanceEntry, i, entries) => { audits: [ console.log("The time to " + performanceEntry.name 'first-meaningful-paint', + " was " + performanceEntry.startTime + 'estimated-input-latency', ”milliseconds."); }); 'first-interactive', } else { 'consistently-interactive', console.log('Performance timing isn\'t supported.'); 'user-timings’ } ] } }; PERFORMANCE PROGRESSIVE WEB APP $ # How long does the Does the page meet standards of a page/app take to show progressive web app content and become usable ACCESSIBILITY BEST PRACTICES " ! Is the page usable by people Does the page follow best with disabilities or practices for modern web LIGHTHOUSE DEMO impairments development Lighthouse is an open-source, automated tool for improving the quality of web pages. You can run it against any web page, public or requiring authentication. It has audits for performance, accessibility, progressive web apps, and more. SOURCE - https://github.com/GoogleChrome/lighthouse 9
- BACKEND PERFORMANCE PSR WHAT DO WE NEED TO ENSURE ON THE BACKEND Commonly used as an Umbrella term. Scalability is the application’s ability to Reliability is the probability of the Performance is concerned with handle additional workload, without application operating without a failure achieving response times, throughput, adversely affecting performance, by for a given period of time. Also and resource-utilization levels that meet adding resources such as processor, measured as MTBF. performance objectives. memory, and storage capacity. PERFORMANCE SCALABILITY RELIABILITY 10
- HEY DEMO hey is a tiny program that sends some load to a web application. SOURCE - https://github.com/rakyll/hey ARTILLERY DEMO Artillery is a modern, powerful & easy-to-use load testing toolkit. Use it to ship scalableapplications that stay performant & resilient under high load. SOURCE - https://github.com/shoreditch-ops/artillery 11
- LOCUST DEMO Define user behavior with Python code, and swarm your system with millions of simultaneous users. SOURCE - https://github.com/locustio/locust MONITORING 12
- MONITORING TIPS DOS AND DONTS NO SILVER NOT A JOB NOT A AUTOMATION USER DON’T BUILD BULLETS CHECKBOX PERSPECTIVE Monitoring is a skill not a Deploying monitoring Building a new monitoring job and not suited for should never have to be a system from scratch is Don’t be tool obsessed Monitoring for the sake of When starting to monitor, someone who doesn’t manual task, it almost almost never a good idea with a tool. Replacing a it almost never helps, start from the perspective know what it is they are guarantees that it will unless you are a tech giant tool to fill a minor gap rather it generates more of the user and then start monitoring. eventually be ignored. that needs a custom starts a never ending noise and causes more expanding outwards. solution. cycle. confusion than it helps. ELK STACK DEMO Reliably and securely take data from any source, in any format, and search, analyze, and visualize it in real time. SOURCE - https://github.com/elastic 13
- TIG STACK DEMO InfluxData provides a Modern Time Series Platform, designed from the ground up to handle metrics and events. SOURCE - https://github.com/influxdata JOIN US ULTIMATE SOFTWARE IS HIRING PERFORMANCE AND QUALITY ENGINEERS ULTIMATE SOFTWARE CAREERS CONTACT Our mission is to deliver unified, end-to-end HCM cloud solutions—everything from HR, to payroll, to benefits, to time & attendance, to recruitment, to talent management—to improve @kaushald the personal work experience for you and ! www.kaushaldalvi.com your people — the power " behind your business. kaushal_dalvi@ultimatesoftware.com https://www.ultimatesoftware.com/careers/ kaushaldalvi 14
- THANK YOU - KAUSHAL DALVI 15
Recommend
More recommend
