predictable latency
play

Predictable Latency Adventures in Java Concurrency Martin Thompson - PowerPoint PPT Presentation

Jul 24, 2023 •207 likes •1.33k views

A Quest for Predictable Latency Adventures in Java Concurrency Martin Thompson - @mjpt777 If a system does not respond in a timely manner then it is effectively unavailable 1. Its all about the Blocking 2. What do we mean by Latency 3.

  1. One big file that goes on forever?

  2. No!!! Page faults, page cache churn, VM pressure, ...

  3. Clean Dirty Active Header Header Message Message Header Header Message Message Header Header Message Message Tail Header Message Header Message

  4. Do publishers need a CAS?

  5. File Header Message 1 Header Message 2 Header Message 3 Tail <<XADD>> Message X Message Y

  6. File Header Message 1 Header Message 2 Header Message 3 Message X Tail <<XADD>> Message Y

  7. File Header Message 1 Header Message 2 Header Message 3 Message X Message Y Tail

  8. File Header Message 1 Header Message 2 Header Message 3 Header Message X Message Y Tail

  9. File Header Message 1 Header Message 2 Header Message 3 Header Message X Message Y Padding Tail

  10. File File Tail Header Message 1 Header Message 2 Header Message 3 Header Message X Message Y Padding

  11. File File Header Header Message 1 Message X Header Tail Message 2 Header Message 3 Header Message Y Padding

  12. Have a background thread do zero’ing and flow control

  13. Burst Length = 1: RTT (ns) Prod # Mean 99% 1 167 189 Baseline (Wait-free) 1 281 361 ConcurrentLinkedQueue 2 381 559 3 444 705 1 170 194 ManyToOneRingBuffer 2 256 279 3 283 340 1 192 210 Aeron IPC 2 251 286 3 290 342

  14. Burst Length = 1: RTT (ns) Prod # Mean 99% 1 167 189 Baseline (Wait-free) 1 281 361 ConcurrentLinkedQueue 2 381 559 3 444 705 1 170 194 ManyToOneRingBuffer 2 256 279 3 283 340 1 192 210 Aeron IPC 2 251 286 3 290 342

  15. Aeron Data Message Header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-------------------------------------------------------------+ |R| Frame Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-------------------------------+ | Version |B|E| Flags | Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-------------------------------+ |R| Term Offset | +-+-------------------------------------------------------------+ | Session ID | +---------------------------------------------------------------+ | Stream ID | +---------------------------------------------------------------+ | Term ID | +---------------------------------------------------------------+ | Encoded Message ... ... | +---------------------------------------------------------------+

  16. Burst Length = 100: RTT (ns) Prod # Mean 99% 1 721 982 Baseline (Wait-free) 1 12,916 15,792 ConcurrentLinkedQueue 2 25,132 35,136 3 39,462 56,768 1 3,773 3,992 ManyToOneRingBuffer 2 11,286 13,200 3 27,255 34,432 1 4,436 4,632 Aeron IPC 2 7,623 8,224 3 10,825 13,872

  17. Burst Length = 100: RTT (ns) Prod # Mean 99% 1 721 982 Baseline (Wait-free) 1 12,916 15,792 ConcurrentLinkedQueue 2 25,132 35,136 3 39,462 56,768 1 3,773 3,992 ManyToOneRingBuffer 2 11,286 13,200 3 27,255 34,432 1 4,436 4,632 Aeron IPC 2 7,623 8,224 3 10,825 13,872

  18. Less “False Sharing” - Inlined data vs Reference Array Less “Card Marking”

  19. Is avoiding the spinning “CAS” loop is a major step forward?

  20. Burst Length = 100: RTT (ns) Prod # Mean 99% 1 721 982 Baseline (Wait-free) 1 12,916 15,792 ConcurrentLinkedQueue 2 25,132 35,136 3 39,462 56,768 1 3,773 3,992 ManyToOneRingBuffer 2 11,286 13,200 3 27,255 34,432 1 4,436 4,632 Aeron IPC 2 7,623 8,224 3 10,825 13,872

Recommend

CROSS-LAYER CROSS-LAYER LATENCY-AWARE AND -PREDICTABLE LATENCY-AWARE AND -PREDICTABLE DATA

CROSS-LAYER CROSS-LAYER LATENCY-AWARE AND -PREDICTABLE LATENCY-AWARE AND -PREDICTABLE DATA

CROSS-LAYER CROSS-LAYER LATENCY-AWARE AND -PREDICTABLE LATENCY-AWARE AND -PREDICTABLE DATA COMMUNICATION DATA COMMUNICATION Andreas Schmidt Apr 07th, 2020 Anywhere on Earth Doctoral Colloquium 1

702 views • 59 slides
Predictable Latency Adventures in Java Concurrency Martin Thompson - @mjpt777 If a system does

Predictable Latency Adventures in Java Concurrency Martin Thompson - @mjpt777 If a system does

A Quest for Predictable Latency Adventures in Java Concurrency Martin Thompson - @mjpt777 If a system does not respond in a timely manner then it is effectively unavailable 1. Its all about the Blocking 2. What do we mean by Latency 3.

1.69k views • 116 slides
Rocksteady: Fast Migration for Low-Latency In-memory Storage Chinmay Kulkarni , Aniraj Kesavan,

Rocksteady: Fast Migration for Low-Latency In-memory Storage Chinmay Kulkarni , Aniraj Kesavan,

Rocksteady: Fast Migration for Low-Latency In-memory Storage Chinmay Kulkarni , Aniraj Kesavan, Tian Zhang, Robert Ricci, Ryan Stutsman 1 Introduction Distributed low-latency in-memory key-value stores are emerging Predictable response

757 views • 50 slides
Quest-V: A Secure and Predictable System for IoT and Beyond Richard West richwest@cs.bu.edu

Quest-V: A Secure and Predictable System for IoT and Beyond Richard West richwest@cs.bu.edu

Quest-V: A Secure and Predictable System for IoT and Beyond Richard West richwest@cs.bu.edu Computer Science Goals Develop system for high-confidence (embedded) systems Mixed criticalities (timeliness and safety) Predictable

573 views • 45 slides
Is It Latency or Do You Just Suck? Latency Can Kill: Precision and Deadline in Online Games

Is It Latency or Do You Just Suck? Latency Can Kill: Precision and Deadline in Online Games

12/6/2011 Is It Latency or Do You Just Suck? Latency Can Kill: Precision and Deadline in Online Games http://www.youtube.com/watch?v=r6PwHkhEAkU Mark Claypool and Kajal Claypool ACM MMSys Scottsdale, AZ, USA February 2010

412 views • 4 slides
Predictable Communication and Migration in the Quest-V Separation Kernel Ye Li, Richard West,

Predictable Communication and Migration in the Quest-V Separation Kernel Ye Li, Richard West,

Introduction Quest-V Overview Inter-Sandbox Communication Predictable Migration Conclusions Predictable Communication and Migration in the Quest-V Separation Kernel Ye Li, Richard West, Zhuoqun Cheng, Eric Missimer Boston University 1 / 29

870 views • 29 slides
How Predictable is Information Diffusion? Travis Martin, Jake Hofman, Amit Sharma, Ashton

How Predictable is Information Diffusion? Travis Martin, Jake Hofman, Amit Sharma, Ashton

How Predictable is Information Diffusion? Travis Martin, Jake Hofman, Amit Sharma, Ashton Anderson, and Duncan Watts How Predictable is Information Diffusion? 1 / 36 How far will this spread? How Predictable is Information Diffusion? 2 / 36

807 views • 51 slides
Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs Jinkyu Jeong

Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs Jinkyu Jeong

Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs Jinkyu Jeong Sungkyunkwan University (SKKU) Source: Gyusun Lee et al., Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs, USENIX

672 views • 48 slides
The Benefits and Burdens of Nuclear Latency by Mehta and Whitlark Andrew Malandra Possible

The Benefits and Burdens of Nuclear Latency by Mehta and Whitlark Andrew Malandra Possible

The Benefits and Burdens of Nuclear Latency by Mehta and Whitlark Andrew Malandra Possible Effects of Latency The Background Virtual Deterrence Latency Provocation Theory Theory You cant deter challengers Essentially its the

273 views • 24 slides
Bibliography on Synchronous Elastic (aka Latency Insensitive) Systems July 20, 2009 Latency

Bibliography on Synchronous Elastic (aka Latency Insensitive) Systems July 20, 2009 Latency

Bibliography on Synchronous Elastic (aka Latency Insensitive) Systems July 20, 2009 Latency insensitive designs [CMSV01,CSV02,CSV03,CM04,BMdS06a,Sve04,VA09] SELF implementation and compilation to elastic designs [CKG06,CK07,HB08] Interlock

313 views • 5 slides
STORM AND LOW-LATENCY PROCESSING www.inf.ed.ac.uk Low latency processing Similar to data

STORM AND LOW-LATENCY PROCESSING www.inf.ed.ac.uk Low latency processing Similar to data

STORM AND LOW-LATENCY PROCESSING www.inf.ed.ac.uk Low latency processing Similar to data stream processing, but with a twist Data is streaming into the system (from a database, or a network stream, or an HDFS file, or ) We want to

573 views • 31 slides
Latency and Throughput Latency (of task): Time elapsed between start of the task and

Latency and Throughput Latency (of task): Time elapsed between start of the task and

Latency and Throughput Latency (of task): Time elapsed between start of the task and and its finish Example : Travel from Houston to NY Takeoff to Landing Arrival at Airport to Baggage pickup Leave for Airport to Arrival at

506 views • 12 slides
Run-time interrupts latency detection in real-time systems Julien Desfossez Michel Dagenais

Run-time interrupts latency detection in real-time systems Julien Desfossez Michel Dagenais

Run-time interrupts latency detection in real-time systems Julien Desfossez Michel Dagenais December 2015 cole Polytechnique de Montreal Latency-tracker Kernel module to track down latency problems at run-time Simple API that can be

686 views • 16 slides
Seek Novelty Personality Environment Predictable Unpredictable Seek Stability Seek Novelty

Seek Novelty Personality Environment Predictable Unpredictable Seek Stability Seek Novelty

Seek Novelty Personality Environment Predictable Unpredictable Seek Stability Seek Novelty Predictable Unpredictable Seek Stability ? X ? ? 1. Establish 1RM (performance) 2. Plan the training block/phase 3. Rinse and repeat How to

726 views • 60 slides
Failures, Latency and Happy Eyeballs Contributions Takeaway Content Caches Latency IPv6

Failures, Latency and Happy Eyeballs Contributions Takeaway Content Caches Latency IPv6

Motivation Vaibhav Bajpai March 20, 2019 https://doi.org/10.1109/TNET.2019.2895165 March 2019 IEEE/ACM Transactions on Networking Garching bei Mnchen, Germany NetSys 2019 (Invited Talk) Technische Universitt Mnchen Failures, Latency

648 views • 16 slides
Green Latency-aware Data Deployment in Data Centers: Balancing Latency, Energy in Networks and

Green Latency-aware Data Deployment in Data Centers: Balancing Latency, Energy in Networks and

Green Latency-aware Data Deployment in Data Centers: Balancing Latency, Energy in Networks and Servers Yuqi Fan, Hongli Ding, Donghui Hu School of Computer and Information Hefei University of Technology August 18, 2014 School of Computer &amp;

351 views • 10 slides
Centers Parmjeet Singh, Myungjin Lee Sagar Kumar, Ramana Rao Kompella Latency-critical

Centers Parmjeet Singh, Myungjin Lee Sagar Kumar, Ramana Rao Kompella Latency-critical

Enabling Flow-level Latency Measurements across Routers in Data Centers Parmjeet Singh, Myungjin Lee Sagar Kumar, Ramana Rao Kompella Latency-critical applications in data centers Guaranteeing low end-to-end latency is important Web

586 views • 15 slides
Reducing input latency on the web bit.ly/reduce-input-latency W3C Games Workshop - June 2019

Reducing input latency on the web bit.ly/reduce-input-latency W3C Games Workshop - June 2019

Reducing input latency on the web bit.ly/reduce-input-latency W3C Games Workshop - June 2019 Navid Zolghadr (nzolghadr@chromium.org) Chromium Input Dev Chromium input dev Goals Smooth user interaction and scrolling across the

295 views • 18 slides
Experiments for Time-Predictable Execution of GPU Kernels Flavio Kreiliger, Joel Matjka, Michal

Experiments for Time-Predictable Execution of GPU Kernels Flavio Kreiliger, Joel Matjka, Michal

Experiments for Time-Predictable Execution of GPU Kernels Flavio Kreiliger, Joel Matjka, Michal Sojka and Zdenk Hanzlek OSPERT 2019 July 9, 2019, Stuttgart, Germany F. Kreiliger et al. Experiments for Time-Predictable Execution of GPU

788 views • 50 slides
Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency Jialin Li, Naveen

Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency Jialin Li, Naveen

Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency Jialin Li, Naveen Kr. Sharma , Dan R. K. Ports and Steven D. Gribble February 2, 2015 1 Introduction What is Tail Latency? What is Tail Latency? 2 Introduction

1.01k views • 73 slides
The State of Composite : a Customizable Component-Based OS for Predictable, Reliable, and

The State of Composite : a Customizable Component-Based OS for Predictable, Reliable, and

The State of Composite : a Customizable Component-Based OS for Predictable, Reliable, and Scalable Computation Gabriel Parmer gparmer@gwu.edu The George Washington University (GWU) OSPERT 2013 Researchers include Qi Wang, Jiguo Song, Jakob

389 views • 38 slides
No Tradeoff Low Latency + High Efficiency Christos Kozyrakis http://mast.stanford.edu

No Tradeoff Low Latency + High Efficiency Christos Kozyrakis http://mast.stanford.edu

No Tradeoff Low Latency + High Efficiency Christos Kozyrakis http://mast.stanford.edu Latency-critical Applications A growing class of online workloads Search, social networking, software-as-service (SaaS), Occupy 1,000s of servers in

597 views • 48 slides
TCP Options for Low Latency: Maximum ACK Delay and Microsecond Timestamps Neal Cardwell Yuchung

TCP Options for Low Latency: Maximum ACK Delay and Microsecond Timestamps Neal Cardwell Yuchung

TCP Options for Low Latency: Maximum ACK Delay and Microsecond Timestamps Neal Cardwell Yuchung Cheng Eric Dumazet IETF 97: Seoul, Nov 2016 Motivation: lower latency, higher throughput Datacenters with commodity 10Gbps Ethernet: RTT &lt;100

1.16k views • 10 slides
FAILURE AT NETFLIX VELOCITY Cannot Connect to the Netflix Service 0 0 Ms % IMPACT LATENCY

FAILURE AT NETFLIX VELOCITY Cannot Connect to the Netflix Service 0 0 Ms % IMPACT LATENCY

THE ART OF CHAOS ENGINEERING FAILURE AT NETFLIX VELOCITY Cannot Connect to the Netflix Service 0 0 Ms % IMPACT LATENCY 1 50 Ms % IMPACT LATENCY 50 50 Ms % IMPACT LATENCY 100 50 Ms % IMPACT LATENCY 25 250 Ms % IMPACT

746 views • 63 slides

More recommend