Predictable Latency Adventures in Java Concurrency Martin Thompson - PowerPoint PPT Presentation

One big file that goes on forever?

No!!! Page faults, page cache churn, VM pressure, ...

Clean Dirty Active Header Header Message Message Header Header Message Message Header Header Message Message Tail Header Message Header Message

Do publishers need a CAS?

File Header Message 1 Header Message 2 Header Message 3 Tail <<XADD>> Message X Message Y

File Header Message 1 Header Message 2 Header Message 3 Message X Tail <<XADD>> Message Y

File Header Message 1 Header Message 2 Header Message 3 Message X Message Y Tail

File Header Message 1 Header Message 2 Header Message 3 Header Message X Message Y Tail

File Header Message 1 Header Message 2 Header Message 3 Header Message X Message Y Padding Tail

File File Tail Header Message 1 Header Message 2 Header Message 3 Header Message X Message Y Padding

File File Header Header Message 1 Message X Header Tail Message 2 Header Message 3 Header Message Y Padding

Have a background thread do zero’ing and flow control

Burst Length = 1: RTT (ns) Prod # Mean 99% 1 167 189 Baseline (Wait-free) 1 281 361 ConcurrentLinkedQueue 2 381 559 3 444 705 1 170 194 ManyToOneRingBuffer 2 256 279 3 283 340 1 192 210 Aeron IPC 2 251 286 3 290 342

Aeron Data Message Header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-------------------------------------------------------------+ |R| Frame Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-------------------------------+ | Version |B|E| Flags | Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-------------------------------+ |R| Term Offset | +-+-------------------------------------------------------------+ | Session ID | +---------------------------------------------------------------+ | Stream ID | +---------------------------------------------------------------+ | Term ID | +---------------------------------------------------------------+ | Encoded Message ... ... | +---------------------------------------------------------------+

Burst Length = 100: RTT (ns) Prod # Mean 99% 1 721 982 Baseline (Wait-free) 1 12,916 15,792 ConcurrentLinkedQueue 2 25,132 35,136 3 39,462 56,768 1 3,773 3,992 ManyToOneRingBuffer 2 11,286 13,200 3 27,255 34,432 1 4,436 4,632 Aeron IPC 2 7,623 8,224 3 10,825 13,872

Less “False Sharing” - Inlined data vs Reference Array Less “Card Marking”

Is avoiding the spinning “CAS” loop is a major step forward?

Burst Length = 100: RTT (ns) Prod # Mean 99% 1 721 982 Baseline (Wait-free) 1 12,916 15,792 ConcurrentLinkedQueue 2 25,132 35,136 3 39,462 56,768 1 3,773 3,992 ManyToOneRingBuffer 2 11,286 13,200 3 27,255 34,432 1 4,436 4,632 Aeron IPC 2 7,623 8,224 3 10,825 13,872

Predictable Latency Adventures in Java Concurrency Martin Thompson - PowerPoint PPT Presentation

A Quest for Predictable Latency Adventures in Java Concurrency Martin Thompson - @mjpt777 If a system does not respond in a timely manner then it is effectively unavailable 1. Its all about the Blocking 2. What do we mean by Latency 3.

CROSS-LAYER CROSS-LAYER LATENCY-AWARE AND -PREDICTABLE LATENCY-AWARE AND -PREDICTABLE DATA

Predictable Latency Adventures in Java Concurrency Martin Thompson - @mjpt777 If a system does

Rocksteady: Fast Migration for Low-Latency In-memory Storage Chinmay Kulkarni , Aniraj Kesavan,

Quest-V: A Secure and Predictable System for IoT and Beyond Richard West richwest@cs.bu.edu

Is It Latency or Do You Just Suck? Latency Can Kill: Precision and Deadline in Online Games

Predictable Communication and Migration in the Quest-V Separation Kernel Ye Li, Richard West,

How Predictable is Information Diffusion? Travis Martin, Jake Hofman, Amit Sharma, Ashton

Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs Jinkyu Jeong

The Benefits and Burdens of Nuclear Latency by Mehta and Whitlark Andrew Malandra Possible

Bibliography on Synchronous Elastic (aka Latency Insensitive) Systems July 20, 2009 Latency

STORM AND LOW-LATENCY PROCESSING www.inf.ed.ac.uk Low latency processing Similar to data

Latency and Throughput Latency (of task): Time elapsed between start of the task and

Run-time interrupts latency detection in real-time systems Julien Desfossez Michel Dagenais

Seek Novelty Personality Environment Predictable Unpredictable Seek Stability Seek Novelty

Failures, Latency and Happy Eyeballs Contributions Takeaway Content Caches Latency IPv6

Green Latency-aware Data Deployment in Data Centers: Balancing Latency, Energy in Networks and

Centers Parmjeet Singh, Myungjin Lee Sagar Kumar, Ramana Rao Kompella Latency-critical

Reducing input latency on the web bit.ly/reduce-input-latency W3C Games Workshop - June 2019

Experiments for Time-Predictable Execution of GPU Kernels Flavio Kreiliger, Joel Matjka, Michal

Tales of the Tail Hardware, OS, and Application-level Sources of Tail Latency Jialin Li, Naveen

The State of Composite : a Customizable Component-Based OS for Predictable, Reliable, and

No Tradeoff Low Latency + High Efficiency Christos Kozyrakis http://mast.stanford.edu

TCP Options for Low Latency: Maximum ACK Delay and Microsecond Timestamps Neal Cardwell Yuchung

FAILURE AT NETFLIX VELOCITY Cannot Connect to the Netflix Service 0 0 Ms % IMPACT LATENCY