Threads at Scale - Daniel Spiewak

Daniel Spiewak explains how JVM threads interact with hardware, how asynchronous I/O works, and how to improve tail latencies in high-scale services.

Multi-threaded concurrency has been a strength of the JVM since Java 1.2, and its implementation in that runtime has been a major driver behind the JVM’s pervasive adoption in modern, highly distributed, high-RPS backend services. Just about everyone who works on the JVM writes software that fundamentally relies on threads, but very few understand how to conceptualize the performance and tuning implications of this deceptively simple construct. In this talk, we will embark on a journey that takes us from the raw hardware all the way up to effect systems and other high-level userspace abstractions, and along the way, we will discover how threads interact, how asynchronous I/O works (and doesn’t work!), and learn One Weird Trick for dramatically improving tail latencies in almost any I/O bound, high-scale backend service. Visit Sphere.it for more talks from the Sphere.it conf 2022. ✨