Skip to content
Hallway
0:00 / 0:00

#jvm#catseffect#threads

Threads at Scale - Daniel Spiewak

Sphereit by VirtusLab

809 subscribers • 3 videos on Hallway

YouTube
  • Recommended by 1 person: 1 in engineering communities.
  • Presented by Daniel Spiewak, creator of Cats Effect

Advanced. Deep dive for people who know the field.

7.5K views3 years ago1 recommendation

Systems programmingDistributed systemsConcurrency and parallelismOperating systemsPerformance and optimization

Daniel Spiewak explains how JVM threads interact with hardware, how asynchronous I/O works, and how to improve tail latencies in high-scale services.

Multi-threaded concurrency has been a strength of the JVM since Java 1.2, and its implementation in that runtime has been a major driver behind the JVM’s pervasive adoption in modern, highly distributed, high-RPS backend services. Just about everyone who works on the JVM writes software that fundamentally relies on threads, but very few understand how to conceptualize the performance and tuning implications of this deceptively simple construct. In this talk, we will embark on a journey that takes us from the raw hardware all the way up to effect systems and other high-level userspace abstractions, and along the way, we will discover how threads interact, how asynchronous I/O works (and doesn’t work!), and learn One Weird Trick for dramatically improving tail latencies in almost any I/O bound, high-scale backend service. Visit Sphere.it for more talks from the Sphere.it conf 2022. ✨

Why it's here

1 person recommends this

What people say. Recommended as an excellent deep dive into JVM concurrency, threading models, and tail latency optimization.

Moments people linked to

u/yanshuaiin r/scala • Recommends

Threads at Scale - Daniel Spiewak
57View on Reddit

u/KagakuNinjain r/scala • Mentions

On “How can I learn the type level , correctness, and compile-time error checking of the scala language?”

I recommend these Daniel Spiewak videos: Threads at Scale, an excellent introduction to performance issues of threads on the JVM. Case for Effect Systems, explains why we need effect systems like Cats Effect and ZIO. Java 21 virtual threads provide some of the benefits, but not all.
8View on Reddit

u/joel5in r/scala • Mentions

On “Cats-Effect 3.6.0”

link to this video "And there you have it, this is actually how Node.js works. And this is actually very, very cool, because this doesn't waste any threads at all." - djspiewak, 2022. So, yeah, they kind of did, but with their own take on it. Do you think they are not aware of Seastar? I think they are.
4View on Reddit

u/KagakuNinjain r/scala • Mentions

On “Did you guys think that?”

Some large telcos and streaming media companies use Scala, you don't hear about it much. Examples are Comcast, Verizon and Disney Streaming. The Typelevel stack is extremely performant, and can scale with no problems. It gives you a highly expressive pure FP API. Cats Effect 3 is inspired by Tokio, and optimized to make efficient use of small thread pools, if you are able to avoid blocking IO. This is in contrast to the traditional Java approach of large thread pools and blocking IO (Java 21 obviously changes things, but from what I have read, virtual threads still have some serious footguns). The ZIO library is also popular. You might watch some Daniel Spiewak videos, he explains it much…
2View on Reddit

u/kbn_in r/scala • Mentions

On “Need more knowledge on thread level functionality”

This talk also dives fairly deeply into the details, going even lower level than the JVM itself. link to this video
2View on Reddit

u/KagakuNinjain r/scala • Mentions

On “Scala without effect systems. The Martin Odersky way. ”

>I'm not sure this is his claim, but anyway, this makes no sense. The JVM can do things in native code. Something you obviously can't do in something that runs on the JVM… I'm working from my failable memory, and apolgize in advance for any inaccuracies... I don't have the motivation to search discord and re-watch videos. I am also ignorant of the underlying technology myself. He said that JVM virtual threads were designed with to be backwards compatibile. Such that a Java dev can make some minor code changes, and everything will just "work". So technically, virtual threads could equal or exceed the performance of CE3, but they chose not to do that. OS schedulers are designed for fairn…
1View on Reddit