ztellmanon Lobsters • Recommends
Everything Will Flow - A Talk on Applied Queueing Theory
0:00 / 0:00
#clojure#queueingtheory#backpressure
Zach Tellman - Everything Will Flow
ClojureTV
111K subscribers • 136 videos on Hallway
- Recommended by 6 people: 4 on Hacker News, 1 on Lobsters and 1 curated list.
- A classic. People kept recommending it from 2015 to 2019.
- Presented by Zach Tellman, software engineer
Intermediate. Assumes working knowledge.
26K views11 years ago6 recommendations
Distributed systemsSystems programmingConcurrency and parallelismSoftware architectureKafka and stream processing
Zach Tellman explains queueing theory, backpressure, and how queues behave and fail under load in Clojure systems.
Queues, it seems, are back in fashion. This is exciting, as queues let us explore problem spaces that were unreachable by pure, immutable primitives. But queues, and the execution models that drive them, are very different than what we're used to. Naive use of libraries like core.async will create brittle, unpredictable systems that fall over when we least expect. This talk will cover the basics of queueing theory, and its practical application in Clojure systems large and small.Why it's here
6 people recommend thisWhat people say. Highly recommended as an essential explanation of queueing theory, backpressure, and system behavior under heavy load.
stingraycharleson Hacker News • Recommends
On “Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart”
…try / it’s difficult to find out what’s going on in the thread pool behind it. I’ve personally settled on Zach Tellman’s Manifold library, which provides a much more sane abstraction and is fully compatible with core.async. For some addition thought and material on these topics, I highly recommend watching this talk by Zach, “Everything Will Flow” link to this video…
programming-talkscurated list on GitHubRecommends
Under “Distributed Systems”
Everything Will Flow 00:40:12 by **Zach Tellman** (2015) A
cabaconon Hacker News • Recommends
On “What happens when you add a new teller? (2008)”
See also a very nice video from ClojureWest about queues in system architectures: link to this video The queue are everywhere - your messaging queue, the threadpool, the hardware threads, and other layers of the stack and APIs you use. The video adds the interesting detail that as you add more tellers (workers) you learn of impending disaster only in the outlier p99 (or higher) latencies; by the time your p85 latency rises, you're already about to stall out.
lkrubneron Hacker News • Recommends
On “Ask HN: What are some examples of beautiful software?”
Anything by Zach Tellman: link link In terms of his ideas, watch "Always be composing": link And he offers some great thoughts about queues and backpressure in "Everything will flow": link to this video
susan_hallon Hacker News • Recommends
On “Notes on Distributed Systems for Young Bloods (2013)”
On dealing with backpressure, I would recommend what Zach Tellman writes about in his video "Everything will flow": link to this video Tellman also tells his own adventure about a system something like this: "a system that would latch up into an overload state because in overload it would log errors at a severity high enough to force the log file to be flushed on every line" He talks about some problems he faced when writing to Amazon S3, when S3 stopped accepting requests. It's a fascinating talk.
lkrubneron Hacker News • Recommends
On “Linux Load Averages: Solving the Mystery”
No. Check out this video by Zach Tellman. He talks about queues and how they break down under load. One of the least intuitive things he points out is that when you have more processors, the breakdown tends to be more of a step function: everything is running smoothly till the moment that it isn't. link to this video The point he makes arises from basic queue theory and is applicable to all kinds of systems, and how those systems react to load. It's got little to due with particular hardware and everything to do with basic math.
lkrubneron Hacker News • Mentions
On “Too small to Kafka but too big to wait: Really simple streaming in Clojure”
I like this essay but I think it is very strange that he doesn't link to the talk that Zach Tellman gave in which Tellman talks about the origins of Factual's Durable Queues: link to this video Factual's Durable Queues are a fairly minimal queue effort that is useful in cases where you need some minimal way of handling back pressure, because a service downstream has failed. In his talk, Tellman talks about the fact that he needed to write a lot of data to AWS S3, and sometimes S3 stops accepting writes, so he needed an easy re-try mechanism. I used Durable Queues everywhere in my code and find it very handy. It's useful when you need a very minimal queue that still has the backing of a file system.
lkrubneron Hacker News • Mentions
On “Complex systems collapse faster”
Queue theory suggests the more nodes you add to handle your queue then the longer your queue can survive (facing rising traffic) but the faster it will collapse when it finally runs out of capacity. Zach Tellman went over this in one of his talks and he had a nice graphic to demonstrate what this looks like: link to this video I suppose any complex system has dynamics like that, more capacity to absorb load, but then a faster collapse when capacity is surpassed. Hemmingway's Laws Of Motion applies: ”How did you go bankrupt?” Bill asked. ”Two ways,” Mike said. ”Gradually and then suddenly.” Put a bit differently by the economist Rudiger Dornbusch: ”The crisis takes a much longer time coming than you thi…