programming-talkscurated list on GitHubRecommends
Under “DevOps”
I got 10 trillion problems, but logging ain't one 00:25:17 by **John Graham-Cumming** (2015)
0:00 / 0:00
dotScale 2015 - John Graham-Cumming - I got 10 trillion problems, but logging ain't one
dotconferences
23K subscribers • 8 videos on Hallway
- Recommended by 2 people: 1 on Hacker News and 1 curated list.
- Presented by John Graham-Cumming
Advanced. Deep dive for people who know the field.
6.7K views11 years ago2 recommendations
Distributed systemsSystems programmingScaling and large-scale architecturePerformance and optimization
John Graham-Cumming explains how Cloudflare handles and extracts signals from ten trillion log lines per month using Nginx, LuaJIT, and Go.
Filmed at dotscale.io in Paris on June 8, 2015. More talks on thedotpost.com Help us caption & translate this video! amara.org/v/GkTo/Why it's here
2 people recommend this, 1 disputes itWhat people say. Highly recommended for its insights into scaling real-time analytics and using LuaJIT for high-performance web traffic processing.
i336_on Hacker News • Recommends
On “Ask HN: Whats your favorite HN thread from 2015?”
I discovered a moderately old video from CloudFlare hiding in there, and had a rather big TIL moment. It's one thing to read "LuaJIT is one of the fastest dynamic languages" etc blah blah on link ... ...it's quite another to learn that EVERY SINGLE CLOUDFLARE REQUEST - all 4 million/sec of them - goes through a LuaJIT-based attack detection system. Oh. Video: link to this video (from link ) Found in the linked thread here: link
mdanielon Hacker News • Disputes
On “How CloudFlare extracts a signal from 10 trillion log lines each month [video]”
What is the deal with the recent "thedotpost.com" envelope for perfectly valid YouTube URLs? I want to flag this post just for that. link to this video
mtourneon Hacker News • Mentions
On “OpenResty – A fast web app server by extending Nginx”
Rewriting the hodge podge of C, Perl and other custom bits of Nginx almost entirely in Lua and hiring its maintainer Yichun in the process where important steps for scaling up the CloudFlare's CDN and make sure each node is running as hot as possible. More on this from dotScale 2015: link to this video
i336_on Hacker News • Mentions
On “Hello Lua”
…rded PostgreSQL) to power realtime graph generation, 12TB SSD per node - 100+ machines for realtime analytics running Go (and presumably LuaJIT) - LuaJIT seems to be the realtime "hmm, interesting" flagging system, while Go is used for in-depth intelligent analysis that's (I assume) slightly behind realtime. Also, the video is actually over here - link to this video --- I'm curious to understand what tail latency is, and very interested to learn more about language latency and performance in general. What subjects would you recommend I learn about / read? (I would describe myself as interested from a hobbyist-intermediate standpont, b…
Artemis2on Hacker News • Mentions
On “Quantifying the Impact of “Cloudbleed””
For reference, two years ago Cloudflare was serving more than three million requests per second0. I can only imagine this figure has gone up a lot. 0: link to this video
exikyuton Hacker News • Mentions
On “Ask HN: If I Close My Data Centers, What About the People/Jobs Lost?”
…some definition of "from scratch", by Quanta. Presumably there's a bit of a decision-making process in there, and presumably Quanta handles everything related to the "why is the devkit sample broken again #51383" side of things and ships a more or less finished product (maybe even with finished enclosures). My reference point for this knowledge is link to this video (2015). Regarding SAS, maybe past a certain ordering quantity (or anything similar that would reveal a massive operation) the market would expect certain behavior like requesting value-adds, ordering with only one supplier, or other things that would generate lock-in. I of course have absolutely no idea [if this sort of thinking is reasonab…