programming-talkscurated list on GitHubRecommends
Under “System Architecture”
Scalability at YouTube 00:38:43 by **Mike Solomon** (long intro; starts at 9:12 minutes in) (2012)
0:00 / 0:00
- Recommended by 3 people: 1 on Hacker News and 2 curated lists.
- Presented by Mike Solomon, early YouTube engineer
Intermediate. Assumes working knowledge.
50K views14 years ago3 recommendations
Distributed systemsSoftware designScaling and large-scale architectureSoftware architecturePython
Mike Solomon shares how YouTube scaled its infrastructure, database architecture, and Python codebase to handle massive traffic.
Shannon -jj Behrens, Mike Solomon This talk covers scalability at YouTube. It's given by one of the original engineers at YouTube, Mike Solomon. It's a rare glimpse into the heart of YouTube which is one of the largest websites in the world, and onWhy it's here
3 people recommend thisWhat people say. Highly recommended as a classic case study on scaling web applications and database sharding.
Moments people linked to
Coding Interview Universitycurated list on GitHubRecommends
Under “System Design, Scalability, Data Handling”
link to this video
gitarron Hacker News • Recommends
On “We need more PyCon US 2013 submissions”
YouTube (owned by Google) is mainly Python. Here a talk from Pycon 2012, the technical part starts around 10 min. in: link to this video
u/sydneytrozin r/compsci • Mentions
Jitter in distributed systems as a grad research topic
I was watching Scalability at YouTube, and Mike Solomon mentions that with a distributed system as large as YouTube, they have to add jitter to prevent creating "thundering herds." He introduces the example of a cache expiration causing everyone trying to view a video to synchronize on waiting for a fresh version of the video, where "everyone is actually removing entropy from the system, so you have to put some back in."
I'm sure YouTube isn't the first system to come across this, but I thought this need for jitter was a pretty cool concept, and it could result in some interesting grad work. Is anybody doing this?
gitarron Hacker News • Mentions
On “Poll: What's Your Favorite Programming Language?”
The real beauty of Python is the fact that it is easy for beginning programmers, yes, but it has also a lot to offer for experienced coders concerning all the advanced things they might need. It's no accident that services like Dropbox1 or YouTube2, both arguably very complex systems, are relying on Python as their language of choice. 1 link 2 link to this video
tbatteriion Hacker News • Mentions
On “Security incident update”
>Facebook and YouTube were both created later, and in PHP youtube was always done in python as far as I understand. link to this video But yeah, back in 2002-3 coldfusion was an option.
Jugurthaon Hacker News • Mentions
On “Ask HN: When to switch startup backend from Python to Golang?”
Scalability at YouTube: link to this video The Python is the less likely think that will suffocate you. What did you suffer from in past projects with each of these languages?
Leftiumon Hacker News • Mentions
On “Ask HN: Can I create a mobile and Web App using a Python/Python Framework?”
For reference, YouTube runs on Python1,2,3: > 1. Python and Django: YouTube’s backend is predominantly written in Python, offering a balance of performance and readability. > 2. Google Cloud Platform... > 3. Java and C++: YouTube also utilizes Java and C++ for specific backend services, as they provide better performance for certain tasks. --- A long time ago, I looked into these Python frameworks: - link - link 1: link 2: link 3: link to this video
teitoklienon Hacker News • Mentions
On “Ask HN: Video streaming is expensive yet YouTube "seems" to do it for free. How?”
They improve their stack on multiple fronts 1 - They use their own custom hardware encoders now optimised for their scale (previously they’d use a ton of intel cpu for encoding) This is often not possible for smaller startups to have their own custom hardware, but one could setup their own Xilinx FPGA based hardware accelerated encoding servers, Xillinx Alveo Boards are popular for media acceleration, they support both ffmpeg and gstreamer for acceleration needs. Nvidia Nvenc encoding is decent too but mostly for the most popular encoding types, and honestly an fpga based accelerator is probably better after cpu unless you have gpus lying around. 2 - Much lower bandwith costs, a lot of start…