Skip to content
Hallway
0:00 / 0:00

#whatsapp#erlang#mnesia

Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp

Erlang Solutions

18K subscribers • 39 videos on Hallway

YouTube
  • Recommended by 6 people: 6 on Hacker News.
  • A classic. People kept recommending it from 2014 to 2022.
  • Presented by Rick Reed

Advanced. Deep dive for people who know the field.

49K views12 years ago6 recommendations

Distributed systemsSystems programmingElixir and ErlangScaling and large-scale architectureConcurrency and parallelism

Rick Reed explains how WhatsApp scaled its messaging platform to billions of active connections using Erlang, FreeBSD, and Mnesia with a tiny team.

March 7th, 2014 Rick Reed

Why it's here

6 people recommend this

What people say. Highly recommended as a classic case study on extreme scalability and the power of Erlang for small teams.

rdtscon Hacker News • Recommends

On “Revisiting 1M Writes per second”

My other favorite "scalability" study is from WhatsApp: link to this video That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp (that walk title was create before the acquisition and was mean to imply message count, after the acquisition it got a secondary meaning). The one thing that is fascinating about it, is how small their team was compared to the volume and complexity of the operation.
View on Hacker News

kornishon Hacker News • Recommends

On “How Discord Scaled Elixir to 5M Concurrent Users”

…However, when product requirements match exceptionally well with a specialized technology, you can see things that would simply be infeasible or extremely tough using a different stack. WhatsApp + Erlang was one of those cases (watch this talk and imagine trying to recreate that system with only a handful of server engineers using any other tech: link to this video ). Discord + Elixir appears to be another. Curious if anyone has any examples that spring to mind from outside the highly concurrent messaging space.…
View on Hacker News

kornishon Hacker News • Recommends

On “Scaling Erlang Developer Experience at WhatsApp [pdf]”

One of my favorite talks is by Rick Reed about scaling Erlang at WhatsApp. What an absolute savage. He flies through an articulate and in-depth curriculum on system performance and bottleneck mitigation. The talk is called "That's 'Billion' with a 'B'" and makes for a great lunchtime watch: link to this video
View on Hacker News

rdtscon Hacker News • Recommends

On “Mnesia and CAP”

Unrelated, but maybe interesting: WhatsApp was running Mnesia and made it work fantastically. Especially with such a small team of engineers. Here is a video of Rick Reed talking about how they did it (hint it is not a one monolithic Mnesia cluster). link to this video
View on Hacker News

rbsnon Hacker News • Recommends

On “Yahoo Mail moving to React”

Just to follow up on topic of WhatsApp and Erlang, here is a presentation given at Erlang Factory 2014 about their goals with scaling WhatsApp to billions of simultaneous users. link to this video
View on Hacker News

rdtscon Hacker News • Recommends

On “Why WhatsApp Only Needs 50 Engineers for Its 900M Users”

Not "parent" but search for Erlang Factory talks they talk about it. Here is one from Rick Reed. He explains how the product works : link to this video
View on Hacker News

ddeckon Hacker News • Recommends

On “Introducing WhatsApp's Desktop App”

…here your messages are stored. Whatsapp don't retain messages/media after they've been delivered to your phone, which is a compelling privacy feature for many. It's also what allows them to serve such an enormous user base with limited hardware. Their technology stack (FreeBSD/Erlang) is pretty interesting, more info here: 2014 talks by Rick Reed: link to this video link Slides: link There's likely no technical reason why you couldn't use a pc instead of a phone for users that want to use the pc as the primary client (with the phone optionally accessing the DB on the pc in the same way that the desktop client does for the phone). Perhaps they've decided that this is a small and declining market. Ed…
View on Hacker News

otterleyon Hacker News • Recommends

On “A comment left on Slashdot”

Here's a great talk from Rick Reed (a former mentor of mine at Yahoo!) about how WhatsApp scaled their Erlang infrastructure: link Here's another talk: link to this video
View on Hacker News

rdtscon Hacker News • Recommends

On “WhatsApp Cofounder on How It Reached 1.3B Users Without Losing Its Focus”

Here is the description of their stack and how Erlang helped them by Rick Reed at Erlang Factory: link to this video Here is Jamshid Mahdavi talking about their stack as well but focusing on development, testing and shipping code. Some of the stuff they do will surprise you if you've been at a shop with a large QA team and deployment pipelines with many stages. link
View on Hacker News

dpeckon Hacker News • Recommends

On “Ask HN: What startsups made the best early-tech decisions?”

WhatsApp. Rick Reed’s “That’s Billion with a B” presentation is a good glimpse into their own view of it link to this video
View on Hacker News

ddeckon Hacker News • Mentions

On “WhatsApp doesn't understand the web”

…only store of the existing conversations. New messages sent also need to be synchronized across devices, which I presume is why it's required to keep your phone connected to the Internet. Otherwise Whatsapp would need keep conversations on their servers until they could be synced, which is very much not their model. Details on their architecture: link to this video link…
View on Hacker News

rdtscon Hacker News • Mentions

On “Mnesia and LevelDB: Liberating Mnesia from the limitations of DETS”

Klarna (the European payment processor) and WhatsApp architectures are built on Mnesia. Mnesia is Erlang's built-in distributed database. Due to its age it has a few bad corner cases, that could be fixed if it just had a better backend (the front-end API is nice, well designed, supports transactions, is built-in to the lanauge). So this effort brought in the ability to have a better backend, and make Mnesia a better option as a general purpose distributed database. Here is an talk on how WhatsApp uses Mnesia: Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp link to this video Here is an example of using Mnesia: link
View on Hacker News

hellbanneron Hacker News • Mentions

On “Ask HN: Who is not betting on Elixir and why not?”

Yes, I understand, I was just wondering if Erlang would be considered a "proven language". From my (limited) Erlang experience, Erlang & Elixir have the same core functionality. Elixir has a "nicer" syntax (I prefer Erlang's syntax but most devs I've talked to like Elixir's). So if Elixir isn't proven but the concurrency model & programming paradigms make sense, maybe Erlang is a good choice. * WhatsApp Scaling to 1 Billion users in erlang: link to this video * link
View on Hacker News

hellbanneron Hacker News • Mentions

On “Ask HN: Who is not betting on Elixir and why not?”

I haven't watched but this would be a good guess link to this video "Scaling with A B: Erlang and WhatsApp"
View on Hacker News

kbensonon Hacker News • Mentions

On “Jan Koum to leave after broad clashes with Facebook”

There's a talk one of the engineers gives1 (that someone else posted here so I'm watching it) about their architecture that's published in March of 2014, and in it he talks about ~550 servers, which includes 250 multimedia servers and 150 chat servers. The only places he mentions 16 is when he talks about the "multimedia database". I think there were 16 sharded database servers. 1: link to this video
View on Hacker News