Chandler Carruth explains how Google uses Clang and MapReduce to automate large-scale C++ API refactoring across millions of lines of code.
llvm.org/devmtg/2011-11/
—
Clang MapReduce -- Automatic C++ Refactoring at Google Scale - Chandler Carruth, Google
Slides:
—
Google has over 100 million lines of code, and our biggest programming language is C++. We have a single, shared codebase developed primarily on mainline. We build every binary and all of its libraries from scratch every time, allowing us to incrementally evolve APIs and libraries over time. The entire development process is extremely incremental in nature, and even API-breaking changes are a regular occurrence. However, for core libraries used throughout the codebase, this development model is a huge challenge: how do we incrementally evolve an API in use by tens of thousands of other libraries? The answer is to use Clang to automatically refactor APIs and their users across the codebase. How do we scale Clang up to possibly the single largest unified codebase in the world? The same way Google scales anything else: MapReduce. By coupling Clang’s library design and architecture to existing Google infrastructure we can automatically compile, analyze, and refactor the entire Google codebase in minutes. In this talk, I will dive into the challenges of refactoring C++ code, how we’re using Clang and making it even better at solving them, and how we scale these solutions to the size of our codebase.
—
Why it's here
1 person recommends this
What people say. Cited as a prime example of how large-scale automated refactoring is achieved at Google using compiler tooling.
Moments people linked to
S
u/Saefrochin r/rust • Recommends
On “What's the big deal with bounds checking?”
I think both you and /u/OS6aDohpegavod4 are misunderstanding the scale of programming at Google. Data on this is of course hard to come by. But [in 2011, they could run an automated refactoring across all C++ code at Google in 10 minutes by running clang in a MapReduce.] Since then, the scale of the problem has grown, but the ability to refactor all of Google is fundamental to their operations. This is mentioned by both of Matt Kulukundis's talks on replacing the hashmap implementation, and basically every time Titus Winters talks. They're not compiling Tensorflow, they're compiling Chrome, Ad Delivery (a monolith larger than Chrome), and a whole load of other code besides. And in her excell…
