Skip to content
0:00 / 0:00

#c++#benchmarking#compileroptimization

CppCon 2015: Chandler Carruth "Tuning C++: Benchmarks, and CPUs, and Compilers! Oh My!"

CppCon

189K subscribers • 101 videos on Hallway • Conference channel

YouTube
  • Recommended by 4 people: 3 on Hacker News and 1 curated list.
  • A classic. People kept recommending it from 2015 to 2020.
  • Presented by Chandler Carruth

Intermediate. Assumes working knowledge.

188K views11 years ago4 recommendations

Systems programmingProgramming languagesC++Performance and optimizationCompilers and interpreters

Chandler Carruth explains how to write effective C++ benchmarks and analyze performance without compiler optimizations getting in the way.

Cppcon.org — A primary use case for C++ is low latency, low overhead, high performance code. But C++ does not give you these things for free, it gives you the tools to control these things and achieve them where needed. How do you realize this potential of the language? How do you tune your C++ code and achieve the necessary performance metrics? This talk will walk through the process of tuning C++ code from benchmarking to performance analysis. It will focus on small scale performance problems ranging from loop kernels to data structures and algorithms. It will show you how to write benchmarks that effectively measure different aspects of performance even in the face of advanced compiler optimizations and bedeviling modern CPUs. It will also show how to analyze the performance of your benchmark, understand its behavior as well as the CPUs behavior, and use a wide array of tools available to isolate and pinpoint performance problems. The tools and some processor details will be Linux and x86 specific, but the techniques and concepts should be broadly applicable. -- Chandler Carruth leads the Clang team at Google, building better diagnostics, tools, and more. Previously, he worked on several pieces of Google’s distributed build system. He makes guest appearances helping to maintain a few core C++ libraries across Google’s codebase, and is active in the LLVM and Clang open source communities. He received his M.S. and B.S. in Computer Science from Wake Forest University, but disavows all knowledge of the contents of his Master’s thesis. He is regularly found drinking Cherry Coke Zero in the daytime and pontificating over a single malt scotch in the evening. -- Videos Filmed & Edited by Bash Films: BashFilms.com Work at Hudson River Trading (HRT): tinyurl.com/safxfctf

Why it's here

4 people recommend this

What people say. Recommended as an eye-opening guide to benchmarking C++ and understanding how compilers optimize or eliminate benchmark code.

Moments people linked to

humanrebaron Hacker News • Recommends

On “Why I don't spend time with Modern C++ anymore”

Here's a few: - "Tuning C++: Benchmarks, and CPUs, and Compilers! Oh My!" link to this video - "Understanding Compiler Optimization" link - "Optimizing the Emergent Structures of C++" link - "Efficiency with Algorithms" link The first one is really good if you've never benchmarked anything in a Linux environment. Or if you otherwise want to learn how to investigate how C++ gets turned into machine code.

programming-talkscurated list on GitHubRecommends

Under “C++”

Tuning C++: Benchmarks, and CPUs, and Compilers! Oh My! 01:29:53 by **Chandler Carruth** (2015)

kbensonon Hacker News • Recommends

On “Do you know how much your computer can do in a second?”

Interestingly, there's quite a bit that's needed to ensure you bypass some compiler optimizations. Here's an ey-opening (for me) talk1 on benchmarking C++ and compilers which touched on it. 1: link to this video

thomashusaon Hacker News • Recommends

On “When Bloom filters don't bloom”

There is a very interesting talk 1 by Chandler Carruth that I stumbled upon last weekend that is very much related to this 1 link to this video

Chandler Carruthon Stack Overflow • Mentions

On “Enforcing statement order in C++”

I'd like to try to provide a somewhat more comprehensive answer after this was discussed with the C++ standards committee. In addition to being a member of the C++ committee, I'm also a developer on the LLVM and Clang compilers. Fundamentally, there is no way to use a barrier or some operation in the sequence to achieve these transformations. The fundamental problem is that the operational semantics of something like an integer addition are totally known to the implementation. It can simulate them, it knows they cannot be observed by correct programs, and is always free to move them around. We could try to prevent this, but it would have extremely negative results and would ultimately fa…

u/camel-cdrin r/C_Programming • Mentions

On “A C Standard Library”

…cs. It's always preferable to run a benchmark multiple times and output the average, deviation and other useful metrics, over just outputting the time of a single run. You should also consider adding compiler specific `DoNotOptimize` macros, that help the programmer from stopping the compiler the code that is supposed to be benchmarked. (check out link to this video 3. Think about implementing a custom PRNG for `randnum_gen`, since `rand()` is quite slow and has low quality randomness on many platforms, mostly for historical reasons. Resources to check out: rand considered harmful, https://pcg-random.org, https://prng.di.unimi.it/, https://romu-random.org, and for a bit of self promo…

u/__n2tin r/C_Programming • Mentions

On “Trying to understand cache effects for reading and writing into arrays”

…rol over the assembly generated. And this is not to mention that the same assembly, on distinct hardware, may have distinct performance behavior. I think that it is only reasonable to talk about how cache misses or branch predictors or wtv affects our code when we are analysing a change in benchmark. I learned about perf in the following video: link to this video It is quite cool: depending on your hardware you may even have a flag that explicitly counts cache misses, and even tells what kind of cache miss you had. He starts using perf around 30min. I remembered I did not like the website, but learned some cool things from --help and manpage.…

u/bgs11235in r/ProgrammingLanguages • Mentions

How to measure (Question) I would first like to start where I'm coming from using Chandler Carruth's 2015 Cpp con talk. There is a problem where a bottleneck was traced to an access control system that checked user queries against a long table of rules to see what access they were allowed, a problem very similar to how a firewall or file system checks permissions Ken Thompson was able to get a general 20x improvement where it was outperformed five fold (100x over the first implementation) with a trivial re-ordering by just making the most used rule the first one to be looked at. My question is how should we measure our programming languages in a way where we have clean data couple…

u/skeetoin r/C_Programming • Mentions

On “Why is there no C conference?”

Every year you can think of a subset of CppCon as being like a C conference. There are a bunch of presentations that are entirely or mostly relevant to C, and not just in a broader sense of being relevant to *any* language. I plucked out a bunch from CppCon 2017 for you. These are *roughly* ordered from best to worst: * [Matt Godbolt “What Has My Compiler Done for Me Lately? Unbolting the Compiler's Lid”] * Chandler Carruth “Going Nowhere Faster” * James McNellis “Everything You Ever Wanted to Know about DLLs” * John Regehr “Undefined Behavior in 2017 (part 1 of 2)” * John Regehr “Undefined Behavior in 2017 (part 2 of 2)” * Piotr Padlewski “Undefined Behaviour is awesome!” * [Tob…

P. Mattioneon Stack Overflow • Mentions

…optimize. But I know that foo() will modify the memory somewhere, and the result will be observable eventually. Will the following prevent statement reordering and get the correct timing in this case? #include <chrono> #include <iostream> //I believe this tells the compiler that all memory everywhere will be clobbered? //(from his cppcon talk: link to this video __attribute__((always_inline)) inline void DoNotOptimize() { asm volatile("" : : : "memory"); } // The compiler has full knowledge of the implementation. s……
6View the answer

u/camel-cdrin r/C_Programming • Mentions

On “How to Properly Benchmark C Code”

One of the most important things for micro benchmarking, is to make sure that you benchmark what you are trying to benchmark. This can be achieved using something like the `DoNotOptimize`. This is a good talk on the topic: link to this video For my benchmarking library, I use macros to tell the compiler not to optimize away a value, pre-running the benchmark to "warm up" the CPU/scheduler and calculate the average and deviation of multiple runs of the benchmark.

misnomeon Hacker News • Mentions

On “Why Const Doesn't Make C Code Faster”

> Anecdote: Optimum C blew away all the other C compilers on a magazine benchmark article because it figured out that the benchmark code did nothing and deleted it. I remember this problem being discussed in a CPPCon talk a few years ago, and some ways to "trick" the (modern) compilers into not optimising away the benchmark code - I think it was Chandler Carruth's "Tuning C++" talk: link to this video

u/raoul15978in r/C_Programming • Mentions

On “[C Learning] How to measure efficiency of programs?”

Chandler Carruth gave a nice presentation at CPPCon : link to this video (C++ but should apply to C)

u/DownloadRedditin r/C_Programming • Mentions

On “How do I go about finding performance cost of inbuilt c functions?”

link to this video This should answer a lot of your questions. You could also watch this one - link to this video - for an explanation that probably is more along the lines of what a good answer would be.

u/ventuspilotin r/C_Programming • Mentions

On “What do people here usually do for benchmarking code? (rdtsc? clock_getime? QPC? etc.)”

In the video link to this video Chandler Carruth talks about benchmarking and a Google benchmark framework. You may find it interesting whether you use said benchmark framework or roll your own. "perf -record" is covered as well.

u/camel-cdrin r/C_Programming • Mentions

On “How to properly validate a code profiler?”

There is an excellent cppcon talk about benchmarking: link to this video TLDR: you can write macros that stop the compiler from optimizing the code away without disabling optimizations completely. The macros from the talk aren't the best ones you can use, because they still force the value to be on the stack. I use other macros for my benchmarking library, although they require more knowledge to use. (My benchmarking library, it isn't the best, but works for my use cases) Another thing I can recommend is running the benchmarks multiple times and printing the mean/median and the deviation.