0:00 / 0:00
1000x: The Power of an Interface for Performance by Joran Dirk Greef
TigerBeetle
14K subscribers • 8 videos on Hallway
- Recommended by 1 person: 1 on Lobsters.
- Presented by Joran Dirk Greef
- Made by the vendor. The company whose product it covers made or sponsored it.
- An opinion. It argues one side of a question engineers disagree on.
Advanced. Deep dive for people who know the field.
43K views1 year ago1 recommendation
Systems programmingDatabases and dataPerformance and optimizationDatabase internalsAPI design
Joran Dirk Greef argues that massive performance gains come from designing the right high-level interfaces rather than just low-level optimizations.
As systems programmers, we tend to think of performance as all the low-level techniques we can and should apply in the implementation of a system. For example, static memory allocation, amortization, zero-deserialization, zero-copy, direct memory access, memory locality, data-oriented design, cache-line alignment, explicit limits, concurrency, parallelism, pipelining, scheduling, hardware acceleration, branch prediction, vectorization, loop unrolling, profiling, io_uring, or more explicit systems languages such as Zig or Rust. However, while these techniques can increase performance by 10% or even 2x or 10x, and while they are important (and all add up!), they typically do not achieve the two to three order of magnitude gains required to really move the needle, and break the sound barrier. Worse, no matter the skill of the craftsperson, or the quality of their implementation, these techniques may be powerless to push past the bottleneck of broken protocols or flawed function signatures, if these interfaces are not co-designed in view of the system. Yet, for all their power to destroy performance, interfaces are also the doors and windows of our architectures, and as mediators of information exchange to and from our implementations, they have power to unlock performance! Join us as we deconstruct and reconstruct one of the world's most popular interfaces, to show the power of an interface for performance. linkedin.com/in/joran-dirk-greef/ x.com/jorandirkgreef tigerbeetle.com/Why it's here
1 person recommends thisu/BosonColliderin r/softwarearchitecture • Mentions
On “What about dedicated database engineers?”
…o in that case is have the application server build a batch of transactions per tick and send them to the DB to be handled by a stored procedure.
Here's a good talk about this from the authors of tigerbeetle, though you can have postgres close much of the latency gap claimed in the video by using batched stored procedures and the unnest trick:
[video…