u/rahuul_shin r/Zig • Recommends
Practical Data Oriented Design used in Zig compiler itself
0:00 / 0:00
#data-orienteddesign#zig#cachelocality
Andrew Kelley: A Practical Guide to Applying Data Oriented Design (DoD)
ChimiChanga
1.5K subscribers • 1 video on Hallway
- Recommended by 6 people: 4 on Hacker News, 1 on Lobsters and 1 in engineering communities.
- Presented by Andrew Kelley, creator of Zig
Intermediate. Assumes working knowledge.
415K views2 years ago6 recommendations
Systems programmingSoftware designData-oriented designPerformance and optimizationMemory management
Andrew Kelley explains practical data-oriented design techniques to reduce memory footprint and improve cache friendliness, using the Zig compiler as a case study.
Copyright: Belongs to Handmade Seattle (https ://vimeo.com/649009599). I'm not the owner of the video and hold no copyright. And the video is not monetized. In this video Andrew Kelley (creator of Zig programming language) explains various strategies one can use to reduce memory footprint of programs while also making the program cache friendly which increase throughput. At the end he presents a case study where DoD principles are used in Zig compiler for faster compilation. References: - CppCon 2014: Mike Acton "Data-Oriented Design and C++": youtube.com/watch?v=rX0ItVEVjHc - Handmade Seattle: handmade-seattle.com/ - Richard Fabian, 'Data-Oriented Design': dataorienteddesign.com/dodbook/ - IT Hare, 'Infographics: Operation Costs in CPU Clock Cycles': ithare.com/infographics-operation-costs-in-cpu-clock-cycles/ - The Brain Dump, 'Handles are the better pointers': floooh.github.io/2018/06/17/handles-vs-pointers.htmlWhy it's here
6 people recommend thisWhat people say. Recommended as an excellent practical guide to data-oriented design and memory optimization in systems programming.
Moments people linked to
emmanueloga_on Hacker News • Recommends
On “Rust for tokenising and parsing”
That talk is great, but I remember some discussion later about Go actually NOT using this technique because of goroutine scheduling overhead and/or inefficient memory allocation patterns? The best discussion I could find is 1. Another great talk about making efficient lexers and parsers is Andrew Kelley's "Practical Data Oriented Design" 2. Summary: "it explains various strategies one can use to reduce memory footprint of programs while also making the program cache friendly which increase throughput". -- 1: link 2: link to this video
iamcreasyon Hacker News • Recommends
On “Writing memory efficient C structs”
This reminds of an amazing talk by Mike Acton on Data oriented design: link And there is a related talk by Andrew Kelley implementing these ideas on Zig toolchain: link to this video
Rendelloon Hacker News • Recommends
On “Your data model is your destiny”
…last few days redesigning it according to Data-Oriented Design. I'm trying to get the wall-clock time down by more than an order of magnitude. I'm not sure how it's going to turn out, wish me luck :) Since I mentioned DoD, these three links will probably come up in conversation: Mike Acton's famous performance talk: link DoD in the Zig compiler: link to this video The DoD book: link…
gethlyon Hacker News • Recommends
On “Why Object of Arrays beat interleaved arrays: a JavaScript performance issue”
SoA is nothing new. Not sure why it needs to be even discussed... Anyway, here's few videos of interest: link link to this video Odin also supports SoA natively link
Rendelloon Hacker News • Recommends
On “Everyone Should Know SIMD”
…dups from that, but that's also where your compiler (or you as a programmer) can get incredible SIMD gains. SoA is not a silver bullet as it may not align well with your access patterns, but it can great to add to your toolkit. --- Mike Acton: Data-Oriented Design and C++ : link Andrew Kelley: A Practical Guide to Applying Data Oriented Design : link to this video Richard Fabian: Data-Oriented Design : link…
u/-not_a_knifein r/C_Programming • Mentions
On “Is C good enough for independent game development?”
…feel a bit like paint by numbers at first since I'm leaning on Raylib so heavily but it's fun and still challenging to get games working and off the ground.
If you're committed to start from the very bottom and make everything, here are some resources I found:
https://www.raylib.com/
https://handmadehero.org/
https://www.computerenhance.com/
[video
[video
https://www.gameenginebook.com/
There is a lot going on in these resources but these are the direction I want to work towards and you sound like you have a similar mindset. I'd like to point out that raylib is written in C and you can see in the examples how you could make a 2d or 3d game with it. My thoughts are that I would…
u/igors84in r/Zig • Mentions
On “I made an x86_64 machine code generator”
Very nice start. Be sure to watch this talk from Andrew Kelly [video since you might decide to optimize your data storage a bit and that is better done earlier than later :)
u/miyoyoin r/Zig • Mentions
On “Why not backticks for multiline strings?”
It being doable in one line doesn't mean it isn't possible in multiple.
The actual reason for strings being done this way is for general parsing/tokenization purposes, since every line can be parsed out of context, everything can be parallelized (and, in the example above, highlighted) without depending on anything else. This is why there are also no multiline comments.
A lot of decisions in Zig are taken specifically to make parts of, or the entire compiling process faster, and this is one of them. The following videos may give additional context:
[video
[video
u/tj6200in r/ProgrammingLanguages • Mentions
On “October 2024 monthly "What are you working on?" thread”
I watched a talk recently from Andrew Kelley explaining some data-oriented design optimizations that were implemented in the zig compiler. This inspired me to do a major rework of my lexer token representation and my parse tree representation in a "Struct of Arrays" format to conserve memory and improve cache locality. Link to the video on youtube if anyone is interested
u/Pretty_Jellyfish4921in r/ProgrammingLanguages • Mentions
On “How do you get good error reporting once you've stripped out the tokens?”
Here’s a great talk where Andrew Kelley talks about that (not sure the time right now, but the video is worth to watch)
link to this video
u/woupiestekin r/ProgrammingLanguages • Mentions
On “October 2025 monthly "What are you working on?" thread”
I have been experimenting with data-oriented design in Rlox again, my bytecode interpreter for Lox--the language of Crafting Interpreters--written in Rust. I got the idea last year from this YouTube videoby Andrew Kelley, the inventor of Zig. It has its origins in the Entity Component System and game programming.
For Rlox, data-oriented design means pooling all the objects--not the objects in the language but the BoundMethods, Classes, Closures, etc., in the implementation of the machine--into structs of arrays. So the machine works with offsets into these arrays instead of pointers. Some advantages:
* Little or no unsafe code is needed, since offsets are just integers, not raw pointer…
u/Judsen_Hembreein r/Zig • Mentions
On “AVL Tree, get closest scores to target matchmaking algorithm implementation from scratch in zig”
Nj man. Would be cool to see your old python dug up.
You sound primed for link to this video
(Talks about how much software essentially abuses how good hardware is)
And
link to this video
(Data oriented design)
Cool thing with zig is how easy it makes it to manage memory. Arena allocation and whatnot.
brabelon Hacker News • Mentions
On “Aro – Zig's new C compiler”
The talk I am referring to is from 2024 and shows some metrics improving by 30% or more. And Andrew said that's not yet shipped: link to this video So, are you sure about that?? EDIT: ok, the talk was uploaded just last week, but someone in the comments clarified that: > The original talk is from 2021.. Thank you for reuploading. I feel stupid now :/
Rendelloon Hacker News • Mentions
On “Forth: The programming language that writes itself”
TL;DR: I'm trying to Forth a Lisp. This is long winded, but maybe you have some thoughts here. I've been building a DOM-builder API recently in Rust. The existing options (there are many) tend to use textual templating, which can't reason well about anything, or basic macros, which never support indentation. I wanted something that was just code, where I'd be in full control on the indentation (or lack thereof) programmatically. The closest equivalent is Python's Dominate 1, though its indentation mechanisms are buggy. So I built a system using the traditional tree where Nodes own other Nodes at random addresses, and I built a renderer that renders those nodes and concatenates their string…