Skip to content
Hallway
0:00 / 0:00

#handmadehero#customallocators#arenaallocation

Handmade Hero Day 626 - Cleaning Up Traversables

Molly Rocket

148K subscribers • 32 videos on Hallway

YouTube
  • Recommended by 3 people: 1 on Lobsters and 2 in engineering communities.
  • A classic. People kept recommending it from 2021 to 2024.

Advanced. Deep dive for people who know the field.

15K views5 years ago3 recommendations

Game developmentSystems programmingGame programmingMemory managementC

A live-coding session focusing on cleaning up traversables and memory management in a custom game engine.

Day 626 of coding on Handmade Hero. See handmadehero.org for details.

Why it's here

3 people recommend this

What people say. Engineers frequently reference Handmade Hero's memory management style to advocate for arena allocation and grouped element thinking over individual mallocs.

Moments people linked to

u/skeetoin r/C_Programming • Recommends

On “How to learn and write secure C code from the start?”

First step, enable sanitizers when you build for all testing and development: $ cc -g3 -fsanitize=address,undefined ... These place low-to-moderate cost checks throughout your program so that you can catch mistakes sooner. Platform support varies, but with GCC and Clang, Undefined Behavior Sanitizer is available in some form on all platforms. They don't have the best defaults, so adjust those as well: export ASAN_OPTIONS=abort_on_error=1:halt_on_error=1:detect_stack_use_after_return=1 export UBSAN_OPTIONS=abort_on_error=1:halt_on_error=1 The options make it a little more thorough, and errors will trap in your debugger so that you can investigate when they occur. Speaking of w…
75View on Reddit

u/N-R-Kin r/C_Programming • Recommends

On “Should malloc be used as rarely as possible?”

> Is there a performance difference if I use malloc 5 times to allocate 10 Bytes each, or one time to allocate 50 Bytes The short answer is "yes". But the bigger reason to prefer single malloc is that it simplifies memory management by managing them in groups rather than individually. So in you example, instead of managing 5 different allocations - now you need to manage only one. But context obviously matters, if I knew my upper bound is 50 bytes, I wouldn't even bother with `malloc`, I'd just create a automatic variable on the stack (e.g `char bytes50;`). This is both faster than malloc and also doesn't involve manually having to `free` the allocation. On the other hand if the uppe…
47View on Reddit

u/skeetoin r/C_Programming • Recommends

On “Should I try to limit the number of calls to malloc() by allocating large chunks and dishing out portions as needed?”

If you're just rewriting your own general purpose allocator, then there's little point to doing so. However, a general purpose allocator isn't tuned to your specific use case. It has to account for cases you don't care about (multithreading, free order, etc.), and will spend time — probably most of its time — [doing things you don't need it to do]. You know your problem and its constraints, so you can do better than `malloc()` — a lot better in fact. You mentioned linked list nodes. Rather than use a general purpose allocator, you could allocate a large buffer of many nodes (slab allocation, ex.) and draw nodes from this buffer. It will be more efficient than `malloc()`, both in terms of sp…
31View on Reddit

u/skeetoin r/C_Programming • Recommends

On “It’s 2024, you start a new C project, how are you managing strings safely?”

For me, no null terminators except when interfacing with something that uses them. Instead a string is a struct with a pointer and a length: typedef struct { uint8_t *data; ptrdiff_t len; } str; Such objects are passed by copying. For example: str trim(str); bool equals(str, str); uint64_t hash(str); str clone(str, arena *); Usually a program only needs a handful of string functions shaped like these, and not some heavyweight string library. It has no ownership semantics, and strings do not have individual lifetimes. In general, [objects are managed as groups] using region-based allocation and a fixed number of regions. That elimina…
24View on Reddit

u/skeetoin r/C_Programming • Recommends

On “Lifetime and ownership attributes in gcc and clang”

The problems solved by smart pointers and such are the result of *individual element thinking*. A better solution is *grouped element thinking*: Don't make lots of little allocations but create large collections all at once. If a function returns an object, it simply belongs to a pool and you don't need to manage its individual lifetime.
11View on Reddit

u/N-R-Kin r/C_Programming • Recommends

On “So you want custom allocator support in your C library”

Writing malloc wrappers for debugging is rather outdated nowadays due to ASan's existence. The bigger reason to use custom allocators is because they simplify memory management by allowing you to manage allocations in groups instead of micro-managing each allocation. This effectively means you need to only manage a *handful of lifetimes* rather than hundreds or thousands of individual allocations.
10View on Reddit

u/skeetoin r/C_Programming • Recommends

On “Is making an ownership system worth it?”

Ownership, reference counting, smart pointers, etc. exist to solve the problem of managing O(n) lifetimes. In most cases you can change the problem around so that [you instead have O(1) lifetimes], which is simpler and faster.
10View on Reddit

vfoleyon Lobsters • Recommends

Individual element thinking vs grouped elements thinking
9View on Lobsters

u/N-R-Kin r/C_Programming • Mentions

On “Yet Another Linked List Implementation in C”

A couple minor issues (at a glance): 0. Allocations aren't checked. If you're assuming that allocations cannot fail (not a bad assumption on most hosted environments) you should still do something like `if (ptr == NULL) abort();` rather than deref-ing a null pointer, which is undefined-behavior. 0. Identifiers starting with underscores are reserved by the standard. If you compile with clang's `-Wreserved-identifer` it would warn you about these. 0. Rather than returning error code for user-error, you should consider using `assert()` instead (e.g if someone calls list_push with a null list). Returning an error code to a caller that has proven itself to be buggy makes things *harder to debug*…
8View on Reddit

u/skeetoin r/C_Programming • Mentions

On “Need a code review”

By that I'm talking about programs designed such that every object has a distinct lifetime to be managed one way or another. Better to organize your program so that [groups of objects share a lifetime] (~10 minutes started at linked moment) and allocating/deallocating is done in waves. Undergraduate C courses, and similar, place undue attention on balancing every `malloc` with a `free`, but that stuff just isn't a concern in well-designed programs. Don't worry too much about this stuff right now, and worry more about getting more of the fundamentals under your feet. Just know that down the road memory management isn't as big a deal as people make it, and doesn't have to work as shown in the…
5View on Reddit

u/N-R-Kin r/C_Programming • Mentions

On “Ownership Pattern”

> The idea of a safe structure is to avoid segmentation failures and double frees, through an ownership control, where even if everyone tries to delete the object, only the current owner can If the code is structured so poorly that "everyone" is trying to delete an object that they shouldn't - then you've already lost. Adding some over-engineered solution on top of that is just fixing the *symptoms* of the problem, not the root cause. In my experience, if you have two objects that are dependent on each other then about 80%+ of the time, what that means is that they should share the same lifetime. I.e they should be managed in group rather than individually. And even in cases where you h…
4View on Reddit

u/skeetoin r/C_Programming • Mentions

On “Best Practice Questions and Request for Source Critique of Huffman Coding Compressor”

At `src/decoder.c:59` the program read uninitialized `_Bool` values, which is both undefined behavior and probably wrong regardless. This was caught by UBSan. It failed an assertion on my very first run. Sure, this isn't compressed input, but that shouldn't matter! $ ./decoder sample-files/engineering /dev/null decoder: src/bitbuffer.c:33: buffer_drop_left_bits: Assertion `number_of_bits <= buffer->length' failed. I also got this assertion any time I mixed outputs and inputs from different architectures (i.e. the format depends on the host due to that `fwrite` on raw memory). I wanted to fuzz the decoder, but I couldn't get it to run reliably enough to produce useful results. I fu…
3View on Reddit

u/N-R-Kin r/C_Programming • Mentions

On “string implementation”

vstring* vstring_new(); If you have a function that doesn't accept any arguments, you need to explicitly use `(void)` in the declaration - otherwise it has a different meaning in C where it will accept *any* number of arguments. (*vstring_obj)->hdr.cap = cap; Mostly a style thing, but you can also use the sprong operator instead of using paren. Other than that, I don't have anything else to add that hasn't already been pointed out by others in the thread. - - - Although I think something probably should be said about the design and/or the objective of the library itself. I generally dislike "string" that has a `capacity` member. Why? Because it tangles *allocation* with the string,…
3View on Reddit

u/skeetoin r/C_Programming • Mentions

On “Easiest video format to output to?”

I'm glad you liked it! > what do you think of C23? Outside of a couple of areas where vendors have been stubborn, it makes little difference to me. I haven't put it into practice yet, but one such feature is `#embed`. GCC and Clang have left it to Binutils to provide the functionality (e.g. via `ld`, or via `.incbin`), where it's half-broken and/or awkward. Another is `u8` prefixes for `unsigned char` strings, as part of an improved paradigm. Oh, and declarations after case statements (i.e. labels) is overdue, and GCC and Clang have dragged their feet far too long. The C standard really isn't so important anymore, and simply writing whatever GCC and Clang supports is perfectly reasonable.…
2View on Reddit

u/M_e_l_v_i_nin r/C_Programming • Mentions

On “[RANT] C++ developers should not touch embedded systems projects”

Because they are a very shortsighted solution to the problem of keeping track of freeing memory allocated on the heap. It's shortsighted because it assumes that the only way you allocate memory is 1 object at a time and then you free memory 1 object at a time when it goes out if scope, and smart pointers take away the worry that you may have forgotten to free memory before it goes out of scope so you don't get "memory leak". But what a normal person would do (who was never exposed to c++ ways of thinking) would notice which objects get created together and go out of scope at the same time, and he would notice that it would be better to think about allocating and freeing memory in bulk(memor…
2View on Reddit

u/skeetoin r/C_Programming • Mentions

On “[deleted by user]”

* Embrace grouped element thinking (general) * SIMD intrinsics (general) * Threading. OpenMP is a nice shortcut. (general) * I/O Completion Ports (Windows-specific)
1View on Reddit