u/GelHydroalcooliquein r/rust • Recommends
On “Why linux build of the program consumes several times more RAM?”
I'd also link his other video about profiling memory which shows how chunks of heap memory are used
0:00 / 0:00
#mevi#userfaultfd#virtualmemory
I built my own memory profiler (in Rust, on Linux)
fasterthanlime
57K subscribers • 8 videos on Hallway
- Recommended by 3 people: 3 in engineering communities.
- Presented by Amos Wenger
- Linked in the descriptionfasterthanlime/mevikoute/bytehound
Advanced. Deep dive for people who know the field.
70K views3 years ago3 recommendations
Systems programmingProgramming languagesMemory managementRustLinux
Amos Wenger explains how he built mevi, a custom memory profiler, using Linux userfaultfd to visualize virtual memory allocations in Rust.
To try everything Brilliant has to offer—free—for a full 30 days, visit brilliant.org/fasterthanlime The first 200 of you will get 20% off Brilliant’s annual premium subscription. This video was sponsored by Brilliant. Follow me on Mastodon: hachyderm.io/@fasterthanlime Support me on Patreon or GitHub: fasterthanli.me/donate Source code for the memory visualizer: github.com/fasterthanlime/mevi "The Linux Memory Manager" book (Lorenzo Stoakes): linuxmemory.org/ (Reference continue below) Contents Intro Memory fragmentation explained with cars Virtual memory 101 Observing simple programs with mevi Sponsored segment (real) Why does this Rust program leak memory, visualized mevi is Rust on the front & back userfaultfd: the reason why mevi works (slowly) The hard bits Some bigger programs seen through mevi Y'all are so good to me More references The Quest for Netflix on Asahi Linux (David Buchanan): da.vidbuchanan.co.uk/blog/netflix-on-asahi.html Making our own executable packer: fasterthanli.me/series/making-our-own-executable-packer Swiss Tables Design Notes (Abseil): abseil.io/about/design/swisstables SwissTable CppCon talk (Matt Kulukundis): youtube.com/watch?v=ncHmEUmJZf4 Live migration: en.wikipedia.org/wiki/Live_migration Chromium’s Zygote process: chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/zyg… bytehound memory profiler: github.com/koute/bytehoundWhy it's here
3 people recommend thisWhat people say. Recommended for its excellent visualization of heap memory allocations, virtual memory, and mmap behavior.
Moments people linked to
u/ear7hin r/rust • Recommends
On “Vec::with_capacity() against Vec::new() is a about same perfomance.”
I think the optimization can be even more general than "program has no other allocations". If the vec has a memory mapping to its own (this will happen once the vec gets sufficiency large), the OS can simply change the the memory mapping without having to copy. There's a good fasterthanlime video that covers this, the link is timestamped to the relevant part, but I suggest watching the whole thing:
link to this video
u/RB5009in r/rust • Recommends
On “I made a Vector for large amounts of data, that does not copy when growing, by using full mmap'd pages.”
I found it. The relevant part starts at 7:50
link to this video