PyO3: From Python to Rust and Back Again (with David Hewitt)

David Hewitt explains how PyO3 bridges Python and Rust, detailing the performance benefits and underlying mechanics.

There’s huge pressure on Python at the moment to get faster, ideally without changing at all. One increasingly–popular way of achieving that impossible task is to push the performance critical code down into C, C++, or Rust. And this week we’re focussing on the Python route, as we take a look at PyO3. David Hewitt’s the principal committer to PyO3, and he joins us to go through the easy parts, the hard parts, and the works in progress, giving us an insight into how Python and Rust work under the hood, and quite how much work it takes to make them work as one. – Become a Supporter on Patreon: patreon.com/DeveloperVoices Become a Supporter on YouTube: youtube.com/@DeveloperVoices/join PyO3 User Guide: pyo3.rs/v0.22.0/ PyO3 on Github: github.com/PyO3/pyo3 Polars: pola.rs/ Tokio: tokio.rs/ Trio: trio.readthedocs.io/ Robyn: github.com/sparckles/Robyn Faster CPython: github.com/faster-cpython Maturin: maturin.rs/ David on Mastodon: fosstodon.org/@davidhewitt David on Twitter: x.com/davidhewittdev Kris on Mastodon: mastodon.social/@krisajenkins Kris on LinkedIn: linkedin.com/in/krisjenkins/ Kris on Twitter: x.com/krisajenkins -- 0:00 Intro 3:09 How David Got Involved in PyO3 7:01 So Starting With Python To C... 9:20 Taking That To Rust 17:37 Calling Rust from Python 20:54 Understanding PyO3's Place In The Stack 21:52 Memory Safety 30:37 Rust's Lifetimes 35:07 Applying Lifetimes to Python Code 38:24 Let's Get Into Error Handling 55:55 The Performance Cost 1:01:19 Threading 1:16:05 David's Motivations 1:19:22 Supporting Python's Sub-Interpreters 1:24:14 The Limits of Compile-Time Guarantees 1:27:32 Getting Started with PyO3 1:33:01 Outro