u/Trick-Rip-9065in r/dataengineering • Recommends
Rust in Data Engineering
Interesting talk. Anybody using anything other than python in their production DE usecases? What’s it like?
0:00 / 0:00
Rust for Python data engineers — Karim Jedda
EuroPython Conference
36K subscribers • 31 videos on Hallway • Conference channel
- Recommended by 1 person: 1 in engineering communities.
- Presented by Karim Jedda
Intermediate. Assumes working knowledge.
9.5K views3 years ago1 recommendation
Programming languagesDatabases and dataRustPythonData engineering and warehouses
Karim Jedda explores how data engineers can leverage Rust alongside Python to build high-performance, scalable data pipelines.
[EuroPython 2023 — South Hall 2B on 2023-07-19] ep2023.europython.eu/session/rust-for-python-data-engineers Python is a popular language for data engineering but has some limitations in performance, concurrency, and production deployments. The Rust programming language offers powerful alternatives with strong compile-time and memory safety guarantees. In this talk, I'll explore how data engineers can leverage Rust to build high-performance data pipelines and processing systems. I'll cover the Rust ecosystem for data work, including frameworks and libraries for working with data formats, databases, streaming systems, and scientific computing. By combining Rust and Python, data engineers can harness the benefits of both languages and build robust end-to-end data systems that scale to meet demanding production needs. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License creativecommons.org/licenses/by-nc-sa/4.0/