Catch 'em All: Efficient Massive Tabular Data Handling with Tablecloth by Mey Beisaron

Mey Beisaron explores the Clojure library Tablecloth, explaining how its dataset and column structures achieve high performance and memory efficiency.

In this tech talk, we will explore the Clojure library Tablecloth and dive deep into its main data structures, namely dataset and columns. We will examine the implementation behind these data structures and explore how this library achieves efficient massive tabular data handling while maintaining a simple and user-friendly API. To make things more fun, (who doesn't want to have fun?) we will be using Pokémon data sets as examples throughout the talk. We will start by introducing the dataset data structure, which is used to represent a collection of rows and columns. We will discuss how Tablecloth's dataset is implemented to support a wide range of operations such as filtering, sorting, and grouping, all while maintaining high performance and memory efficiency. Next, we will dive into the columns data structure, which represents a single column in a dataset. We will examine the column's internal representation, which allows for flexible and efficient operations such as selecting, transforming, and aggregating values. By the end of this talk, you will have a deeper understanding of how Tablecloth works and how it can help you handle large and complex data sets with ease. Whether you are a seasoned Clojure developer or just getting started with the language, you won't want to miss this opportunity to catch 'em all with Tablecloth! Mey Beisaron Mey is a Software Engineer, a Public speaker and a Mentor. When she is not spending her weekends coding games or speaking at tech conferences, she experiments with algorithms or playing Super Mario Bros on the Nintendo Switch. As she recently joined the Infrastructure forces at Forter, Clojure is not her main development language anymore, but it will forever be her favorite one. Mey is a sworn Star Wars fan. May the force be with you. Twitter: twitter.com/ladymeyy LinkedIn: linkedin.com/in/ladymey/ Video sponsor – Ada Beat adabeat.com Merch If you want to spread functional programming and support the channel, buy something from the shop: funcprogsweden.myspreadshop.net/ The chapters: 00:00 Welcome and introduction by Magnus 02:43 Presentation start Catch 'em All: Efficient Massive Tabular Data Handling with Tablecloth 03:59 100B/Events per day with Clojure 05:48 Introduction of Mey Beisaron 06:33 Agenda for Catch 'em All: Efficient Massive Tabular Data Handling with Tablecloth 07:27 Data handling in Clojure with Tablecloth 10:33 Play with Pokémon datasets 12:02 Comparing Tablecloth to other ways in Clojure 21:59 Tablecloth deep dive 32:51 What else can I do with Tablecloth? 33:36 Big thanks to Daniel Slutsky #funcprogsweden