A Short Summary of the Last Decades of Data Management • Hannes Mühleisen • GOTO 2024

Hannes Mühleisen reviews the evolution of data management systems, open source engines, and open data formats over the last decades.

This presentation was recorded at GOTO Amsterdam 2024. #GOTOcon #GOTOams gotoams.nl Hannes Mühleisen - Co-founder & CEO of DuckDB Labs @hannesmuehleisen RESOURCES twitter.com/hfmuehleisen linkedin.com/in/hfmuehleisen github.com/hannes hannes.muehleisen.org ABSTRACT Data systems have come a long way from the monolithic vendor hell of the 90s. Data is no longer held hostage with arbitrary licensing models. Open Source engines, open data formats, and huge cloud computing resources have fundamentally changed how we think about data. In the same time, a large variety of specialized systems have popped up, from systems supporting semi-structured data to the hottest and latest vector databases. In my talk, I will try to summarize the most important trends, including those that did not make it in the end. I will take attendees on a journey through this trillion dollar industry and its ever-continuing search for new and exciting ways to manage data. [...] TIMECODES 00:00 Intro 01:36 DuckDB 02:38 Ancient history 11:25 One size fits all? 19:05 No SQL! 22:39 Hadoop 24:02 Hive 25:45 MongoDB 27:09 Cassandra 28:55 Apache Spark 30:54 Tables, SQL, ACID 33:28 New SQL! 35:21 Relational eats everything 42:55 What refuses to be eaten? 43:57 Big data is dead? 48:24 Summary 49:20 Outro Download slides and read the full abstract here: gotoams.nl/2024/sessions/3118 RECOMMENDED BOOKS Mark Needham, Michael Hunger & Michael Simons • DuckDB in Action • amzn.to/45QwSli Simon Aubury & Ned Letcher • Getting Started with DuckDB • amzn.to/3VPk4qS Wei-Meng Lee • DuckDB: Up and Running • amzn.to/3xJsHeu twitter.com/GOTOcon linkedin.com/company/goto- instagram.com/goto_con facebook.com/GOTOConferences #DuckDB #Data #DataManagement #HistoryOfDataManagement #VectorDatabase #Transactional #Analytical #SQL #MapReduce #Hadoop #MongoDB #Cassandra #ApacheSpark #ACID #PostgreSQL #HannesMuehleisen Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at gotopia.tech Sign up for updates and specials at gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. youtube.com/user/GotoConferences/?sub_confirmation=1