Thomas La Piana explains how data teams can increase efficiency by adopting software engineering and DevOps best practices.
ABOUT THE TALK:
A high performance data team can be a force-multiplier for any organization. It is no secret that data and analytics teams usually don’t get the amount of resources and support that they desire. So how does one effect the most change with the smallest team and fewest resources possible? The best place to look for inspiration is software engineering.
It is our belief that data engineering, science, and analytics is a sub-field of software engineering. As this field matures we should avoid reinventing any wheels and learn from the best practices of software engineering and DevOps. The proliferation of high-quality open source data tools is making this easier than ever.
ABOUT THE SPEAKER:
Thomas La Piana is a Data Engineer at GitLab. GitLab is a single application for the entire devops life-cycle. From CI/CD to security and monitoring, GitLab provides tools to cover everything. Previously, Thomas was a Data Engineer for OrderMyGear, a Dallas, TX based startup focused on revolutionizing the team sports apparel industry.
Thomas graduated from the University of Texas at Dallas with a Bachelors in Political Science. He wrote and presented two papers on Chinese politics before deciding that software is what he was truly passionate about.
ABOUT DATA COUNCIL:
Data Council (datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups.
FOLLOW DATA COUNCIL:
Twitter: twitter.com/DataCouncilAI
LinkedIn: linkedin.com/company/datacouncil-ai
Facebook: facebook.com/datacouncilai
Eventbrite: eventbrite.com/o/data-council-30357384520 -
🎟️ GET YOUR TICKET TO AI COUNCIL 2026 🎟️
Meet the world's top AI infrastructure minds where architects of AI share what works. Three days of high-quality technical talks and meaningful interactions.
→ aicouncil.com/sf-2026
⚡ FIND US:
X: x.com/AICouncilConf
LinkedIn: linkedin.com/company/aicouncilconf/
Website: aicouncil.com/
Why it's here
1 person recommends this
What people say. Recommended as a great resource on DataOps and applying software engineering practices to data pipelines.
J
u/Jwelch25in r/dataengineering • Recommends
Data-Ops-ish Design and Workflow Walkthrough
In another thread, u/brendersplide asked about building a regression test suite. In response to my comment, there were a few individuals who wanted to know more about the system I am currently running. ( u/isleepbad and u/htrp) It's more than what should be in a reply so I thought I would do a post for anyone else interested.
WARNING: This is a long post.
DISCLAIMER: This is, by no means, the best way to do things. We are constantly evolving and tweaking.
As a little context, I run data and analytics at a start-up-ish financial company and have a very small team. In order to combat that constraint, we have done a lot to automate our workflow.…
