This month DataEngBytes, Australia’s ultimate data engineering conference, was back for 2021 with more talks than ever.
There was something for everyone, whether you work in a start-up, an established company, or you are just learning the art and science of data engineering.
Talks aligned to four themes:
– Data at rest – data warehousing, data lakes, and data storage.
– Data in motion – event-driven architecture, data pipelines, and streaming.
– Data for machine learning – data pipelines for machine learning and managing data artefacts.
– Data you trust – data management, lineage, testing, and security.
Here are just some of the talks from the three-day online conference.
The first day of DataEngBytes featured talks from industry experts on data privacy, productionising machine learning models, building data products, data pipelines, and MLOps.
- The Rise & Downfall of the Data Engineer
- Data Engineers: Privacy Is Your Problem
- From experiment to production – a journey of a machine learning model
- Chain-speed inference for Computer Vision Pipelines
- Trust, Knowledge and your Data: building a great data product at KADA
- Shift-left testing : Building reliable Data Pipelines
- Beginners guide to Azure MLOps
Day two kicked off with our keynote on Data Mesh which led on to talks on data quality, reliability, building a data platform and a panel on all things data in the enterprise.
- What is a Data Mesh – And How Not To Mesh it Up
- Data Quality with Great Expectations and Airflow in a Reverse-ETL World
- Reliable data engineering made easy
- Data quality: the key to long term happiness
- Building a Data Platform at Assembly Payments
- Streaming data analytics with Apache Flink
- Teleport Data: The future of data
The final day of the conference focused on tools for the modern data stack with Apache Flink, dbt, Snowflake and data warehousing.
- From telescope to data centre: adventures in astronomical data pipelines
- Gone Streaming: dbt+Materialize
- A Single Data Platform for All of Your Workloads
- Putting your data warehouse to work – Reverse ETL & Operational Analytics
- Snowflake and dbt — Our Journey to the Cloud
- Situational Intelligence with Analytics in Motion
- Albero – Decision Tree: when to use what – the full Azure data estate
That wraps up this year’s conference! Hopefully, we’ll be back to in-person events real soon but in the meantime check out the DataEngBytes YouTube channel for more data engineering talks.