Welcome to Data Engineering. In the last decade, pretty much every industry has become digital. Digital communications are pervasive, and digital data is replacing pieces of paper as the primary mechanism for how information is stored in healthcare, finance, manufacturing, education, tech, and pretty much all industries. But with this tidal wave of data comes risks and also new challenges for how to store and process all this data to create value Whether your organization is looking to improve how it serves customers or outmaneuver the competition, having good data pipelines is increasingly critical, which is why there's also skyrocketing demand for data engineers. I'm thrilled to welcome Joe Reis as your instructor for this specialization. Joe's a data engineer and data architect with decades of experience serving companies in their data needs. He's also a professor, podcaster, and also co-author of this wonderfully written and best-selling book, Fundamentals of Data Engineering. Joe, I remember you once described yourself as a recovering data scientist. What does that mean? A recovering data scientist is, I guess, a nickname I was given back in maybe the mid-2010s. So this is about the time when data science was becoming very, very hot and popularized. And my background was in machine learning, analytics, and similar roles. And what I had noticed was companies were hiring data scientists without giving them the proper support and foundational infrastructure to succeed at their jobs. You'd often be going into a job as a data scientist with no data upon which to do the science. So and seeing this over and over again, I started realizing that what was also important to help data science succeed was data engineering. And so this is about building the foundation of infrastructure, data quality, and providing the means for data scientists to succeed at their job. So although I got my start in data science, I moved into data engineering to help solve these problems for data scientists. Once you have a large data set, when a data scientist can pose a question and get an answer in five minutes an hour, as opposed to a week or a month, that drives massive efficiency differences. And sometimes the difference is whether the data engineering is there to support that rapid development. That's absolutely right. Yeah. And so what I've been seeing over the years is as companies have seen data as an invaluable asset. Now, what I'm starting to notice is companies are investing in building that foundational part again to get that time to value down from days, months, weeks down to hours, minutes, sometimes seconds. When your data set is a small CSV file on your laptop, it's OK. There are still complexities there. But once you go beyond that small set of data, the complexity entering the pipeline makes a huge difference in performance. Oh, absolutely. And with the rise of AI today, I think it's more important than ever to really take a data centric first approach, which we can talk more about. And so I'm glad that finally the recognition is there that data and the infrastructure and the methodologies to support data are finally here. Because again, especially with AI becoming front and center with every company and every workload, this really feels like it's the time for data engineering. But I want to ask a question to you. You popularized a term called data centric AI. What does that mean for data engineers? Yeah. So data centric AI, as you all know, is the discipline of systematically engineering the data to build a successful AI system. And I think for data set sizes ranging from that small CSV file on your laptop to the massive data sets stored in data warehouses, including the way that large language model is trained today with trillions of tokens. The ability to engineer the data, to get the AI, the machine learning algorithm, the data it needs to succeed. That's a wildly exciting, exploding discipline. And so I'm actually excited about what you'll be sharing about data centric AI in this specialization as well. Thank you. The job of a data engineer is a really important one and is also a specialized skill. So can you say more about the things someone will need to learn to gain that skill? Absolutely. So this program is all about framework principles, getting you to think like a data engineer. And but it's not only that. We're excited to be partnering with Amazon Web Services, also known as AWS, on this program to provide you with hands on practical exercises to develop your skills, building data systems on the cloud. Today, most data engineers are building systems on the cloud, and right now AWS is the most widely used public cloud. So getting familiar with building systems on AWS will really be an asset to you as a data engineer. So I'm really excited that these courses will be combining the foundational principles of data engineering, the kind of stuff that's platform agnostic, together with practical hands on exercises. If you're an aspiring data engineer, then these courses are for you. Or maybe you're already working as a data engineer and you're looking to further refine your skills. These courses are also great for you if you are working adjacent to data engineering, whether that's in data science, machine learning, software engineering, analytics or other data related roles. In terms of prerequisite expectations, the only thing you really need is prior working experience with data and Python. If you have some experience with SQL and cloud tools, that's great, too, but it's not necessarily required. We'll show you what you need to know and provide you with suggested resources to learn more. This program comprises four courses. In this first course, we'll be looking at the big picture, forming a mental framework for data engineering and spinning up some cloud data pipelines. In the second course, we'll focus on ingestion of data from source systems, data ops and orchestration of your data pipelines. The third course is all about data storage in the cloud and how storage is an essential part of all stages of a data engineering lifecycle. And the fourth course is about modeling, transforming and serving data for end use cases. So there's a lot to cover and I'm looking forward to sharing all this with you. All that sounds fantastic. Let's go on to the next video and get started.