This session will be a deep dive into different data movement scenarios using first-class tooling in Azure data echo-system and Azure Data Factory (ADF). We will learn about handy new features and data connectors, while copying and transforming datasets from a Data Lake and SQL Relational Database storages. Thus, tune in to learn about latest developments in Microsoft Azure data transformation services.
In my second session, I will share following journey with you:
Abstract of the upcoming session provided below 😉
We will learn about what is ETL and ELT stands for in data world, and how Azure Data Factory (ADF) service could help you. Along the way, we will look into inner-workings and fundamentals of a cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale.
Finally, we will conclude the session with ADF demo and Q&A
Please let me know, what topics are you interested in?
Last week, I was able to knockout the second “Azure Data Engineer” exam and get the certification. Most importantly, I got a chance to have a deep dive into Azure Data technologies from a holistic solutioning perspective. The main focus areas of this certification are neatly visualized below:
Those exam objectives, are high level focus areas that cover DP-200 and DP-201 exams. The first exam is more technical and has focus toward implementation while the second one is shifting gears toward design and solutioning options based on project requirements.
As dealing with data solutions was not my primary work responsibility, I was really impressed by depth of service options, capabilities and tools that could accelerate many data solutioning projects.
Starting an “Azure Data Engineering” learning journey will help any cloud professional to get deeper understanding on designing different data solutions based on project requirements. In fact, it helped me to get a well informed understanding technical limitations in each solution; like deciding between different data processing and/or storage options and their respective security configurations. Well, enough about intro, let’s look into what it means to be an “Azure Data Engineer” 😑🤔
Azure Data Engineer vs Database Administrator
A Database Administrator’s (aka, DBA) job is to ensure everything is working smoothly with things like performance tuning and monitoring, data migrations of third party systems, performing backups, checking performance, and load balancing, and everything to do with databases operations. On the other hand, DBA could also perform data engineering tasks in a small or mid-size organisations.🤠
Azure data engineers are responsible for data-related implementation tasks that include provisioning data storage services, ingesting streaming and batch data, transforming data, implementing security requirements, implementing data retention policies, identifying performance bottlenecks, and accessing external data sources. They are responsible for data-related design tasks that include designing Azure data storage solutions that use relational and non-relational data stores, batch and real-time data processing solutions, and data security and compliance solutions.👩🚀👨🎓
Getting ready for each exam 📆
There is no right or wrong starting point with this certification. I would start with the one that you find more familiar. Here is the exam flow diagram from the official Microsoft learn website:
Exam study resources for DP-200: Implementing an Azure Data Solution
This exam really tests your ability to demonstrate good understanding of setting-up a data processing pipeline and configurations of each component. A lot of questions were on optimizing, operating, loading and reading data from Azure SQL DB and Synapse Analytics (formerly SQL DW), I found them tricky and quite challenging.
The DP-200 exam has around 40-48 questions and is somewhat similar to other Azure Associate exams. The exam duration is fixed around 180 minutes with additional 30 minutes for reading instructions, signing the non-disclosure agreement, and giving feedback to each question at the end.
Expect different types of questions (single choice, multi-choice, binary choice and sorting order) and one or two case studies.
Exam study resources for DP-201: Designing an Azure Data Solution
My exam experience with DP-201 was much smoother, as mainly questions were focused on design related topics and solutioning options. As an Azure Architect – you may find them familiar and a bit easier than the previous exam.
This exam was 150 minutes long and had 42 questions, with much more case studies (i got three on my exam). There were few questions with set of drop-down choices that were easy and tricky at the same time.
That is it folk, i tried to share and illustrate my exam experiences in this post. I hope you will find it helpful and apply to your “Data Engineer-ing” journey.
Please, feel free to share your experience or thoughts, as i am planning to keep this post up to date with your valuable contributions going forward. #keeplearning