Job Overview
An Azure Data Engineer designs and develops data pipelines using Azure services. They build high-performance solutions with Apache Spark on Azure Databricks, collaborating with stakeholders to deliver clean datasets.
Key Responsibilities
* Data Pipeline Design: Develop scalable ETL/ELT pipelines using Azure services for efficient data processing
* Spark Solutions: Build high-performance data processing solutions using Apache Spark on Azure Databricks
* Collaboration: Work with data scientists, analysts, and stakeholders to understand data requirements and optimize workflows
* Performance Optimization: Ensure data pipelines are optimized for performance and cost efficiency in a cloud environment