About the Role
This position requires a highly skilled Azure Data Engineer to design, develop, and maintain data pipelines using Azure Data Factory (ADF).
You will be responsible for implementing and managing data storage solutions with Azure Data Lake Storage (ADLS) Gen 2, developing data pipelines with Azure Synapse Analytics, and utilizing Azure Stream Analytics for real-time data processing and analytics.
In addition, you will ensure data quality and integrity through data governance practices, implement CI/CD pipelines with Azure DevOps for automated deployment, and collaborate with data analysts and stakeholders to meet business data needs.
The ideal candidate should have experience with Azure Functions, Azure Logic Apps, Azure Stream Analytics, and Azure API Management (APIM), as well as understanding of Microsoft Purview for data governance and compliance.
Proficiency in PySpark and Azure Data Bricks is required, along with strong experience with Azure Data Factory, Azure Synapse Analytics, and ADLS Gen 2.
Additionally, problem-solving skills, attention to detail, and excellent communication skills are essential for this role.
Key Responsibilities
* Data Engineering: Design, develop, and maintain data pipelines using Azure Data Factory (ADF)
* Data Storage: Implement and manage data storage solutions with Azure Data Lake Storage (ADLS) Gen 2
* Data Analytics: Develop data pipelines with Azure Synapse Analytics and utilize Azure Stream Analytics for real-time data processing and analytics
* Data Governance: Ensure data quality and integrity through data governance practices
* CI/CD Pipeline Implementation: Implement CI/CD pipelines with Azure DevOps for automated deployment