Job Summary
We are seeking a skilled Azure Data Engineer to join our team. As a key member, you will design, develop, and maintain data pipelines using Azure Data Factory (ADF).
You will also implement and manage data storage solutions with Azure Data Lake Storage (ADLS) Gen 2, develop data pipelines with Azure Synapse Analytics, and utilize Azure Stream Analytics for real-time data processing and analytics.
In addition, you will ensure data quality and integrity through data governance practices, implement CI/CD pipelines with Azure DevOps for automated deployment, and collaborate with data analysts and stakeholders to meet business data needs.
Responsibilities include:
* Data Pipeline Development: Design, develop, and maintain data pipelines using Azure Data Factory (ADF)
* Data Storage Management: Implement and manage data storage solutions with Azure Data Lake Storage (ADLS) Gen 2
* Data Analytics: Develop data pipelines with Azure Synapse Analytics and utilize Azure Stream Analytics for real-time data processing and analytics
* Data Governance: Ensure data quality and integrity through data governance practices
* CI/CD Pipeline Implementation: Implement CI/CD pipelines with Azure DevOps for automated deployment
* Collaboration: Collaborate with data analysts and stakeholders to meet business data needs
The ideal candidate should have experience with Azure Functions, Azure Logic Apps, Azure Stream Analytics, and Azure API Management (APIM), as well as understanding of Microsoft Purview for data governance and compliance.
Additionally, proficiency in PySpark and Azure Data Bricks is required, along with strong experience with Azure Data Factory, Azure Synapse Analytics, and ADLS Gen 2.
Problem-solving skills, attention to detail, and excellent communication skills are essential for this role.
Preferred Skills
* Azure Logic Apps
* Azure Stream Analytics
* Azure API Management
* Microsoft Purview