Job Description:
We are seeking an experienced Data Engineer to join our team. The ideal candidate will have a strong background in designing, developing, and maintaining data pipelines using Azure Data Factory (ADF) and Azure Synapse Analytics.
The successful candidate will be responsible for implementing and managing data storage solutions using Azure Data Lake Storage (ADLS) Gen 2, as well as utilizing Azure Stream Analytics for real-time data processing and analytics.
To succeed in this role, you will need to ensure data quality and integrity through effective data governance practices and implement CI/CD pipelines using Azure DevOps for automated build and deployment.
You will also collaborate with data analysts and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Responsibilities:
* Data Pipeline Development: Design, develop, and maintain data pipelines using Azure Data Factory (ADF) and Azure Synapse Analytics.
* Data Storage Management: Implement and manage data storage solutions using Azure Data Lake Storage (ADLS) Gen 2.
* Real-Time Data Processing: Utilize Azure Stream Analytics for real-time data processing and analytics.
* Data Governance: Ensure data quality and integrity through effective data governance practices.
* CI/CD Implementation: Implement CI/CD pipelines using Azure DevOps for automated build and deployment.
* Collaboration: Collaborate with data analysts and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Required Skills and Qualifications:
The successful candidate will possess the following skills and qualifications:
* Azure Experience: Strong experience with Azure Functions for serverless computing, Azure Logic Apps for workflow automation, and Azure Stream Analytics for real-time data processing.
* Cloud Technologies: Familiarity with cloud technologies, including Microsoft Purview for data governance and compliance.
* Programming Languages: Proficiency in PySpark for data processing and transformation.
* Strong Problem-Solving Skills: Strong problem-solving skills and attention to detail.
* Excellent Communication Skills: Excellent communication and collaboration skills.
Benefits:
This role offers a range of benefits, including opportunities for professional growth and development, a dynamic and supportive work environment, and competitive compensation and benefits packages.
Others:
This is an exciting opportunity for a skilled Data Engineer to join our team and contribute to the success of our organization. If you are a motivated and enthusiastic individual with a passion for data engineering, we encourage you to apply.