Job Description
We seek a Data Engineering professional to design, develop and maintain scalable data pipelines using Azure Data Services. As an expert in cloud-based data engineering, you will create high-performance solutions with Apache Spark (PySpark) on Azure Databricks, collaborating with stakeholders to deliver reliable datasets.
You will optimize data workflows for performance and cost efficiency, implement best practices around data security, governance and compliance, and develop CI/CD pipelines for data engineering workflows. Additionally, you will monitor, troubleshoot and enhance existing data solutions, documenting design patterns and operational procedures along the way.
Responsibilities:
* Design, develop and maintain scalable ETL/ELT pipelines using Azure Data Services.
* Create high-performance data processing solutions with Apache Spark (PySpark) on Azure Databricks.
* Collaborate with data scientists, analysts and stakeholders to deliver clean, reliable datasets.
* Optimize data workflows for performance and cost efficiency.
* Implement best practices around data security, governance and compliance.
* Develop CI/CD pipelines for data engineering workflows.
* Monitor, troubleshoot and enhance existing data solutions.
* Document design patterns and operational procedures.
What We Offer:
* A collaborative environment that encourages teamwork and innovation.
* Opportunities for professional development and growth.
* A dynamic work environment that fosters creativity and excellence.