Azure Data Engineer Role Summary
This data engineering position requires expertise in designing, developing and maintaining scalable ETL/ELT pipelines using Azure Data Services. The ideal candidate will have experience with columnar and table formats, data quality tools and best practices for API security. A collaborative environment that encourages teamwork and continuous learning is offered.
Responsibilities:
* Design and develop high-performance data processing solutions with Apache Spark (PySpark) on Azure Databricks.
* Collaborate with data scientists, analysts and business stakeholders to understand data requirements and deliver clean reliable datasets.
* Optimize data workflows/pipelines for performance and cost efficiency in a cloud environment.
* Implement best practices around data security, governance and compliance.
* Develop CI/CD pipelines for data engineering workflows.
* Monitor troubleshoot and enhance existing data solutions for reliability and performance.
* Document design patterns best practices and operational procedures.
Key Benefits:
* Professional development and constant evolution of your skills.
* Opportunities to work in a diverse innovative environment.
About the Role:
* This role involves working closely with cross-functional teams to deliver high-quality data solutions.