Azure Data Engineer Role
We are seeking a skilled professional to design, develop and maintain scalable data pipelines using Azure Data Services.
* Develop high-performance data processing solutions with Apache Spark (PySpark) on Azure Databricks.
* Collaborate with data scientists and business stakeholders to understand data requirements and deliver clean, reliable datasets.
* Optimize data workflows/pipelines for performance and cost efficiency in a cloud environment.
The ideal candidate will have experience with columnar and table formats such as Parquet, Delta, Hudi and Iceberg. You will also be proficient in using data quality tools like Great Expectations and Soda. Additionally, you should possess soft skills such as good communication, teamwork, problem-solving and adaptability to agile environments.