Data Engineering Role
We are seeking an experienced data engineer to join our team and help drive the development of scalable ETL/ELT pipelines using Azure Data Services.
Key responsibilities include designing, developing, and maintaining high-performance data processing solutions with Apache Spark on Azure Databricks, collaborating with data scientists, analysts, and business stakeholders to understand data requirements, and delivering clean, reliable datasets.
To succeed in this role, you should have a strong understanding of columnar and table formats (Parquet, Delta, Hudi, Iceberg), experience with Data Quality tools (Great Expectations, Soda), and best practices for API security (Cognito, WAF, IAM Policies).
In addition to technical skills, you should possess soft skills like good communication and teamwork, proactivity in problem-solving, and ability to handle agile environments and rapid changes.