Azure Data Engineer Role
As a key member of our team, you will play a pivotal role in designing and developing scalable ETL/ELT pipelines using Azure Data Services.
With the ability to collaborate closely with data scientists, analysts, and business stakeholders, you will deliver high-quality datasets that meet the needs of our organization.
Responsibilities:
* Design and implement high-performance data processing solutions using Apache Spark (PySpark) on Azure Databricks.
* Collaborate with cross-functional teams to understand data requirements and develop innovative solutions.
* Optimize data workflows/pipelines for performance and cost efficiency in a cloud environment.
Requirements:
* Proven experience with columnar and table formats, including Parquet, Delta, Hudi, and Iceberg.
* Proficiency in using Data Quality tools, such as Great Expectations and Soda.
* Knowledge of Step Functions, EventBridge, or Kinesis.
As a valued member of our team, you will have opportunities to learn and grow, working on exciting projects that drive real impact. You will be part of a collaborative and diverse environment that encourages innovation and professional development.
What We Offer:
Our comprehensive benefits package includes:
* Health insurance.
* Dental Plan.
* Life insurance.
* Transportation vouchers.
* Meal/Food Voucher.
* Childcare assistance.
* Gympass.
* Psychological assistance and support services.