Job Description
We are seeking an experienced Data Engineer to join our team and play a crucial role in designing, developing, and maintaining scalable ETL/ELT pipelines using Azure Data Services.
This is an excellent opportunity for a skilled professional to build high-performance data processing solutions with Apache Spark (PySpark) on Azure Databricks and collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
The ideal candidate will have experience with columnar and table formats (Parquet, Delta, Hudi, Iceberg) and knowledge of Step Functions, EventBridge, or Kinesis.
Key responsibilities include:
* Designing and developing scalable ETL/ELT pipelines using Azure Data Services
* Building high-performance data processing solutions with Apache Spark (PySpark)
* Collaborating with data scientists, analysts, and business stakeholders
* Optimizing data workflows/pipelines for performance and cost efficiency
Required Skills and Qualifications
To be successful in this role, you will need:
* Experience with Azure Data Services, including Data Factory, Synapse Analytics, and Databricks
* Strong programming skills in Python and/or Scala
* Familiarity with Apache Spark and its ecosystem
* Knowledge of data warehousing and business intelligence concepts
* Excellent problem-solving and analytical skills
Benefits
As a member of our team, you will enjoy a range of benefits, including:
* A competitive salary and bonus structure
* A comprehensive health insurance package
* A generous retirement savings plan
* Ongoing training and development opportunities
What We Offer
We offer a dynamic and supportive work environment that fosters collaboration, innovation, and growth. Our team is passionate about delivering high-quality results and making a real impact on our customers' lives.