Big Data Analytics Engineer
We are seeking an experienced Big Data Analytics Engineer to drive the development of scalable data pipelines using Spark, PySpark, SQL, and Delta Lake.
This role focuses on ingesting data from diverse sources, transforming it for analytics, and publishing high-quality datasets and visualizations.
* Build and optimize ETL/ELT pipelines in Databricks using PySpark, Spark SQL, and Delta Lake.
* Develop reusable pipeline frameworks, data validation logic, and performance-tuned transformations.
* Design curated datasets and deliver insights through Power BI dashboards.
Our ideal candidate will have hands-on experience with big data technologies and a strong understanding of data engineering principles. Strong problem-solving skills and excellent communication skills are essential for this role.
Responsibilities:
1. Develop and maintain data pipelines that ensure data accuracy, reliability, and quality.
2. Collaborate with cross-functional teams to design and implement data solutions.
3. Troubleshoot pipeline performance issues and implement best practices for lakehouse development, orchestration, and version control.