We are seeking an experienced Senior Data Engineer to join our team. In this role, you will architect and implement cloud-native data pipelines, optimize platform performance, and ensure seamless scalability across Azure and Kubernetes environments. This is a high-impact opportunity for a senior professional to shape the next generation of our data infrastructure.
Key Responsibilities
Architect, build, and optimize scalable data pipelines using Azure Databricks, Azure Data Factory, and Apache Airflow
Lead the end-to-end data migration, ensuring zero-downtime cutover and data integrity
Implement Infrastructure-as-Code with Terraform to provision and manage Azure resources and Kubernetes clusters
Establish monitoring, alerting, and observability standards using Datadog for proactive issue detection and resolution
Collaborate with data scientists, analysts, and platform engineers to translate business requirements into robust technical solutions
Need to Haves
Hands-on experience with Azure Databricks and Azure Data Factory in production environments
Proven expertise orchestrating workflows with Apache Airflow and containerizing workloads on Kubernetes
Strong proficiency in Terraform for cloud infrastructure automation
Advanced monitoring and troubleshooting skills with Datadog or similar observability platforms
Strong English skills – able to lead technical discussions and author detailed design documents
Demonstrated leadership in delivering complex data platform migrations or green-field initiatives