Cloud Data Engineers are in high demand, and we're looking for a skilled professional to join our team. As a key player in building robust, cloud-native data infrastructure, you'll work closely with platform engineers, DevOps, and data scientists to create efficient and scalable data pipelines.
We're seeking someone with hands-on experience in Databricks, who can build and maintain ETL/ELT pipelines using Spark, Delta Lake, and Python. You should also have orchestration logic skills using tools like AWS Step Functions, Lambda, or Databricks Workflows.
Key Responsibilities:
* Data Pipeline Development: Design, implement, and maintain large-scale data pipelines on Databricks using Spark, Delta Lake, and Python.
* Orchestration Logic: Develop and manage orchestration logic using AWS Step Functions, Lambda, or Databricks Workflows.
* Infrastructure Provisioning: Collaborate with the team to provision infrastructure using Terraform and GitHub Actions.
* Troubleshooting: Troubleshoot Spark job performance and ensure reliable, efficient data pipelines.
Required Skills:
* 3-6 years of experience in data engineering or data platform roles.
* Solid experience with Databricks and Delta Lake, including job and cluster setup.
* Strong in PySpark, SQL, and scripting for data transformation.
* Familiarity with AWS services: S3, Lambda, Step Functions, IAM, CloudWatch.
* Exposure to CI/CD practices and infrastructure automation using Terraform.
Benefits:
* Remote work opportunities.
* Coworking space financial coverage.
* Flexible working hours.
* Performance-based financial incentives.
* Paid courses and certifications.
* Participation at international conferences.