Data Engineer / Senior Data Engineer
Location: Costa Rica, Brazil or Mexico
Focus: Digital & Customer Experience Applications
Role Summary
We are seeking a Data Engineer / Senior Data Engineer to design, build, and operate scalable data pipelines and curated datasets. This role is pivotal in enabling Digital and Customer Experience applications across our global enterprise. You will work hands-on with Python, PySpark, and Databricks, championing modern engineering practices through Git-based workflows, CI/CD, and Azure DevOps to ensure high-quality data delivery.
Key Responsibilities
Pipeline Development: Design and optimize ETL/ELT pipelines using Databricks and PySpark to transform and serve data for digital customer journeys.
Data Modeling: Build performant SQL transformations and curated dimensional data layers for downstream analytics and application consumption.
Engineering Excellence: Implement best practices for Databricks (notebooks vs. repos), modular code, cluster configuration, and cost-aware performance tuning.
Automation & Scripting: Utilize Python for data validation, orchestration helpers, and the creation of reusable libraries.
DevOps & CI/CD: * Manage source control via Git (GitHub/Bitbucket), enforcing strict PR reviews and branching strategies.
Build and maintain Azure DevOps pipelines to automate testing, packaging, and deployment across environments.
Quality & Reliability: Ensure data integrity through automated reconciliation checks, monitoring, and detailed operational runbooks.
Collaboration: Partner with product teams, analysts, and platform engineers to translate complex requirements into scalable data products.
Leadership (Senior Level): Lead module-level design, mentor junior engineers, and take ownership of critical pipeline delivery and enterprise code standards.
Technical Skills
Primary (Must-Have)
Advanced Data Engineering: Expert-level Python, PySpark, and SQL (advanced querying and optimization).
Databricks Ecosystem: Proficiency in Databricks jobs, workflows, clusters, and repos.
Version Control: Strong experience with Git (GitHub and/or Bitbucket) and enterprise branching workflows.
Azure DevOps: Hands-on experience with Boards, Repos, and Pipelines for build/release automation in data workloads.
Secondary (Good-to-Have)
Orchestration: Experience with Azure Data Factory (ADF) for ingestion and scheduling patterns.
Data Visualization: Understanding of Power BI datasets, measures, and refresh patterns to support analytics enablement.