Job Title: Azure DevOps/Databricks Engineer
Job Summary:
Hiring for a skilled Azure DevOps/Databricks Engineer to lead data engineering initiatives in LATAM/Brazil.
Technical Skills and Qualifications:
* Azure DevOps pipeline development, Git repository management, artifact handling
* Terraform, Infrastructure-as-Code governance patterns
* Azure Data Lake Storage Gen2, hierarchical namespace administration, Access Control List (ACL) models
* Azure Data Factory, Databricks, Synapse Analytics, Spark workflows
* Azure Functions, Key Vault, network architecture (VNets, private endpoints, firewalls)
* Monitoring stacks: Log Analytics, Application Insights, Azure Monitor
* Scripting: PowerShell, Python, Bash
* Security controls: Role-Based Access Control (RBAC), managed identities, secrets management, encryption
* Continuous Integration/Continuous Deployment (CI/CD) patterns, release strategy design, automated testing frameworks
* Jira, Confluence, ServiceNow
Key Responsibilities:
* Design and implement CI/CD pipelines for data lake components: Data Factory, Databricks, Functions, Synapse, Spark workloads, storage configurations.
* Implement Terraform-based infrastructure provisioning for storage accounts, networking, compute resources, identity, and security layers.
* Enforce branching discipline, artifact integrity, automated testing, and controlled release gates.
* Automate environment setup, ACL management, key rotation, lifecycle policies, and cluster configuration.
* Integrate DevOps processes with enterprise security: RBAC, managed identities, Key Vault, private networking, encryption controls.
* Built observability: logging, metrics, alerting, dashboards for pipelines and platform components.
* Maintain backup, restoration, disaster-recovery patterns and test them for reliability.
* Eliminate configuration drift through standardized templates and environment baselines.
* Maintain and optimize agents, service connections, and deployment runtimes.
* Perform incident response and root-cause analysis, document systemic fixes.
* Deliver reusable automation modules for data engineering teams.
* Optimize workload performance and cost within the data lake environment.
* Ensure compliance with governance, audit requirements, and data protection mandates.
* Drive continuous reduction of manual operational work through automation.
Benefits:
This role offers an opportunity to work on cutting-edge projects, collaborate with a talented team, and drive innovation in data engineering.
What We Offer:
A competitive salary package, opportunities for growth and professional development, and a collaborative work environment.