Data Pipeline Architect
We are seeking a highly skilled professional to design and implement enterprise-grade data pipelines on the Azure Cloud using Azure Databricks, Azure Data Factory, SQL Server and Python.
* Data Pipeline Development:
* Data Flows & Transformations:
* Data Processing:
* Database & Query Optimization:
* SSIS & Migration Support:
* Collaboration & DevOps:
* Data Governance & Security:
* API & External Integration:
* Automation:
* Advanced SQL Expertise:
Required Qualifications:
* 5+ years of hands-on expertise with Azure Databricks, Python, PySpark and Delta Lake.
* 5+ years of proven experience with Azure Data Factory for orchestrating and monitoring pipelines.
* Strong SQL Server/T-SQL experience with a focus on query optimization, indexing strategies and coding best practices.
* Demonstrated experience in SSIS package design, deployment and performance tuning.
* Hands-on knowledge of Unity Catalog for governance.
* Experience with Github (Azure DevOps Repos) and CI/CD practices in data engineering projects.
Nice to Have:
* Exposure to Change Data Capture (CDC), Change Data Feed (CDF) and Temporal Tables.
* Experience with Microsoft Purview, Power BI and Azure-native integrations.
* Familiarity with Profisee Master Data Management (MDM).
* Working in Agile/Scrum environments.
Preferred Qualifications:
* Microsoft Certified: Azure Data Engineer Associate (DP-203)
* Microsoft Certified: Azure Solutions Architect Expert or equivalent advanced Azure certification
* Databricks Certified Data Engineer Associate or Professional