Job Description
Modernize Enterprise-Grade Data Platforms
We seek a skilled professional to design, implement, and optimize data pipelines using Azure Databricks, Azure Data Factory, SQL Server, and Python.
Key Responsibilities:
* Develop enterprise-grade data pipelines with Azure Databricks (PySpark, Delta Lake) and Azure Data Factory.
* Design and build ETL/ELT pipelines for seamless data extraction, transformation, and loading.
* Develop complex transformations and data flows with ADF, PySpark, and T-SQL.
* Optimize database performance through SQL query tuning, index optimization, and code improvements.
* Maintain and enhance SSIS package design and deployment for legacy workloads.
* Collaborate with cross-functional teams using Git (Azure Repos) for version control and Azure DevOps pipelines (CI/CD) for deployment.
Required Skills and Qualifications
Essential Experience:
* 5+ years of expertise with Azure Databricks, Python, PySpark, and Delta Lake.
* Proven experience with Azure Data Factory for orchestrating and monitoring pipelines.
* Strong SQL Server / T-SQL experience with a focus on query optimization, indexing strategies, and coding best practices.
* Demonstrated experience in SSIS package design, deployment, and performance tuning.
* Hands-on knowledge of Unity Catalog for governance.
* Experience with Git (Azure DevOps Repos) and CI/CD practices in data engineering projects.
Nice to Have:
* Exposure to Change Data Capture (CDC), Change Data Feed (CDF), and Temporal Tables.
* Experience with Microsoft Purview, Power BI, and Azure-native integrations.
* Familiarity with Profisee Master Data Management (MDM).
* Working in Agile/Scrum environments.
Preferred Qualifications
Microsoft Certifications:
* Microsoft Certified: Azure Data Engineer Associate (DP-203)
* Microsoft Certified: Azure Solutions Architect Expert or equivalent advanced Azure certification
* Databricks Certified Data Engineer Associate or Professional
Additional Certifications:
* Microsoft SQL Server or Azure certifications demonstrating advanced database and cloud expertise