Job Overview
As an Azure Data Engineer, you will be responsible for designing, developing, and maintaining scalable ETL/ELT pipelines using Azure Data Services.
You will build high-performance data processing solutions with Apache Spark (PySpark) on Azure Databricks, collaborate with data scientists, analysts, and business stakeholders to understand data requirements, and deliver clean, reliable datasets.
You will optimize data workflows/pipelines for performance and cost efficiency in a cloud environment, implement best practices around data security, governance, and compliance, develop CI/CD pipelines for data engineering workflows, monitor, troubleshoot, and enhance existing data solutions for reliability and performance, and document design patterns, best practices, and operational procedures.
Key Responsibilities
1. Design and Develop Scalable Data Pipelines: Utilize Azure Data Services to design, develop, and maintain efficient ETL/ELT pipelines that process large datasets with optimal performance.
2. Collaborate with Cross-Functional Teams: Work closely with data scientists, analysts, and business stakeholders to understand data requirements, identify opportunities for improvement, and ensure seamless data flow across teams.
3. Optimize Data Workflows: Continuously monitor, analyze, and optimize data workflows/pipelines to ensure maximum performance, minimize costs, and maintain data integrity.
4. Implement Best Practices: Adhere to industry-standard best practices for data security, governance, compliance, and documentation to ensure data quality and reliability.
Benefits
TCS offers a collaborative, diverse, and innovative environment that encourages teamwork, professional development, and constant evolution of your skills. You will have opportunities to work outside Brazil, enjoy a competitive salary, and access to various benefits, including health insurance, dental plan, life insurance, transportation vouchers, meal/food voucher, childcare assistance, Gympass, and more.
What We Offer
* A collaborative and dynamic work environment that fosters growth and innovation.
* A competitive salary and comprehensive benefits package.
* Opportunities to work on cutting-edge projects and technologies.
* Access to ongoing training and professional development programs.