Job Title
Expand your expertise in data engineering by designing and implementing large-scale data processing systems.
Key Responsibilities:
* Design an efficient AWS Serverless DataLake architecture to handle large volumes of data and support various data processing workflows.
* Develop data ingestion pipelines and integration processes to ensure the smooth transfer of data from various sources into the DataLake.
* Implement data transformation and enrichment processes using AWS Lambda, Glue, or similar serverless technologies to ensure data quality and consistency.
* Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas in the DataLake.
* Optimize data storage and retrieval mechanisms to provide high-performance access to the data.
* Monitor and troubleshoot the DataLake infrastructure, identifying and resolving performance bottlenecks, data processing errors, and other issues.
* Continuously evaluate new AWS services and technologies to enhance the DataLake architecture, improve data processing efficiency, and drive innovation.
* Mentor junior data engineers, fostering their growth and ensuring adherence to best practices.
Required Skills and Qualifications:
Extensive experience (5+ years) working as a Data Engineer, with a strong focus on AWS technologies and serverless architectures.
Experience working as a Data Engineer, with focus on Azure is valuable.
In-depth knowledge of AWS services such as S3, Lambda, Glue, Athena, Redshift, and DynamoDB, and their capabilities for building scalable data processing systems.
Proven expertise in designing and implementing AWS serverless architectures for large-scale data processing and storage.
Strong programming skills in languages like Python, Java, or Scala, along with experience using SQL for data manipulation and querying.
Hands-on experience with data integration and ETL tools, such as AWS Glue or Apache Spark, for transforming and processing data.
Benefits:
* Professional development and constant evolution of your skills, always in line with your interests.
* Opportunities to work outside Brazil.
* A collaborative, diverse, and innovative environment that encourages teamwork.
* Health insurance.
* Dental Plan.
* Life insurance.
* Transportation vouchers.
* Meal/Food Voucher.
* Childcare assistance.
* Gympass.
* TCS Cares – free 0800 that provides psychological assistance (24 hrs/day), legal, social, and financial assistance to associates.
* Partnership with SESC.
* Reimbursement of Certifications.
* Free TCS Learning Portal – Online courses and live training.
* International experience opportunity.
* Discount Partnership with Universities and Language Schools.
* Bring Your Buddy – By referring people you become eligible to receive a bonus for each hire.
* TCS Gems – Recognition for performance.
* Xcelerate – Free Mentoring Career Platform.