Job Title:
Big Data Processing Engineer
Transform your career in a collaborative environment where you can expand ideas through the right tools and contribute to our success.
Responsibilities:
* Design and implement scalable data processing systems using AWS services like S3, Lambda, Glue, Athena, Redshift, and DynamoDB.
* Develop data ingestion pipelines and integration processes for smooth and reliable data transfer from various sources into the DataLake.
* Implement data transformation and enrichment processes using AWS Lambda, Glue, or similar serverless technologies to ensure data quality and consistency.
* Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas in the DataLake.
* Optimize data storage and retrieval mechanisms leveraging AWS services to provide high-performance access to the data.
* Monitor and troubleshoot the DataLake infrastructure identifying and resolving performance bottlenecks, data processing errors, and other issues.
Requirements:
* 5+ years of experience working as a Big Data Processing Engineer with a strong focus on AWS technologies and serverless architectures.
* Experience working with Azure is valuable.
* Deep knowledge of AWS services like S3, Lambda, Glue, Athena, Redshift, and DynamoDB.
* Proven expertise in designing and implementing big data processing systems for large-scale data processing and storage.
* Strong programming skills in languages like Python, Java, or Scala along with experience using SQL for data manipulation and querying.
* Hands-on experience with data integration and ETL tools like AWS Glue or Apache Spark.
* English language proficiency.
Benefits:
* Professional development and constant evolution of your skills.
* Opportunities to work outside Brazil.
* A diverse and innovative environment that encourages teamwork.
* Health insurance.
* Dental Plan.
* Life insurance.
* Transportation vouchers.
* Meal/Food Voucher.
* Childcare assistance.
* Gympass.
* TCS Cares - free 0800 psychological assistance.