Job Title
We are seeking a skilled Data Engineer to join our team.
* Design, develop, and maintain APIs using API Gateway.
* Implement integrations with Lambda or Fargate for data ingestion.
* Develop processing pipelines in PySpark (AWS Glue or EMR).
* Manage data in S3 and Glue Data Catalog.
* Ensure data quality and integrity.
* Optimize pipeline costs and performance.
* Create and maintain technical documentation and development standards.
About This Role
This is an exciting opportunity to design, develop, and maintain complex data systems using AWS services. The ideal candidate will have hands-on experience with AWS, PySpark, and data integration technologies.
Key Responsibilities
The successful candidate will be responsible for:
1. Designing and developing APIs using API Gateway.
2. Implementing integrations with Lambda or Fargate for data ingestion.
3. Developing processing pipelines in PySpark (AWS Glue or EMR).
4. Managing data in S3 and Glue Data Catalog.
5. Ensuring data quality and integrity.
6. Optimizing pipeline costs and performance.
7. Creating and maintaining technical documentation and development standards.
Requirements
To be successful in this role, you will need:
* A proven track record of working with AWS services.
* Strong knowledge of PySpark and data integration technologies.
* Experience with Glue Data Catalog and querying via Athena.
* Knowledge of infrastructure as code (Terraform or CloudFormation).
* Good practices in version control (Git) and CI/CD.
* Experience with columnar and table formats (Parquet, Delta, Hudi, Iceberg).
* Use of Data Quality tools (Great Expectations, Soda).
* Knowledge of Step Functions, EventBridge, or Kinesis.
* Best practices for API security (Cognito, WAF, IAM Policies).
What We Offer
As a valued member of our team, you can expect:
* A collaborative, diverse, and innovative environment that encourages teamwork.
* TCS Benefits – Brazil:
Health insurance, Dental Plan, Life insurance, Transportation vouchers, Meal/Food Voucher, Childcare assistance, Gympass, TCS Cares – free 0800 that provides psychological assistance (24 hrs/day), legal, social, and financial assistance to associates, Partnership with SESC, Reimbursement of Certifications, Free TCS Learning Portal – Online courses and live training, International experience opportunity, Discount Partnership with Universities and Language Schools, Bring Your Buddy – By referring people you become eligible to receive a bonus for each hire, TCS Gems – Recognition for performance, Xcelerate – Free Mentoring Career Platform