Job Title
A Data Engineer is required to design and implement an AWS Serverless DataLake architecture to efficiently handle large volumes of data and support various data processing workflows.
About the Role
* The ideal candidate will have extensive experience (5+ years) working as a Data Engineer, with a strong focus on AWS technologies and serverless architectures.
Key Responsibilities
1. Develop data ingestion pipelines and data integration processes, ensuring the smooth and reliable transfer of data from various sources into the DataLake.
2. Implement data transformation and data enrichment processes using AWS Lambda, Glue, or similar serverless technologies to ensure data quality and consistency.
3. Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas in the DataLake.
4. Optimize data storage and retrieval mechanisms, leveraging AWS services such as S3, Athena, Redshift, or DynamoDB, to provide high-performance access to the data.
5. Maintain and troubleshoot the DataLake infrastructure, identifying and resolving performance bottlenecks, data processing errors, and other issues.
Requirements
* Strong programming skills in languages like Python, Java, or Scala, along with experience using SQL for data manipulation and querying.
* Hands-on experience with data integration and ETL tools, such as AWS Glue or Apache Spark, for transforming and processing data.
* Familiarity with data modeling techniques and data warehousing concepts, including star and snowflake schemas.
What We Offer
* Professional development and constant evolution of your skills, always in line with your interests.
* A collaborative, diverse and innovative environment that encourages teamwork.
* Opportunities to work outside Brazil.