Key Roles and Responsibilities
As a Data Engineer, you will be responsible for designing and implementing an AWS Serverless DataLake architecture to efficiently handle large volumes of data and support various data processing workflows. You will need to develop data ingestion pipelines and data integration processes to ensure the smooth and reliable transfer of data from various sources into the DataLake.
* Design and implement efficient data transformation and data enrichment processes using AWS Lambda, Glue, or similar serverless technologies to ensure data quality and consistency.
* Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas in the DataLake.
Requirements
* Extensive experience (5+ years) working as a Data Engineer with a strong focus on AWS technologies and serverless architectures.
* Experience working with Azure is valuable.
* In-depth knowledge of AWS services such as S3, Lambda, Glue, Athena, Redshift, and DynamoDB and their capabilities for building scalable data processing systems.