Data Engineer Position
As a Data Engineer, you will create and implement an AWS Serverless DataLake architecture to efficiently handle large volumes of data and support various data processing workflows.
* Develop data ingestion pipelines and integration processes, ensuring smooth and reliable transfer of data from diverse sources into the DataLake.
* Implement transformation and enrichment processes using AWS Lambda, Glue, or similar technologies to ensure data quality and consistency.
* Collaborate with data scientists and analysts to understand their requirements and design appropriate data models and schemas in the DataLake.
* Optimize storage and retrieval mechanisms, leveraging AWS services such as S3, Athena, Redshift, or DynamoDB, for high-performance access to the data.
Requirements
* Extensive experience (5+ years) working as a Data Engineer, with strong focus on AWS technologies and serverless architectures.
* In-depth knowledge of AWS services like S3, Lambda, Glue, Athena, Redshift, and DynamoDB, and their capabilities for building scalable systems.
* Strong programming skills in languages like Python, Java, or Scala, along with SQL experience for data manipulation and querying.
* Familiarity with data modeling techniques and warehousing concepts, including star and snowflake schemas.