Job Description
A data engineering specialist is needed to design and implement a serverless data architecture.
The ideal candidate will have extensive experience in AWS technologies and serverless architectures, with a strong focus on scalability and high-performance access to data.
* Design a scalable serverless data architecture to efficiently handle large volumes of data and support various data processing workflows.
* Develop data ingestion pipelines and integration processes using AWS services such as S3, Lambda, Glue, and DynamoDB.
* Implement data transformation and enrichment processes using AWS Lambda or similar technologies to ensure data quality and consistency.
* Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas.
* Optimize data storage and retrieval mechanisms using AWS services such as S3, Athena, Redshift, or DynamoDB.
* Monitor and troubleshoot the data infrastructure, identifying performance bottlenecks and resolving issues.