Job Description
As a seasoned data professional, you will be responsible for designing and implementing an AWS Serverless DataLake architecture to efficiently handle large volumes of data and support various data processing workflows.
This is an excellent opportunity for someone with extensive experience working as a Data Engineer, with a strong focus on AWS technologies and serverless architectures. You will work closely with cross-functional teams to understand business requirements, prioritize tasks, and deliver high-quality solutions within defined timelines.
Key Responsibilities:
* Design and implement an AWS Serverless DataLake architecture to efficiently handle large volumes of data and support various data processing workflows;
* Develop data ingestion pipelines and data integration processes, ensuring the smooth and reliable transfer of data from various sources into the DataLake;
* Implement data transformation and data enrichment processes using AWS Lambda, Glue, or similar serverless technologies to ensure data quality and consistency;
* Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas in the DataLake;
* Optimize data storage and retrieval mechanisms, leveraging AWS services such as S3, Athena, Redshift, or DynamoDB, to provide high-performance access to the data;
* Monitor and troubleshoot the DataLake infrastructure, identifying and resolving performance bottlenecks, data processing errors, and other issues;
* Continuously evaluate new AWS services and technologies to enhance the DataLake architecture, improve data processing efficiency, and drive innovation;
* Mentor and provide technical guidance to junior data engineers, fostering their growth and ensuring adherence to best practices;
,