Data Engineering Lead
We are seeking an experienced Data Engineering Lead who can drive the development and implementation of a scalable data architecture.
* Design and develop an efficient AWS Serverless DataLake infrastructure to handle large volumes of data and support various data processing workflows;
* Evaluate, design, and implement data ingestion pipelines and integration processes, ensuring smooth and reliable data transfer from various sources into the DataLake;
* Implement data transformation and enrichment processes using AWS Lambda, Glue, or similar serverless technologies to ensure data quality and consistency;
* Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas in the DataLake;
* Optimize data storage and retrieval mechanisms, leveraging AWS services such as S3, Athena, Redshift, or DynamoDB, to provide high-performance access to the data;
* Maintain and troubleshoot the DataLake infrastructure, identifying and resolving performance bottlenecks, data processing errors, and other issues;
* Evaluate new AWS services and technologies to enhance the DataLake architecture, improve data processing efficiency, and drive innovation;
Benefits:
* Opportunity to work with cutting-edge technologies and innovative projects;
* Collaborative and dynamic work environment;
* Possibility to learn and grow within the company;