Key Data Engineer Role
We are seeking a seasoned professional to spearhead the design and implementation of an AWS Serverless DataLake architecture.
* We require someone who can develop data ingestion pipelines and integration processes, ensuring seamless transfer of data from various sources into the DataLake;
* To implement data transformation and enrichment processes using AWS Lambda, Glue, or similar serverless technologies to guarantee data quality and consistency;
* The successful candidate will collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas in the DataLake;
* Additionally, they will be responsible for optimizing data storage and retrieval mechanisms, leveraging AWS services such as S3, Athena, Redshift, or DynamoDB, to provide high-performance access to the data;
* Monitors and troubleshoots the DataLake infrastructure, identifying and resolving performance bottlenecks, data processing errors, and other issues;
* Continuously evaluates new AWS services and technologies to enhance the DataLake architecture, improve data processing efficiency, and drive innovation;
* Mentors junior data engineers, providing technical guidance and fostering growth while ensuring adherence to best practices;
* Collaborates with cross-functional teams to understand business requirements, prioritize tasks, and deliver high-quality solutions within defined timelines