Key Data Engineer Role
We are looking for a senior data engineer to lead the development of large-scale data processing and storage systems.
* Design and implement serverless architectures using AWS services such as S3, Lambda, Glue, Athena, Redshift, and DynamoDB;
* Develop data ingestion pipelines and integration processes to ensure smooth transfer of data from various sources into the data lake;
* Implement data transformation and enrichment processes using AWS Lambda, Glue, or similar technologies to ensure data quality and consistency;
* Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models and schemas;
* Optimize data storage and retrieval mechanisms to provide high-performance access to the data;
* Monitor and troubleshoot the infrastructure to identify and resolve performance bottlenecks, data processing errors, and other issues;
* Continuously evaluate new AWS services and technologies to enhance the architecture, improve efficiency, and drive innovation;
* Mentor junior engineers and provide technical guidance to foster growth and adherence to best practices;
* Collaborate with cross-functional teams to understand business requirements, prioritize tasks, and deliver solutions within defined timelines.
This role requires extensive experience working as a data engineer, with a strong focus on AWS technologies and serverless architectures. The ideal candidate should have in-depth knowledge of AWS services and proven expertise in designing and implementing serverless architectures for large-scale data processing and storage.
A strong background in programming languages like Python, Java, or Scala, along with experience using SQL for data manipulation and querying, is also required. Familiarity with data integration and ETL tools, such as AWS Glue or Apache Spark, is highly valued.