Job Title: Data Engineering Specialist
Data Engineering Job Description
We are seeking an experienced Data Engineering Specialist to design and implement scalable data processing systems using AWS serverless architectures.
* Design and implement efficient data ingestion pipelines, integrating data from various sources into our DataLake.
* Develop data transformation and enrichment processes using AWS Lambda, Glue, or similar technologies to ensure high-quality data.
* Collaborate with data scientists and analysts to understand their data requirements and design appropriate data models in the DataLake.
* Optimize data storage and retrieval mechanisms using AWS services such as S3, Athena, Redshift, or DynamoDB for high-performance access to data.
* Monitor and troubleshoot the DataLake infrastructure, identifying performance bottlenecks, data processing errors, and other issues.
* Stay up-to-date with new AWS services and technologies to enhance the DataLake architecture and drive innovation.
Requirements:
* 5+ years of experience working as a Data Engineer, with expertise in AWS technologies and serverless architectures.
* In-depth knowledge of AWS services like S3, Lambda, Glue, Athena, Redshift, and DynamoDB for building scalable data processing systems.
* Proven ability to design and implement scalable serverless architectures for large-scale data processing and storage.
* Strong programming skills in languages like Python, Java, or Scala, along with SQL for data manipulation and querying.
* Hands-on experience with data integration and ETL tools like AWS Glue or Apache Spark for transforming and processing data.
* Familiarity with data modeling techniques and data warehousing concepts, including star and snowflake schemas.
* Solid understanding of data security, access control, and compliance requirements in a data-driven environment.
* Experience with data visualization tools (e.g., Tableau, Power BI) and collaboration with analysts and data scientists to deliver actionable insights.
* Strong problem-solving and analytical skills, with attention to detail for data accuracy and integrity.
* Excellent communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.