Data Engineer Job Description
Data engineers design, implement, and maintain large-scale data processing systems using various technologies such as AWS services, Python, Java, Scala, SQL, and data visualization tools like Tableau or Power BI.
The ideal candidate will have extensive experience working with AWS technologies and serverless architectures. They should be proficient in designing and implementing scalable data processing systems, leveraging AWS services like S3, Lambda, Glue, Athena, Redshift, and DynamoDB.
Key responsibilities include:
* Designing and implementing an AWS Serverless DataLake architecture to efficiently handle large volumes of data and support various data processing workflows
* Developing data ingestion pipelines and data integration processes to ensure smooth and reliable transfer of data from various sources into the DataLake
* Implementing data transformation and data enrichment processes using AWS Lambda, Glue, or similar serverless technologies to ensure data quality and consistency
* Collaborating with data scientists and analysts to understand their data requirements and design appropriate data models and schemas in the DataLake
* Optimizing data storage and retrieval mechanisms, leveraging AWS services such as S3, Athena, Redshift, or DynamoDB, to provide high-performance access to the data
Benefits
Data Engineers at our organization can expect professional development opportunities, a collaborative and innovative work environment, and benefits including health insurance, dental plan, life insurance, transportation vouchers, meal/food voucher, childcare assistance, and more.