We are seeking an experienced Data Engineer to join our team.
Job Summary
The successful candidate will be responsible for designing, building, and maintaining data systems, including data pipelines, data warehouses, and data lakes. They will work closely with data architects, data scientists, and other stakeholders to ensure that the entire data systems meet the needs of our business.
Main Responsibilities
* Design and build large-scale data systems using scalable technologies such as Apache Beam, Apache Spark, and AWS Glue.
* Implement data warehouses using Amazon Redshift, Google BigQuery, and Snowflake.
* Develop automated testing solutions using Ruby/Selenium-based frameworks.
* Maintain data pipelines, ensuring high-quality data flow and efficient processing.
* Collaborate with data architects to design and implement data models and data architectures.
* Work with data scientists to develop and deploy machine learning models and data products.
Requirements
* 5+ years of experience in data engineering or a related field.
* Experience with programming languages such as Python, Java, and Scala.
* Strong knowledge of data modeling and data architecture principles.
* Proficiency in working with data engineering tools such as Apache Beam, Apache Spark, AWS Glue, Amazon Redshift, Google BigQuery, and Snowflake.
* Excellent collaboration and communication skills.
* Bachelor's degree in Computer Science, Engineering, or a related field.
Nice to Have
* Experience with machine learning and data science concepts.
* Familiarity with cloud-based data platforms such as AWS, GCP, or Azure.
* Knowledge of containerization using Docker and Kubernetes.
* Understanding of agile development methodologies such as Scrum or Kanban.