We are seeking a skilled Data Engineer to join our team and contribute to the design, development, and maintenance of large-scale data systems. The ideal candidate will have experience in designing and implementing data pipelines, data warehouses, and data lakes using tools such as Apache Beam, Apache Spark, and AWS Glue.
Key Responsibilities:
* Design and build scalable data systems
* Develop and maintain data pipelines using Apache Beam and Apache Spark
* Implement data warehouses using Amazon Redshift, Google BigQuery, and Snowflake
* Develop and maintain data lakes using Apache Hadoop and Amazon S3
* Collaborate with data architects to design and implement data models and architectures
* Work with data scientists to develop and deploy machine learning models and data products
* Evaluate and ensure data quality and integrity by developing and implementing data validation and data cleansing processes
Requirements:
* 5+ years of experience in data engineering or a related field
* 2-4 years of experience in Ruby programming and the Ruby on Rails framework
* 5+ years of experience with programming languages such as Python, Java, and Scala
* 3+ years of experience with data modeling and architecture
* Strong experience with data engineering tools such as Apache Beam, Apache Spark, AWS Glue, Amazon Redshift, Google BigQuery, and Snowflake
* Strong collaboration and communication skills
* Bachelor's degree in Computer Science, Engineering, or a related field
Nice to Have:
* Experience with cloud-based data platforms such as AWS, GCP, or Azure
* Experience with containerization using Docker and Kubernetes
* Experience with agile development methodologies such as Scrum or Kanban
* Experience with data governance and security
This is an exciting opportunity for a skilled Data Engineer to work remotely and contribute to the growth of our company. We thank all candidates who apply. Only selected candidates for interviews will be contacted.