An exciting opportunity exists for a Data Engineer to join a collaborative environment and contribute to building and maintaining data infrastructure.
The successful candidate will be responsible for designing, building, and maintaining large-scale data systems, including data pipelines, data warehouses, and data lakes. They will work closely with data architects, data scientists, and other stakeholders to ensure that the entire data systems meet the needs of our business.
* Design and implement scalable automated testing solutions using Ruby/Selenium-based frameworks.
* Develop and maintain data pipelines using tools such as Apache Beam, Apache Spark, and AWS Glue.
* Develop and maintain data lakes using tools such as Apache Hadoop, Apache Spark, and Amazon S3.
Key Responsibilities:
* Design, build, and maintain data systems.
* Design and implement data warehouses using tools such as Amazon Redshift, Google BigQuery, and Snowflake.
* Work with data architects to design and implement data models and data architectures.
* Collaborate with data scientists to develop and deploy machine learning models and data products.
Qualifications:
* A minimum of 5 years of experience in data engineering or a related field.
* A minimum of 2-4 years of experience in Ruby products, including the Ruby on Rails framework.
* A minimum of 5 years of experience with programming languages such as Python, Java, and Scala.
* A minimum of 3 years of experience with data modeling and data architecture.
* A minimum of 3 years of experience with data engineering tools such as Apache Beam, Apache Spark, AWS Glue, Amazon Redshift, Google BigQuery, and Snowflake.
Nice to Have:
* Experience with machine learning and data science.
* Experience with cloud-based data platforms such as AWS, GCP, or Azure.
* Experience with containerization using Docker and Kubernetes.