Data Engineer Role
This is an exciting opportunity for a skilled Data Engineer to join a collaborative environment and contribute to building and maintaining the data infrastructure.
The successful candidate will be responsible for designing, building, and maintaining large-scale data systems, including data pipelines, data warehouses, and data lakes.
Key responsibilities include:
* Designing, building, and maintaining large-scale data systems
* Implementing data warehouses using tools such as Amazon Redshift, Google BigQuery, and Snowflake
* Developing scalable automated testing solutions using Ruby/Selenium-based frameworks
* Creating and maintaining data pipelines using tools such as Apache Beam, Apache Spark, and AWS Glue
* Building and maintaining data lakes using tools such as Apache Hadoop, Apache Spark, and Amazon S3
* Collaborating with data architects to design and implement data models and architectures
* Working with data scientists to develop and deploy machine learning models and data products
* Ensuring data quality and integrity by developing and implementing data validation and cleansing processes
* Collaborating with other teams to ensure that data systems meet business needs
Requirements for this role include:
* 5+ years of experience in data engineering or a related field
* 2-4 years of experience in Ruby products, including Ruby on Rails framework
* 5+ years of experience with programming languages such as Python, Java, and Scala
* 3+ years of experience with data modeling and architecture
* 3+ years of experience with data engineering tools such as Apache Beam, Apache Spark, AWS Glue, Amazon Redshift, Google BigQuery, and Snowflake
* Strong experience with data warehousing and data lakes
* Strong experience with data validation and cleansing
* Strong collaboration and communication skills
* Bachelor's degree in Computer Science, Engineering, or a related field
Prior experience with machine learning and data science, cloud-based data platforms, containerization, agile development methodologies, and data governance and security are considered desirable skills.