Data Engineering Expert Sought
We are looking for a highly skilled Data Engineer to join our team and contribute to the design, development, and maintenance of large-scale data systems.
The successful candidate will be responsible for designing and implementing scalable automated testing solutions using Ruby/Selenium-based frameworks, developing and maintaining data pipelines using tools such as Apache Beam, Apache Spark, and AWS Glue, and ensuring data quality and integrity by developing and implementing data validation and data cleansing processes.
Key Responsibilities:
* Design and implement large-scale data systems
* Develop and maintain data pipelines using Apache Beam, Apache Spark, and AWS Glue
* Implement scalable automated testing solutions using Ruby/Selenium-based frameworks
* Ensure data quality and integrity through data validation and data cleansing processes
* Collaborate with data architects to design and implement data models and data architectures
* Work with data scientists to develop and deploy machine learning models and data products
Required Skills and Qualifications:
* 5+ years of experience in data engineering or a related field
* 2-4 years of experience in Ruby products, including Ruby on Rails framework
* 5+ years of experience with programming languages such as Python, Java, and Scala
* 3+ years of experience with data modeling and data architecture
* 3+ years of experience with data engineering tools such as Apache Beam, Apache Spark, AWS Glue, Amazon Redshift, Google BigQuery, and Snowflake
* Strong experience with data warehousing and data lakes
* Strong experience with data validation and data cleansing
About This Opportunity
This is a fully remote opportunity with the potential to become a permanent position. We offer a collaborative environment and opportunities for growth and development.
Nice to Have:
* Experience with machine learning and data science
* Experience with cloud-based data platforms such as AWS, GCP, or Azure
* Experience with containerization using Docker and Kubernetes
* Experience with agile development methodologies such as Scrum or Kanban
* Experience with data governance and data security
We Thank All Candidates in Advance
Only selected candidates for interviews will be contacted. For other exciting opportunities, please visit us at VTRAC is an equal-opportunity employer.