**Data Engineer Position Description**
This is an exciting opportunity for a skilled Data Engineer to join a collaborative environment and contribute to building and maintaining data infrastructure.
* The successful candidate will be responsible for designing, building, and maintaining large-scale data systems, including data pipelines, data warehouses, and data lakes.
* He/She will work closely with data architects, data scientists, and other stakeholders to ensure that the entire data systems meet the needs of our business.
This is a fully remote opportunity with the potential to become a permanent position.
Key Responsibilities:
* Data Systems Design and Development: Design, build, and maintain large-scale data systems.
* Data Warehousing: Design and implement data warehouses using tools such as Amazon Redshift, Google BigQuery, and Snowflake.
* Automated Testing Solutions: Design and implement scalable automated testing solutions using Ruby/Selenium-based frameworks.
* Data Pipelines: Develop and maintain data pipelines using tools such as Apache Beam, Apache Spark, and AWS Glue.
* Data Lakes: Develop and maintain data lakes using tools such as Apache Hadoop, Apache Spark, and Amazon S3.
* Data Modeling and Architecture: Work with data architects to design and implement data models and data architectures.
* Machine Learning Models: Collaborate with data scientists to develop and deploy machine learning models and data products.
* Data Quality and Integrity: Ensure data quality and integrity by developing and implementing data validation and data cleansing processes.
* Collaboration and Communication: Collaborate with other teams to ensure that data systems meet the business's needs.
* Technology Updates: Stay up-to-date with new technologies and trends in data engineering and make recommendations for adoption.
Qualifications and Skills:
* Experience: 5+ years of experience in data engineering or a related field.
* Programming Languages: Proficiency in programming languages such as Python, Java, and Scala.
* Data Modeling and Architecture: 3+ years of experience with data modeling and data architecture.
* Data Engineering Tools: Strong experience with data engineering tools such as Apache Beam, Apache Spark, AWS Glue, Amazon Redshift, Google BigQuery, and Snowflake.
* Data Warehousing and Lakes: Strong experience with data warehousing and data lakes.
* Data Validation and Cleansing: Strong experience with data validation and data cleansing.
* Collaboration and Communication: Strong collaboration and communication skills.
* Education: Bachelor's degree in Computer Science, Engineering, or a related field.
Nice to Have Qualifications:
* Machine Learning and Data Science: Experience with machine learning and data science.
* Cloud-Based Data Platforms: Experience with cloud-based data platforms such as AWS, GCP, or Azure.
* Containerization: Experience with containerization using Docker and Kubernetes.
* Agile Development Methodologies: Experience with agile development methodologies such as Scrum or Kanban.
* Data Governance and Security: Experience with data governance and data security.