Senior Data Engineer
Design and implement data architectures on Google Cloud Platform (GCP) using services such as BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Composer, and more.
Key responsibilities include developing and optimizing scalable, high-performance extract, transform, load (ETL)/extract, load, transform (ELT) pipelines, ensuring data quality, integrity, and security throughout the data lifecycle, creating and maintaining data models aligned with business requirements, collaborating with data scientists, analysts, and software engineers to support advanced analytics and machine learning use cases, automating ingestion, transformation, and data delivery processes, monitoring and optimizing cost and performance of GCP resources, and implementing best practices for data operations (DataOps) and data governance.
The ideal candidate will have proficiency in BigQuery, Dataflow, AlloyDB, API Gateway, with a focus on GCP. Experience with structured query language (SQL), Oracle Database, and PostgreSQL is also required. Additionally, knowledge of orchestration using Cloud Composer and hands-on experience with continuous integration/continuous deployment (CI/CD) applied to data pipelines are necessary. Previous involvement with data mesh and distributed architectures, understanding of data lake layers, knowledge of batch and streaming processing, and experience with data modeling (relational, dimensional, and NoSQL) are highly desirable.