Cloud Data Specialist
We're seeking a skilled Cloud Data Specialist to help drive our data architecture initiatives.
The ideal candidate will have experience with cloud-based technologies and be proficient in designing and implementing scalable data pipelines.
* Data Ingestion: Proficiency in BigQuery, Dataflow (Apache Beam), Cloud Storage, and Pub/Sub is essential for this role.
* Database Management: Experience with SQL, Oracle Database, and PostgreSQL is necessary for managing and optimizing data storage solutions.
* Data Orchestration: Knowledge of orchestration using Cloud Composer (Airflow) is required for streamlining data processing workflows.
* CICI/CD: Hands-on experience with CI/CD applied to data pipelines (Git, Terraform) is crucial for ensuring efficient deployment and management of data assets.
* Cost Optimization: Experience with cloud cost and performance optimization is vital for reducing unnecessary expenses and maximizing resource utilization.
* Certifications: GCP certifications are highly desirable for demonstrating expertise in Google Cloud Platform.
* Kubernetes and APIs: Knowledge of Kubernetes (GKE) and APIs on GCP is necessary for integrating with other cloud services and automating data processes.
* Machine Learning Pipelines: Experience with Machine Learning pipelines (Vertex AI, AI Platform) is beneficial for creating intelligent data-driven applications.
* Data Mesh and Distributed Architectures: Previous involvement with Data Mesh and distributed architectures is valuable for designing scalable and fault-tolerant data systems.
* Data Lake Layers: Understanding of Data Lake layers is essential for architecting and implementing data warehousing solutions.
* Batch and Streaming Processing: Knowledge of batch and streaming processing is necessary for handling both real-time and batch data workloads.
Responsibilities will include designing and implementing data architectures on GCP, developing and optimizing scalable ETL/ELT pipelines, ensuring data quality, integrity, and security, creating and maintaining data models aligned with business needs, collaborating with data scientists, analysts, and software engineers, automating ingestion, transformation, and data delivery processes, monitoring and optimizing cost and performance of GCP resources, and implementing best practices for DataOps and Data Governance.
This role offers a unique opportunity to work on cutting-edge projects, collaborate with talented professionals, and contribute to the development of innovative data solutions.