Title: Software Engineer
Location: Remote in Brazil
Type: Full-time (CLT)
Job Description:
* Pipeline Development & Migration: Design, build, and maintain robust ETL/ELT pipelines across hybrid environments (AWS, GCP, Snowflake). A primary focus will be the migration and expansion of Adobe-based pipelines.
* Data Integration: Lead the integration of Orion datasets into existing architectures and create new pipelines for Orion Event collection.
* Performance Optimization: Proactively optimize pipeline runtimes, compute usage, and storage tiering. You will be responsible for managing query costs, with a specific emphasis on BigQuery optimization.
* Data Integrity: Establish and automate rigorous data quality checks, validation logic, and reconciliation processes to ensure \"single source of truth\" reliability.
* Architecture & Documentation: Apply advanced data warehousing concepts (Star Schemas, Dimensional Modeling) to structure data for high-performance OLAP needs. Maintain comprehensive technical documentation for all developed systems.
Required Qualifications
* Education: Bachelor’s Degree in Computer Science, Information Technology, or a related technical discipline.
* Overall Experience: 5-8 years of professional experience in the technology sector.
* Engineering Expertise: 5-8 years of direct, applicable experience in Data Engineering or Backend Engineering.
Technical Skills & Competencies
The GCP Stack & Orchestration:
* Google Cloud Platform (GCP): Expert-level proficiency in BigQuery.
* Orchestration: Hands-on experience with Apache Airflow and Google Cloud Composer.
* DevOps: Strong mastery of GitHub and CI/CD workflows for automated data deployments.
Programming & Languages:
* Python: Advanced proficiency for data manipulation and automation.
* SQL: Expert level for complex analytical querying and transformation.
Data Modeling & Design:
* Warehousing Concepts: Deep understanding of Star Schemas and Dimensional Modeling.
* Systems Knowledge: Clear understanding of the trade-offs between OLTP vs. OLAP environments.
* Hybrid Cloud: Experience moving and transforming data across AWS, GCP, and Snowflake.