Title: Software Engineer Location: Remote in Brazil
Type: Full-time (CLT)
Job Description:
Pipeline Development & Migration: Design, build, and maintain robust ETL/ELT pipelines across hybrid environments (AWS, GCP, Snowflake). A primary focus will be the migration and expansion of Adobe-based pipelines.
Data Integration: Lead the integration of Orion datasets into existing architectures and create new pipelines for Orion Event collection.
Performance Optimization: Proactively optimize pipeline runtimes, compute usage, and storage tiering. You will be responsible for managing query costs, with a specific emphasis on BigQuery optimization.
Data Integrity: Establish and automate rigorous data quality checks, validation logic, and reconciliation processes to ensure \"single source of truth\" reliability.
Architecture & Documentation: Apply advanced data warehousing concepts (Star Schemas, Dimensional Modeling) to structure data for high-performance OLAP needs. Maintain comprehensive technical documentation for all developed systems.
Required Qualifications
Education: Bachelor’s Degree in Computer Science, Information Technology, or a related technical discipline.
Overall Experience: 5-8 years of professional experience in the technology sector.
Engineering Expertise: 5-8 years of direct, applicable experience in Data Engineering or Backend Engineering.
Technical Skills & Competencies
The GCP Stack & Orchestration:
Google Cloud Platform (GCP): Expert-level proficiency in BigQuery.
Orchestration: Hands-on experience with Apache Airflow and Google Cloud Composer.
DevOps: Strong mastery of GitHub and CI/CD workflows for automated data deployments.
Programming & Languages:
Python: Advanced proficiency for data manipulation and automation.
SQL: Expert level for complex analytical querying and transformation.
Data Modeling & Design:
Warehousing Concepts: Deep understanding of Star Schemas and Dimensional Modeling.
Systems Knowledge: Clear understanding of the trade-offs between OLTP vs. OLAP environments.
Hybrid Cloud: Experience moving and transforming data across AWS, GCP, and Snowflake.