Overview
Our client is a U.S.-based company that provides technical expertise, testing, and certification services to the global food and agricultural industry. Their mission is to ensure food safety, quality, and sustainability across international supply chains.
This role is critical to building, maintaining, and modernizing data pipelines that process large-scale regulatory data from around the world and transform it into usable datasets for downstream applications and APIs. The engineer will work hands-on with Python, SQL, and related tools to untangle legacy code pipelines, migrate processes to maintainable platforms such as Airflow, and ensure that data is accurate, reliable, and ready for client-facing products. The role requires strong technical ability and a consulting mindset—able to learn undocumented systems, troubleshoot gaps, and design scalable solutions for evolving data environments.
Responsibilities
* Build and maintain data pipelines and ETL/ELT processes to support downstream applications and client-facing products.
* Modernize legacy pipelines by migrating to platforms such as Airflow and implement robust data quality checks.
* Collaborate with data engineers, analysts, and stakeholders to understand data needs and deliver reliable datasets.
* Monitor data pipelines for performance and reliability; respond to incidents and resolve data issues with minimal downtime.
* Document operational procedures, job schedules, and system changes; provide regular updates on health and performance.
* Contribute to platform evolution through reusable frameworks, automation, and documentation.
Required Qualifications
* Minimum 7 years’ experience using Python for analyzing, extracting, creating, and transforming large datasets.
* Proficiency in Python 3+ and libraries for data engineering (Pandas, NumPy, Jupyter Notebooks).
* Deep experience with SQL and relational data (Oracle, Postgres, MS SQL Server).
* Solid understanding of database design, data modeling, and data warehousing concepts.
* Excellent troubleshooting skills and problem-solving instincts.
* Curious, self-motivated, and self-directed; comfortable in an Agile software development environment.
* College degree or equivalent experience in computer science, software development, engineering, information systems, math, or related field.
Preferred Qualifications
* NoSQL database design and development (MongoDB, AWS DynamoDB, or Azure Cosmos DB).
* Familiarity with cloud platforms (AWS, Azure, GCP) and data storage/processing services.
* Experience with Infrastructure-as-Code tooling (e.g., Terraform).
* Proficiency with Azure DevOps for source code and pipeline management.
What You Will Do (Role Details)
* Snowflake, DBT, SQL; Agile methodologies.
* Operational monitoring: monitor data jobs and pipelines, respond to alerts, and resolve issues.
* Pipeline maintenance: update DBT models and SQL scripts to support evolving needs.
* Warehouse operations: manage Snowflake operations, including access and performance.
* Incident response: root-cause analysis and preventive measures for data job failures.
* Collaboration: work with data engineers, analysts, and business stakeholders.
* Process optimization: automate manual tasks and improve pipeline efficiency.
* Documentation & reporting: maintain procedural docs and incident logs; provide status updates.
What We Offer / About the Company
We are seeking a dynamic Data Engineer to join a global IT services environment. This role offers opportunities for professional development, collaboration across teams, and international experience where applicable.
English Requirement
English CV is a must.
#J-18808-Ljbffr