We are seeking a seasoned data engineer with expertise in designing, building, and optimizing large-scale data pipelines and cloud-based data platforms.
Key Responsibilities
1. Design, develop, and maintain scalable ETL/ELT processes and data pipelines for structured and unstructured data;
2. Develop and manage data lakes, warehouses, and analytics platforms in the cloud;
3. Ensure data quality, governance, compliance, and security across environments;
4. Translate business and technical requirements into robust data solutions;
5. Implement best practices for data storage, partitioning, and performance tuning;
6. Work with Azure cloud-native services (Data Factory, Synapse, Databricks) for ingestion, transformation, and orchestration;
7. Build real-time and batch data processing solutions using big data frameworks;
8. Partner with analysts and data scientists to support AI/ML and reporting initiatives;
9. Monitor and troubleshoot data pipelines, workflows, and infrastructure issues;
10. Contribute to automation, CI/CD pipelines, and Infrastructure-as-Code practices for data platforms.
Essential Skills:
* Experience in data engineering, architecture, or ETL development;
* Strong programming skills in Python, Java, or Scala for data processing;
* Advanced SQL expertise and proficiency in relational databases (PostgreSQL, MySQL, Oracle, SQL Server);
* Hands-on experience with big data frameworks (Apache Spark, Hadoop, Databricks);
* Strong knowledge of Azure data services (Data Factory, Synapse, Databricks);
* Experience with workflow orchestration tools (Airflow, Luigi, Prefect);
* Familiarity with NoSQL databases (MongoDB, Cassandra, DynamoDB);
* Solid understanding of data modeling, partitioning, and query optimization;
* Strong communication skills in English (written and verbal).
Highly Desirable Skills:
* Experience with streaming platforms (Kafka, Flink, Spark Streaming);
* Familiarity with MLOps / AI pipelines to support data science workflows;
* Knowledge of data governance and regulatory compliance (GDPR, HIPAA);
* Experience in Agile/Scrum delivery models;
* Prior work in a nearshore delivery environment supporting US/European clients;
* Certifications in cloud platforms or data engineering (AWS Big Data Specialty, Azure Data Engineer Associate, GCP Data Engineer).