Transform Data into Insights
We are seeking a seasoned Data Engineer to design and develop scalable data pipelines using Python, Airflow, and PySpark. This role involves processing large volumes of financial transaction data to support our payment processing systems, fraud detection algorithms, and financial analytics solutions.
Key Responsibilities:
* Develop and maintain robust data pipelines to process financial transaction data.
* Implement and optimize MLOps infrastructure on AWS to automate the machine learning lifecycle.
* Build and maintain deployment pipelines for ML models using SageMaker and other AWS services.
* Collaborate with data scientists and business stakeholders to implement machine learning solutions for fraud detection, risk assessment, and financial forecasting.
* Ensure data quality, reliability, and security across all data engineering workloads.
Requirements & Qualifications:
* A minimum of 3-5 years of experience in Data Engineering with a focus on MLOps in production environments.
* Strong proficiency in Python programming and data processing frameworks (PySpark).
* Experience with workflow orchestration tools, particularly Airflow.
* Hands-on experience with the AWS stack, especially SageMaker, Lambda, S3, and other relevant services.
* Working knowledge of machine learning model deployment and monitoring in production.
* Experience with data modeling and database systems (SQL and NoSQL).
* Familiarity with containerization (Docker) and CI/CD pipelines.