As a highly skilled Data Engineer, you will play a crucial role in designing and building robust data pipelines using Python, Airflow, and PySpark to process large volumes of financial transaction data. This is an exciting opportunity to work with state-of-the-art machine learning and cloud infrastructure in a fast-paced fintech environment.
Key Responsibilities:
* Develop scalable data pipelines to support payment processing systems, fraud detection algorithms, and financial analytics solutions.
* Implement and optimize MLOps infrastructure on AWS to automate the full machine learning lifecycle from development to production.
* Build and maintain deployment pipelines for ML models using SageMaker and other AWS services.
* Collaborate with data scientists and business stakeholders to implement machine learning solutions for fraud detection, risk assessment, and financial forecasting.
* Evaluate data quality, reliability, and security across all data engineering workloads.
Requirements:
* 3-5 years of experience in Data Engineering with a focus on MLOps in production environments.
* Strong proficiency in Python programming and data processing frameworks (PySpark).
* Experience with workflow orchestration tools, particularly Airflow.
* Familiarity with AWS stack, especially SageMaker, Lambda, S3, and other relevant services.
* Knowledge of machine learning model deployment and monitoring in production.
* Background in finance or payment processing is a plus.