As a Data Engineer (MLOps) at our organization, you will be responsible for building robust data pipelines and ML infrastructure to support our payment processing systems, fraud detection algorithms, and financial analytics solutions.
We are seeking an experienced Data Engineer with strong MLOps expertise and machine learning modeling experience in the financial domain. Your key responsibilities will include designing, developing, and maintaining scalable data pipelines using Python, Airflow, and PySpark to process large volumes of financial transaction data.
You will implement and optimize MLOps infrastructure on AWS to automate the full machine learning lifecycle from development to production. Additionally, you will build and maintain deployment pipelines for ML models using SageMaker and other AWS services.
Collaboration with data scientists and business stakeholders will be essential in implementing machine learning solutions for fraud detection, risk assessment, and financial forecasting. Ensuring data quality, reliability, and security across all data engineering workloads is also crucial.
To be successful in this role, you should have 3-5 years of experience in Data Engineering with a focus on MLOps in production environments. Strong proficiency in Python programming and data processing frameworks (PySpark) is required. Experience with workflow orchestration tools, particularly Airflow, is also necessary.
Hands-on experience with the AWS stack, especially SageMaker, Lambda, S3, and other relevant services, is highly desirable. Working knowledge of machine learning model deployment and monitoring in production is also essential.
A background in financial services or payment processing is highly desirable. Familiarity with containerization (Docker) and CI/CD pipelines is also beneficial. Excellent problem-solving skills and the ability to work in a fast-paced fintech environment are expected.
Key Responsibilities:
* Design, develop, and maintain scalable data pipelines using Python, Airflow, and PySpark to process large volumes of financial transaction data.
* Implement and optimize MLOps infrastructure on AWS to automate the full machine learning lifecycle from development to production.
* Build and maintain deployment pipelines for ML models using SageMaker and other AWS services.
* Collaborate with data scientists and business stakeholders to implement machine learning solutions for fraud detection, risk assessment, and financial forecasting.
* Ensure data quality, reliability, and security across all data engineering workloads.
* Optimize data architecture to improve performance, scalability, and cost-efficiency.
* Implement monitoring and alerting systems to ensure production ML models perform as expected.