Data Engineering Expert
We are seeking an experienced Data Engineer with strong MLOps expertise and machine learning modeling experience in the financial domain.
About the Role
In this role, you will be responsible for building robust data pipelines and ML infrastructure to support our payment processing systems, fraud detection algorithms, and financial analytics solutions.
Key Responsibilities
* Data Pipeline Design and Development: Design, develop, and maintain scalable data pipelines using Python, Airflow, and PySpark to process large volumes of financial transaction data.
* MLOps Infrastructure Implementation: Implement and optimize MLOps infrastructure on AWS to automate the full machine learning lifecycle from development to production.
* Deployment Pipeline Creation: Build and maintain deployment pipelines for ML models using SageMaker and other AWS services.
* Collaboration and Stakeholder Engagement: Collaborate with data scientists and business stakeholders to implement machine learning solutions for fraud detection, risk assessment, and financial forecasting.
* Data Quality and Security: Ensure data quality, reliability, and security across all data engineering workloads.
* Architecture Optimization: Optimize data architecture to improve performance, scalability, and cost-efficiency.
* Monitoring and Alerting: Implement monitoring and alerting systems to ensure production ML models perform as expected.
Qualifications and Skills
* Experience in Data Engineering: 3-5 years of experience in Data Engineering with a focus on MLOps in production environments.
* Programming and Data Processing: Strong proficiency in Python programming and data processing frameworks (PySpark).
* Workflow Orchestration Tools: Experience with workflow orchestration tools, particularly Airflow.
* AWS Stack: Hands-on experience with AWS stack, especially SageMaker, Lambda, S3, and other relevant services.
* Machine Learning Model Deployment: Working knowledge of machine learning model deployment and monitoring in production.
* Data Modeling and Database Systems: Experience with data modeling and database systems (SQL and NoSQL).
* Financial Services Domain Knowledge: Knowledge of financial services or payment processing domain is highly desirable.
* Containerization and CI/CD Pipelines: Familiarity with containerization (Docker) and CI/CD pipelines.
* Problem-Solving and Adaptability: Excellent problem-solving skills and ability to work in a fast-paced fintech environment.