High-Performance Data Engineering
As a seasoned Data Engineer, you will play a pivotal role in designing and developing robust data pipelines to support our payment processing systems, fraud detection algorithms, and financial analytics solutions. With a strong focus on MLOps expertise, you will be responsible for implementing and optimizing machine learning infrastructure on AWS to automate the full machine learning lifecycle from development to production.
Key Responsibilities:
* Design and Develop Scalable Data Pipelines: Utilize Python, Airflow, and PySpark to process large volumes of financial transaction data.
* Implement MLOps Infrastructure: Automate the full machine learning lifecycle using SageMaker and other AWS services.
* Build Deployment Pipelines for ML Models: Ensure seamless deployment of machine learning models using Docker and CI/CD pipelines.
* Collaborate with Cross-Functional Teams: Work closely with data scientists and business stakeholders to implement machine learning solutions for fraud detection, risk assessment, and financial forecasting.
* Ensure Data Quality and Security: Guarantee data reliability, integrity, and security across all data engineering workloads.
* Optimize Data Architecture: Continuously improve data architecture to enhance performance, scalability, and cost-efficiency.
* Implement Monitoring and Alerting Systems: Ensure production ML models perform as expected through real-time monitoring and alerting.
Requirements:
* 3-5 years of experience in Data Engineering with a focus on MLOps in production environments.
* Strong proficiency in Python programming and data processing frameworks (PySpark).
* Experience with workflow orchestration tools, particularly Airflow.
* Hands-on experience with AWS stack, especially SageMaker, Lambda, S3, and other relevant services.
* Working knowledge of machine learning model deployment and monitoring in production.
* Experience with data modeling and database systems (SQL and NoSQL).
* Knowledge of financial services or payment processing domain is highly desirable.
* Familiarity with containerization (Docker) and CI/CD pipelines.
* Excellent problem-solving skills and ability to work in a fast-paced fintech environment.
About the Role
This position requires an experienced Data Engineer with a proven track record of delivering high-quality data pipelines and ML infrastructure in production environments. If you are passionate about building scalable data architectures and driving business value through data-driven insights, we encourage you to apply for this exciting opportunity.