Senior Data Engineer (AWS, PySpark, Airflow)
Employment Type: independant contractor
About the Role
We are looking for a Senior Data Engineer to join our Business Model & Pricing (BMP) Analytics team. In this role, you will design and build scalable data systems that power analytics, reporting, and AI/ML initiatives across the organization.
You will work closely with data scientists, analysts, and business stakeholders to ensure reliable, high-quality data is available to drive strategic decisions.
What You’ll Do
* Design, build, and maintain scalable data pipelines using SQL, Python, and PySpark
* Develop and optimize data models and data warehouse solutions (Snowflake, data lake)
* Implement and manage workflow orchestration using Airflow
* Build and support data infrastructure on AWS
* Ensure data quality, governance, and reliability across systems
* Collaborate with analytics and business teams to deliver data-driven solutions
* Support AI/ML initiatives by enabling data pipelines and MLOps workflows
* Contribute to data products and applications (e.g., Streamlit dashboards, chatbot integrations)
* Drive best practices in data engineering, automation, and performance optimization
Required Qualifications
* 5+ years of experience in Data Engineering or Data Platform development
* Strong hands-on experience with SQL and Python
* Experience with PySpark / Spark (big data processing)
* Hands-on experience with Airflow or similar orchestration tools
* Strong experience with AWS (S3, Glue, EMR, Redshift, etc.)
* Experience with data warehousing (Snowflake or similar)
* Solid understanding of data modeling, ETL/ELT, and data architecture
* Experience with Git and collaborative development workflows
* Strong problem-solving and communication skills
Preferred Qualifications
* Experience with data lake architectures and hybrid environments
* Exposure to MLOps and machine learning pipelines
* Experience with Streamlit or data applications
* Familiarity with chatbots or conversational data interfaces
* Experience working in global or matrix organizations
* Contributions to documentation platforms (Confluence, etc.)