Job Title: Senior Python Data Engineer
This position involves designing and building high-performance data processing pipelines using Python libraries such as Pandas and Polars.
* Design and develop efficient data processing pipelines using Python, leveraging its vast ecosystem of libraries and frameworks to streamline data analysis and processing tasks.
* Develop and expose RESTful APIs using FastAPI or similar frameworks, ensuring seamless integration with various data sources and applications.
* Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel reports, providing valuable insights into business operations and performance metrics.
* Collaborate on infrastructure-as-code automation using Bicep (or similar IaC tools), working closely with cross-functional teams to design and implement scalable and maintainable infrastructure solutions.
* Participate in design discussions for future migration to Snowflake and/or a data lake architecture, contributing expertise in data modeling principles, enterprise messaging patterns, and data integration from various sources.
Requirements:
* Strong proficiency in Python for data processing, with experience in libraries like Pandas, NumPy, and scikit-learn.
* Experience building backend services or APIs, with knowledge of design patterns and principles.
* Solid understanding of data modeling principles, including data warehousing, ETL processes, and data visualization.
* Familiarity with enterprise messaging patterns and data integration from various sources, including databases, APIs, and microservices.
* Comfortable working with GitHub and CI/CD pipelines, with experience in Agile development methodologies.
* Experience with infrastructure-as-code automation using Bicep or comparable tools, with knowledge of cloud computing platforms and containerization.
Benefits:
* Opportunity to work on cutting-edge projects and technologies.
* Collaborative and dynamic work environment.
* Professional growth and development opportunities.
* Competitive salary and benefits package.
How to Apply:
If you are passionate about data engineering and have the required skills and experience, please submit your application.