Sr Python Data Engineer Job Description
This is a pivotal role in our organization, responsible for designing, building and maintaining high-performance data processing pipelines using Python libraries such as Pandas and Polars.
The ideal candidate will have strong proficiency in Python for data processing and experience with backend services or APIs using frameworks like FastAPI.
* Designing and building data processing pipelines using Python libraries
* Developing and exposing RESTful APIs using FastAPI or similar frameworks
* Consuming and processing normalized Parquet files from multiple upstream sources to generate dynamic Excel reports
* Contributing to a spec-driven development workflow to scaffold and generate API/data pipeline code
* Optimizing report generation logic for speed and scalability
* Integrating with messaging and storage mechanisms such as Service Bus and Storage Accounts
* Collaborating on infrastructure-as-code automation using Bicep or similar IaC tools
Required skills and experience include:
* Strong proficiency in Python for data processing
* Experience building backend services or APIs using frameworks like FastAPI
* Solid understanding of data modeling principles and handling normalized datasets
* Familiarity with enterprise messaging patterns and data integration from various sources
* Experience working with GitHub and CI/CD pipelines
* Infrastructure-as-Code experience with Bicep or comparable tools
We value diversity and welcome applications from all qualified candidates without regard to gender, identity, or personal background.