Job Title: Sr Python Data Engineer
About the Position:
We are seeking a skilled Python Data Engineer to join our team. In this role, you will design, build, and maintain high-performance data processing pipelines using Pandas and Polars.
* Develop and expose RESTful APIs using FastAPI or similar frameworks.
* Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel reports.
* Optimize report generation logic for speed and scalability.
* Integrate with messaging and storage mechanisms like Service Bus and Storage Accounts.
* Collaborate on infrastructure-as-code automation using Bicep or similar IaC tools.
* Participate in design discussions for future migration to Snowflake and/or a data lake architecture.
Required Skills and Qualifications:
* Strong proficiency in Python for data processing, including expertise in Pandas and familiarity with Polars and openpyxl.
* Experience building backend services or APIs using frameworks like FastAPI.
* Solid understanding of data modeling principles and handling normalized datasets.
* Familiarity with enterprise messaging patterns and data integration from various sources.
* Experience working with GitHub and CI/CD pipelines.
* Infrastructure-as-Code experience with Bicep or comparable tools like Terraform and AWS CDK.
* Comfort with spec-driven development and leveraging AI tools like Github Copilot.