Senior AWS Data Engineer - Python, SQL, AirflowLocation: RemoteContract: 3 months initialWe are seeking a skilled Data Engineer with a strong background in Quality Assurance to join our client's team. The ideal candidate will be responsible for developing and executing a comprehensive data quality strategy as they migrate their SQL databases to AWS. Your expertise in Python, SQL, and ETL frameworks like Airflow and dbt will ensure high-quality data pipelines and effective diagnosis of pipeline failures.Key Responsibilities:- Data Quality Strategy Development: Design and implement a robust data quality strategy that addresses current challenges in SQL to AWS migration, ensuring data integrity and reliability throughout the process.- Data Quality Assurance: Conduct thorough quality assessments, validate data transformations, and ensure that all data quality metrics are met. Focus on identifying gaps and areas for improvement within existing ETL workflows.- Technical Proficiency: Utilize Python and SQL for extracting, transforming, and loading data. Implement automated testing frameworks (pytest) to validate data integrity and functionality across pipelines.- ETL Management: Leverage tools like Airflow and dbt to orchestrate data workflows and conduct regular checks to ensure pipeline performance. Identify and troubleshoot sources of failures.- Cloud Infrastructure: Manage data storage and processing on AWS platforms, specifically with Redshift and S3. Optimize data storage solutions and ensure best practices for data security and accessibility.- Root-Cause Analysis: Perform strong root-cause analysis to diagnose and resolve pipeline failures effectively. Document issues and fixes to improve future processes and communication with stakeholders.- Collaboration: Work closely with cross-functional teams, including data scientists and business analysts, to continuously improve data processes and quality metrics. Communicate findings clearly and provide recommendations to enhance system performance.- Documentation: Maintain comprehensive documentation of data quality processes, testing outcomes, and any identified issues, along with their resolutions.Qualifications:- Bachelor’s degree in Computer Science, Data Science, Engineering, or a related field.- Proven experience as a Data Engineer, Data Analyst, or in a similar role with a focus on QA.- Strong proficiency in Python, SQL, and experience with data pipeline tools such as Airflow and dbt.- Familiarity with AWS services, particularly Redshift and S3.- Excellent English speaking and communication skills, with experience presenting technical concepts to non-technical stakeholders.- Strong analytical skills with a keen ability to perform root-cause analysis and troubleshoot complex data issues.- Knowledge of data quality frameworks and methodologies is a plus.