Job Opportunity
We are seeking a highly skilled Senior Data Engineer to join our team. The ideal candidate will have experience building, managing, and optimizing data pipelines, defining data flows and access to data sources, and ensuring compliance with data governance, data architecture, and data security requirements.
Key Responsibilities
* Design, build, and manage data pipelines for data structures that encompass data modeling, data transformation, schemas, metadata, and workload management;
* Drive automation through effective metadata management;
* Ensure compliance and governance during data use, including the maintenance of the data catalog;
* Collaborate and support data owners and other stakeholders around data topics;
* Ensure the proper use of the data architecture defined by the data factory team;
* Respect and follow security principles ensuring compliance with data delivery.
Competencies & Skills
* Strong experience with various data management architectures, mainly in the AWS environment;
* Strong experience with data modeling;
* Ability to design, build, and manage data pipelines;
* Experience working with commercial and open-source message queuing technologies;
* Highly creative and collaborative;
* Excellent communication skills and the ability to collaborate effectively with cross-functional teams.
Technical Knowledge
* Solid knowledge in Databricks, Spark, Python, Terraform, SQL, Scala, S3, AWS Glue, AWS Athena, AWS Lambda, AWS Step Functions, big data, relational databases;
* Experience in EMR, Gitflow, Agile Methodology, DevOps.
Languages
* English: Advanced or Fluent
Additional Information
* Remote work is available.