Job Title:
A Data Engineer is required to modernize, maintain and develop an e-commerce platform for a retail company. The platform serves millions of customers each week.
* Develop a solution to process large data volumes and integrate it with the current architecture.
Required Skills:
* Strong expertise in Azure Data Factory and Synapse (3+ years).
* Experience in leading distributed teams.
* Proficiency in Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks.
* Ability to integrate data from disparate sources using API integration or message queuing.
* Proficiency in designing and implementing data warehousing solutions.
* Proficiency in SQL to perform complex queries, data transformations, and performance tuning on cloud-based data storages.
* Experience integrating metadata and governance processes into cloud-based data platforms.
* Certification in Azure, Databricks, or other relevant technologies is an advantage.
* Experience with cloud-based analytical databases.
* Experience with Python and Python-based ETL tools.
* Experience with shell scripting in Bash, Unix, or Windows shell.
Nice-to-Have Skills:
* Experience with Elasticsearch.
* Familiarity with containerization and orchestration technologies (Docker, Kubernetes).
* Troubleshooting and Performance Tuning: Ability to identify and resolve performance bottlenecks in data processing workflows and optimize data pipelines for efficient data ingestion and analysis.
* Collaboration and Communication: Strong interpersonal skills to collaborate effectively with stakeholders, data engineers, data scientists, and other cross-functional teams.
Language: English - B2 Upper Intermediate