The objective of this project is to modernize and develop an eCommerce platform for a major US retail company. The project involves multiple Product Teams focused on various domains, including Customer, Loyalty, Search and Browse, Data Integration, and Cart.
* Key priorities include onboarding new brands, re-architecting the platform, database migrations, and migrating microservices to a unified cloud-native solution without disrupting business operations.
This role focuses on designing solutions that support processing big data volumes and integrating with current architecture. This includes developing solutions for big retail companies that process big data volumes and integrate with existing architectures.
* Azure Data Factory and Synapse are mandatory tools for this role. Recent hands-on experience (3+ years) with these tools is required.
* Leading a distributed team is also essential for this position.
* Strong expertise in designing and implementing data models, including conceptual, logical, and physical data models, is necessary for efficient data storage and retrieval.
* Deep knowledge of Microsoft Azure is required, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks.
* Extensive experience in building robust and scalable ETL/ELT pipelines to extract, transform, and load data from various sources into data lakes or data warehouses is necessary.
* Data integration from disparate sources, including databases, APIs, and external data providers, can be achieved using API integration or message queuing.
* Proficiency in designing and implementing data warehousing solutions, dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, and Data Vault is necessary.
* SQL skills to perform complex queries, data transformations, and performance tuning on cloud-based data storages are crucial.
* Experience in integrating metadata and governance processes into cloud-based data platforms is necessary.
* Certification in Azure, Databricks, or other relevant technologies is an added advantage.
* Cloud-based analytical databases and Azure MI, Azure Database for Postgres, Azure Cosmos DB, Azure Analysis Services, and Informix require experience.
* Python and Python-based ETL tools require experience.
* Bash, Unix, or windows shell scripting is preferable.
Nice-to-Have Skills:
* Elasticsearch experience.
* Familiarity with containerization and orchestration technologies (Docker, Kubernetes).
* Troubleshooting and Performance Tuning: ability to identify and resolve performance bottlenecks in data processing workflows and optimize data pipelines for efficient data ingestion and analysis.
* Collaboration and Communication: Strong interpersonal skills to collaborate effectively with stakeholders, data engineers, data scientists, and other cross-functional teams.
Languages:
English: B2 Upper Intermediate