Company Description
The Senior Data Engineer will collaborate with our team to build, enhance, and maintain high-performance data systems and machine learning models, ensuring their scalability and reliability.
Responsibilities include designing and maintaining high-throughput data pipelines, optimizing real-time and batch data processing using Scala, and integrating advanced data solutions to improve performance. The role involves working with large-scale distributed systems, handling terabytes of data per second and processing millions of writes per second.
Our team is at the forefront of AdTech innovation, enabling cutting-edge solutions for data-driven advertising. We value hard work and flexibility, offering an environment where engineers can experiment, innovate, and push technological boundaries.
Our client is a leading AdTech company specializing in high-performance data systems and machine learning models, processing terabytes of data per second and millions of writes per second.
It is a large-scale AdTech platform designed to process massive amounts of data in real time, optimizing data-driven advertising with advanced machine learning models. It ensures scalability and reliability, enabling businesses to maximize efficiency and performance while leveraging cutting-edge data solutions.
Job Description
Work with Apache Spark batch and real-time streaming to process data at scale.
Work with Scala microservices hosted on Kubernetes (GKE) to support our products.
Utilize MlLib and TensorFlow on Vertex AI to solve prediction and simulation challenges, as well as various machine learning problems.
Deploy new models and engines for innovative ideas and analysis.
Qualifications
5+ years of experience as a Data Engineer.
Advanced knowledge of various non-relational schema models (Column Family, Graph, Document, Object).
Experience with advanced topologies and architecture design, such as Kappa and Lambda architectures.
GCP Dataflow experience.
Proficiency in Scala and Python.
Strong background working with Large Language Models (LLMs).
Experience with high-throughput systems.
Excellent communication skills and at least C1 level of English.
WOULD BE A PLUS
Experience in the AdTech industry.
Additional Information
PERSONAL PROFILE
Problem-solving mindset.
Ability to work closely with cross-functional teams.
Willingness to experiment with new technologies and approaches.
#J-18808-Ljbffr