Overview
Join the Software Engineer ID45371 role at AgileEngine. This Inc. 5000 company creates award‑winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in application development and AI/ML, and our people‑first culture has earned us multiple Best Place to Work awards.
Responsibilities
Design and develop the team’s microservices – Java services running on Kubernetes (K8s).
Scale the data pipeline to support the processing of billions of events in both low‑latency real‑time and T+1 batch using ClickHouse, Spark, and Kafka.
Tackle data duplication, velocity, schema adherence (and schema versioning), high availability, data governance, and more.
Develop and maintain the data pipeline written mostly in Java and running on K8s in a micro‑service architecture.
Plan and communicate integrations with other teams that consume the data and use it for insights creation.
Improve the way data is stored and served – optimize queries and data formats to enable consumption by a variety of clients.
Perform query optimizations in our data warehouses to ensure data completeness and reliability at scale.
Qualifications
BSc in Computer Science from a top university, or equivalent.
Strong background as a software engineer with at least 3+ years experience with Java.
Experience in data engineering and data pipeline development.
Proven experience with relational and non‑relational databases. Proficient in SQL and query optimizations – preferably in ClickHouse.
Experience with monitoring systems (Prometheus, Grafana, Zabbix, Datadog).
Experience in object‑oriented development and strong software engineering foundations.
Curiosity, ability to work independently and proactively identify solutions.
Excellent verbal and written communication skills in a remote environment.
Upper‑intermediate English level.
Nice to have
Experience working in low‑latency, real‑time systems processing billions of events a day.
Experience with data‑engineering cloud technologies such as Apache Airflow, K8s, ClickHouse, Snowflake, Redis, Spark, caching technologies and/or Kafka.
Benefits
Professional growth: Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps.
Competitive compensation: USD‑based compensation and budgets for education, fitness, and team activities.
A selection of exciting projects: Join projects with modern solutions development and top‑tier clients, including Fortune 500 enterprises and leading product brands.
Flextime: Tailor your schedule for an optimal work‑life balance, with options to work from home and go to the office.
Additional details
Seniority level: Mid‑Senior levelEmployment type: Full‑timeJob function: IT Services and IT Consulting
#J-18808-Ljbffr