Key Responsibilities:
* Design, develop, and optimize large-scale AI/LLM models for efficient search across vast medical and scientific document repositories.
* Transform and embed over half a billion documents to ensure seamless searchability and contextual accuracy.
* Evolve semantic chunking strategies and enhance the automated evaluation pipeline.
* Fine-tune LLMs for textual RAG use cases.
The role involves hands-on experimentation, model development, and backend engineering with deployments to non-prod environments and collaboration with DevOps for production rollout. Strong communication and collaboration with the team are essential.
* 1+ years of experience as a Search Engineer or AI Engineer required.
* Proficiency in OpenSearch and building scalable search systems essential.
* 2+ years of Python development experience including API creation, model training, testing, and general backend programming required.
* Familiarity with LangChain for building LLM workflows using tools, memory, and retrieval desired.
AWS infrastructure experience and exposure to RAG architectures, specifically textual RAG use cases, are valuable skills.