About this Role
We are seeking a highly skilled Senior Software Engineer to join our team in evaluating and training Large Language Models (LLMs) for software development tasks. This role involves working at the intersection of software engineering, open-source ecosystems, and AI-assisted development.
Key Responsibilities
* Evaluate model-generated code responses using a structured ranking system.
* Analyze code diffs for correctness, quality, style, and efficiency.
* Provide clear rationales explaining evaluation decisions.
* Maintain high consistency and objectivity across evaluations.
Requirements
* 7+ years of professional software engineering experience.
* Strong fundamentals in software design, coding best practices, and debugging.
* Excellent ability to assess code quality and correctness.
* Proficient with code review processes and reading diffs in real-world repositories.
* Exceptional written communication skills.
Additional Requirements
* Prior experience with LLM-generated code or evaluation work is a plus.
* Bonus points for experience in LLM research, developer agents, or AI evaluation projects.
Engagement Details
* Commitment: ~20 hours/week.
* Type: Contractor.
* Duration: 1 month (potential extensions based on performance).