About the Role
We are seeking a skilled Senior Software Engineer to join our team on a remote, part-time basis. This individual will play a crucial role in shaping the future of AI-assisted software development by contributing to the evaluation and training of Large Language Models (LLMs).
Key Responsibilities:
* Evaluate model-generated code responses using a structured ranking system.
* Analyze code diffs for correctness, quality, style, and efficiency.
* Provide clear rationales explaining the reasoning behind each ranking decision.
* Maintain consistency and objectivity across evaluations.
Requirements:
* 7+ years of professional software engineering experience.
* Strong fundamentals in software design, coding best practices, and debugging.
* Excellent ability to assess code quality, correctness, and maintainability.
* Proficient with code review processes and reading diffs in real-world repositories.
* Exceptional written communication skills.
Preferred Qualifications:
* Prior experience with LLM-generated code or evaluation work.
* Experience in developer tools or automation systems.
Work Arrangements:
* Commitment: ~20 hours/week.
* Type: Contractor.
* Duration: 1 month (with potential extensions based on performance and fit).