About the Role
We’re seeking an experienced Senior Code Reviewer with deep Python expertise to review evaluations of AI-generated code responses. You’ll ensure annotation teams maintain high standards for instruction-following, correctness, and functionality in AI training datasets.
- Review and audit annotator evaluations of AI-generated Python code
- Validate code functionality, security, and best practices across multiple programming
- Execute proof-of-work methodology using Docker-based environments
- Provide constructive feedback to maintain annotation quality standards
- Work within established guidelines for evaluation consistency
- Minimum 4+ years in Python development, QA, or code review
- Strong Python expertise, including syntax, debugging, and testing
- Docker proficiency for environment isolation and testing
- Excellent written communication and documentation skills
- Experience with structured QA or annotation workflows
- Experience in AI training, LLM evaluation, or model alignment
- Familiarity with RLHF pipelines and annotation platforms
- Advanced Docker knowledge for complex environment replication
- Work at the intersection of AI and software development
- Remote-first culture with flexible arrangements
- Competitive hourly compensation
- Early-stage pipeline project with long-term potential
Two-Stage Assessment:
- Technical Task (60 minutes) – Coding assessment with recorded responses
- Technical Interview (25 minutes) – Only for candidates who pass the task
- Individual candidate engagement model with managed oversight
- Early submission with competitive rates preferred
- Multiple coding languages required (list provided separately)
- Available for immediate start upon project finalization
Ready to shape the future of AI? Apply now and become our Senior Code Reviewer! – https://forms.gle/qq8iMdYNNnNJeEK3A
