GradeLab is building the AI operating system for assessments. We help schools, universities, and organisations automate handwritten exam evaluation using OCR, large language models, and intelligent grading pipelines, so every student gets a fair, consistent, and accurate result.
We're looking for a QA Engineer who thinks like both an attacker and an end user. This isn't a traditional QA role: you'll validate AI behaviour, grading accuracy, OCR quality, and system reliability across a fast-moving production platform.
What you'll do
- AI evaluation and benchmarking — build golden datasets from real exam papers, measure grading accuracy against human evaluators, and catch regressions before they ship.
- AI observability — use Langfuse and evaluation frameworks to monitor grading accuracy, human agreement, OCR confidence, rubric compliance, cost, and latency in production.
- OCR validation — stress-test the pipeline against poor handwriting, multiple languages, low-resolution scans, folded papers, diagrams, equations, and every real-world document quirk you can find.
- Grading validation — test partial marking, step-wise marking, rubric compliance, and score calculations for consistency between human and AI evaluators.
- Prompt evaluation and AI safety — run adversarial testing including prompt injection, jailbreak attempts, and context poisoning to keep the system secure.
- Edge case testing — break the system with real-world scenarios: crossed-out answers, missing pages, torn sheets, blank pages, and everything students actually submit.
- API, load, and release testing — validate backend APIs, run performance tests under production-scale load, and sign off on feature rollouts before they go live.
- Bug investigation — write clear, actionable reports with reproduction steps, root cause hypotheses, and Langfuse trace IDs.
What we're looking for
- Strong manual testing experience with excellent analytical and debugging instincts.
- Hands-on API testing experience and solid SQL fundamentals.
- Strong documentation skills and real attention to detail.
- Comfort working with bug tracking systems and thinking adversarially about a product.
Nice to have: Langfuse, Promptfoo, OpenAI Evals, Playwright, Cypress, Selenium, k6, Locust, JMeter, Postman, Bruno, Docker, Git, GitHub Actions, Sentry, Grafana, OpenTelemetry, AI/LLM evaluation experience, OCR systems, or EdTech/SaaS background.
Why GradeLab
- Own quality across AI, backend, frontend, and infrastructure at a fast-growing startup.
- Work directly with the founders, with real ownership from day one.
- Help shape the QA and AI evaluation practice from the ground up.
- Competitive salary, learning opportunities, and ESOPs for high performers.
Hiring process
- Resume screening
- Practical QA assignment (a real GradeLab evaluation challenge)
- Technical interview
- Founder discussion
- Final offer
Pay: From ₹20,000.00 per month
Benefits:
- Flexible schedule
- Work from home
Work Location: Hybrid remote in Ahmedabad, Gujarat
