Job Summary
LymData Labs is looking for a QA / Test Engineer to take ownership of quality across our AI-powered software products and client projects.
This is a build-from-scratch QA role where you will define the testing strategy, establish release standards, build automation, perform API and web testing, and help us develop effective approaches for testing AI-powered features.
You will work across both the conventional web/API layer and the AI layer, including testing AI outputs, prompt changes, model upgrades, RAG systems, and edge cases.
Key ResponsibilitiesTest Strategy & Quality
- Define and own the overall QA and testing strategy.
- Decide what should be tested manually versus automated.
- Define release gates and pre-release checklists.
- Establish and maintain quality standards across client and in-house products.
Manual & Exploratory Testing
- Create test plans, test scenarios, and detailed test cases from requirements, user stories, and designs.
- Perform functional, regression, integration, and UAT testing.
- Conduct exploratory and ad-hoc testing to identify issues beyond documented test cases.
- Perform cross-browser testing across Chrome, Safari, Firefox, and Edge.
- Perform cross-device testing across Android and iOS.
- Validate UI and usability against designs.
- Perform basic accessibility checks.
- Maintain regression test suites and test documentation.
- Manually review AI outputs and evaluate their quality.
Automation & API Testing
- Build and maintain the automation framework from scratch.
- Work with tools such as Playwright, Cypress, or Selenium.
- Identify high-value test scenarios suitable for automation.
- Perform API testing using Postman, REST Assured, or equivalent tools.
- Integrate automated tests into CI pipelines so failures are identified early.
AI / LLM Testing
- Build and maintain evaluation datasets for AI-powered features.
- Test prompt changes and model version upgrades.
- Perform regression testing when AI providers or models change.
- Test retrieval quality and grounding in RAG-based systems.
- Identify hallucinations, prompt injection vulnerabilities, unsafe outputs, and edge-case failures.
- Test failure scenarios such as rate limits, timeouts, malformed model responses, token limits, and fallback behaviour.
- Monitor AI output quality in production and identify quality drift.
Collaboration & Reporting
- Own the bug lifecycle from discovery through verified resolution.
- Provide clear reproduction steps and severity/priority assessments.
- Participate in requirement discussions and identify ambiguity early.
- Report on escaped defects, regression trends, and AI evaluation results.
- Collaborate closely with developers and the engineering team.
Required Qualifications
- 3+ years of experience in software testing / QA, with strong experience testing web applications.
- Strong manual testing and exploratory testing skills.
- Ability to create comprehensive test cases from requirements and designs.
- Good understanding of:
- Boundary Value Analysis
- Equivalence Partitioning
- Decision Tables
- Defect Lifecycle
- Severity vs. Priority
- Risk-Based Testing
- Experience with cross-browser and cross-device testing.
- Experience with a test management tool such as TestRail, Zephyr, Xray, Qase, or equivalent.
- Hands-on experience with UI automation using Playwright, Cypress, or Selenium.
- Experience with API testing and understanding API documentation.
- Proficiency in Python or JavaScript for writing test scripts.
- Working knowledge of SQL.
- Experience with bug tracking tools such as Jira, Linear, or ClickUp.
- Familiarity with Git.
- Ability to write clear and actionable bug reports.
- Good written and spoken English communication skills.
Preferred Qualifications
Experience with the following will be an advantage:
- Testing LLM or ML-powered applications.
- AI evaluation tools such as Promptfoo, DeepEval, Ragas, LangSmith, or Langfuse.
- Working directly with LLM APIs such as OpenAI, Anthropic, or open-source models.
- Understanding of RAG architecture, embeddings, vector search, retrieval, and grounding.
- CI/CD tools such as GitHub Actions, GitLab CI, or Jenkins.
- Performance testing using JMeter or k6.
- Setting up QA/testing infrastructure in a startup or small-team environment.
- ISTQB Foundation certification.
AI testing experience is a bonus, not a requirement. We value strong QA fundamentals, scripting ability, curiosity, and the ability to reason about how AI systems should be tested.
What We’re Looking For
We are looking for someone who:
- Can independently own QA and quality standards.
- Thinks like a user, not just a test case executor.
- Enjoys exploratory testing and finding unexpected issues.
- Is comfortable testing systems where outputs may vary.
- Can determine what "correct enough" means for AI-powered features.
- Is confident enough to challenge a release when quality standards are not met.
- Looks for opportunities to automate repetitive testing tasks.
- Is curious about AI and enjoys finding ways to break systems before users do.
Why Join LymData Labs?
- Real ownership: Build the QA process rather than inheriting an existing one.
- Work on AI-powered products: Solve new and evolving testing challenges.
- Direct impact: Work closely with a small engineering team with short feedback cycles.
- Growth opportunity: Potential progression into a QA Lead role as the team grows.
- Learning support: Learning budget available for certifications and courses.
Application Process
Interested candidates can apply with their updated CV.
Email: [email protected]
Phone: 9495834213
As part of the application, please include a short note describing one bug you are proud of finding and how you discovered it.
Screening Questions
- How would you test a login flow you've never seen before? What would you check beyond the happy path?
- Which automation frameworks have you personally set up from scratch?
- How do you decide what to automate and what to keep manual?
- Have you tested an AI/LLM-powered feature? If yes, how did you determine whether the output was acceptable?
- What percentage of your previous role involved manual testing versus automation?
- Which test management tools have you used?
- Have you worked as the only QA person on a team? If yes, what was the most challenging part?
- What are your current CTC, expected CTC, and notice period?
Pay: From ₹30,000.00 per month
Work Location: Hybrid remote in Kochi, Kerala (Kochi)
