Overview
Quality Assurance Engineer owns quality end-to-end — from test strategy through release sign-off — for AI-driven insurance workflows. They are responsible for validating agentic / LLM-based systems where outputs are probabilistic, workflows are autonomous, and accuracy is business-critical. They also need too make sure that flows are meeting the expectations of the business as well as IT owners of the customer organizations
Key Responsibilities
- Define and own the overall test strategy for the platform, covering functional, regression, integration, API, UAT and AI/agent behavior testing
- Create detailed test plans, test designs, test scenarios and test cases mapped to business requirements and workflows for insurance workflows (submissions, quoting, policy servicing, endorsements, renewals, claims/FNOL, COIs)
- Design evaluation frameworks for agentic AI outputs — accuracy, hallucination detection, grounding, guardrail adherence, prompt/response regression and human-in-the-loop checkpoints
- Validate integrations with AMS platforms (Applied Epic, AMS360, HawkSoft, etc.), carrier portals, ACORD forms and document extraction pipelines
- Manage defect lifecycle: triage, root-cause analysis, severity/priority assessment and closure tracking with engineering and product teams
- Define QA metrics and reporting (test coverage, defect density, escape rate, AI eval scores) and drive release readiness / go-no-go decisions
- Support client UAT during implementations, including test data preparation with realistic datasets for insurance domain/edge-case coverage
- Build and maintain automation suites (UI, API, and data validation) and integrate them into CI/CD pipelines
- Mentor junior QA engineers and establish QA best practices, standards and documentation.
Required Skills & Experience
Must Have
- 3–5 years of QA/testing experience
- Demonstrated experience creating test strategy, test plans and test design from scratch for complex enterprise products
- Hands-on experience testing agentic AI / LLM-based platforms — evaluating non-deterministic outputs, prompt regression, RAG/grounding accuracy and agent workflow correctness
- Strong API testing skills (Postman/REST Assured), SQL for data validation
- Experience with test management and defect tools (Jira, TestRail/Zephyr/Xray) and Agile/Scrum delivery
- Excellent analytical, documentation and communication skills; able to work directly with product, engineering and client teams
Good to Have
- 2+ years’ experience in testing applications in the U.S. P&C insurance domain (broker, carrier, or MGA systems)
- Automation experience using tools such as Selenium/Playwright/Cypress or similar
- Familiarity with insurance data standards and artifacts: ACORD forms, policy documents, loss runs and carrier downloads.
- Exposure to AI evaluation tooling/frameworks (e.g., LLM eval harnesses, golden datasets, model output scoring)
- Experience in client-facing implementation or UAT support roles; ISTQB or equivalent certification.
Pay: From ₹8,000,000.00 per year
Work Location: In person
