Senior Software Test Engineer – AI Applications
Location
Remote
Experience
5+ years in Software Testing, with atleast 3+ years of experience testing AI/ML-based applications.
Job Summary
We are looking for a highly skilled Senior QA/Test Engineer with hands-on experience in testing Artificial Intelligence (AI), Machine Learning (ML), and Generative AI applications. The ideal candidate should possess strong expertise in functional, automation, API, and performance testing while understanding the unique challenges associated with validating AI models, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and intelligent automation systems.
The candidate will work closely with Product Managers, AI Engineers, Data Scientists, Developers, and DevOps teams to ensure high-quality AI-powered products are delivered to customers.
Key Responsibilities
- Design, develop, and execute comprehensive test strategies for AI-powered applications.
- Validate AI/ML models for functionality, accuracy, consistency, reliability, and performance.
- Test Generative AI applications including chatbots, virtual assistants, document intelligence, and AI copilots.
- Develop detailed test plans, test cases, and test scenarios for AI workflows.
- Perform functional, regression, integration, system, exploratory, and user acceptance testing.
- Validate AI model outputs for correctness, relevance, hallucinations, bias, and edge cases.
- Test prompt engineering scenarios and evaluate LLM responses across different prompts.
- Perform API testing using tools such as Postman or Swagger.
- Develop and maintain automation frameworks using Selenium, Playwright, Cypress, or similar tools.
- Execute performance and load testing using tools like JMeter, Locust, or K6.
- Validate AI inference latency, scalability, and response time under varying workloads.
- Work closely with AI Engineers to verify model deployments and production readiness.
- Identify, log, prioritize, and track defects using Jira or similar defect management tools.
- Participate in Agile ceremonies including Sprint Planning, Daily Stand-ups, Retrospectives, and Release Planning.
- Review requirements and provide quality recommendations during the design phase.
- Ensure compliance with security, privacy, and responsible AI testing standards.
- Mentor junior QA engineers and review test artifacts.
Required Skills
Core QA Skills
- Strong experience in Manual Testing.
- Automation Testing using Selenium, Playwright, Cypress, or equivalent.
- API Testing (Postman, REST Assured, Swagger).
- Regression Testing.
- Integration Testing.
- System Testing.
- Smoke and Sanity Testing.
- Performance Testing.
- Security Testing fundamentals.
- Cross-browser and Cross-platform testing.
- Mobile Application Testing (preferred).
AI Testing Skills
Experience testing one or more of the following:
- AI-powered web applications
- Machine Learning applications
- Large Language Models (LLMs)
- ChatGPT/OpenAI APIs
- Anthropic Claude
- Google Gemini
- Microsoft Azure OpenAI
- AI Chatbots
- Intelligent Document Processing
- OCR-based AI solutions
- Speech-to-Text systems
- Text-to-Speech systems
- Computer Vision applications
- Recommendation Engines
- Predictive Analytics platforms
Should have knowledge of:
- Prompt Engineering validation
- Hallucination testing
- AI bias testing
- Explainability testing
- Model accuracy validation
- Confidence score validation
- Dataset validation
- Model drift monitoring
- AI safety testing
- Responsible AI principles
Job Type: Full-time
Pay: ₹40,000.00 - ₹70,000.00 per month
Benefits:
Application Question(s):
- What is your current ctc,expected and notice period ?
- How many years of experience do you have in Automation Testing?
- How many years of experience do you have testing AI/ML applications?
- Have you worked on Generative AI or LLM-based applications (e.g., ChatGPT, Gemini, Claude)?
- Have you created and maintained automated test scripts?
- Do you have experience validating AI model outputs for accuracy, hallucinations, or bias?
- Do you have hands-on experience with Automation Testing using Selenium, Playwright, or Cypress?
- Have you performed performance or load testing using JMeter, K6, or similar tools?
- Do you have experience testing AI or Machine Learning applications?
Education:
Work Location: Remote
