Required Skills:
- Selenium
-
Test Automation
-
Python
-
API Testing
-
Performance Testing
-
CI/CD
-
SQL / Databases
Nice to Have:
- Cloud (Azure/AWS)
-
AI / Chatbot Testing
-
ERP / CRM Testing
Lead Software Engineer – Automation Testing We are looking for a highly skilled AI Automation Test Engineer with 6–8 years of experience in designing, developing, and executing automation frameworks for AI/ML-driven applications. The ideal candidate will have hands-on expertise with LangSmith, PyTest, Agentic frameworks, and GenAI testing approaches, and a strong foundational understanding of AI/ML concepts, model evaluation, and data-driven testing. This role requires a deep technical mindset, strong problem‑solving skills, and the ability to collaborate closely with AI engineers, data scientists, and product teams. Responsibilities • AI/GenAI Testing • Design and execute test strategies for GenAI, LLM-based, and agentic systems. • Evaluate AI models for accuracy, performance, safety, robustness, hallucinations, bias, and functional correctness. • Use LangSmith and similar tools for tracing, monitoring, and evaluating LLM outputs. • Validate prompt effectiveness and ensure consistency across different model versions. • Test Automation • Build scalable automation frameworks using PyTest and integrate them into CI/CD pipelines. • Develop automated test suites for APIs, microservices, and AI pipelines. • Implement automation for agentic workflows, including task planning, execution, and error handling validation. Required Experience: • 6–8 years of experience in software testing and automation. • Strong programming expertise in Python. • Strong verbal & written communication: should be able to articulate concisely & clearly. • Should have worked earlier in a multi-stakeholder delivery model, Onshore offshore delivery experience is desired • Experience in Chatbot and AI products. • Azure/AWS Cloud knowledge • Qualification – B.E/B.Tech • Perform testing on sanity, integration, functional and regression testing across all environments (qa, UAT, prod ) • Previous experience developing test scripts for ERPs, CRM and creating test data for ML algorithms will be an added advantage • Familiarity with databases like MySQL, Oracle, PostgreSQL, SQL Server, NoSQL, etc. • PyTest for automation frameworks. Relevant hands on experience (Prior work in these areas is beneficial.) • LangSmith for LLM observability and evaluation. • Agentic frameworks (e.g., ReAct, AutoGen, LangChain Agents, or similar). • Model evaluation techniques and GenAI testing strategies. Understanding and implementation of different test automation approaches (keyword-driven, Data-driven, BDD, hybrid) • Qualification – B.E/B.Tech
