Are you passionate about ensuring the quality and reliability of AI-driven software that transforms how professionals work? Join a dynamic and highly skilled team at Thomson Reuters, where we invest deeply in AI technologies and explore emerging fields with the backing of a global leader. As a Senior QA Software Engineer - AI, you will play a key role in testing and validating intelligent systems that power the next generation of expert solutions across legal, tax, risk, and compliance domains.
About The Role:
As a Senior QA Software Engineer - AI, you will collaborate with a cross-functional team of product managers, UX designers, AI engineers, and software developers to design, implement, and execute comprehensive testing strategies for AI-powered solutions.
- Design and implement comprehensive test strategies for advanced AI systems, including multi-component pipelines, retrieval-augmented generation (RAG), and custom AI agents with multi-step reasoning.
- Develop automated testing frameworks for AI model performance, accuracy, and reliability in production environments.
- Create test cases specific to AI/ML systems, including model validation, output quality assessment, and edge case testing.
- Validate AI model integrations with production software through robust APIs and scalable data pipelines.
- Test AI systems adapted to specialized domains, ensuring quality for expert systems in areas such as legal, tax, and compliance.
- Evaluate and prototype cutting-edge testing techniques for AI systems to address unique quality challenges.
- Develop proof-of-concept automated testing frameworks for new AI-driven features.
- Stay current with AI testing methodologies, quality metrics, and emerging technologies.
- Implement testing approaches for AI model drift, bias detection, and fairness validation.Break down functional requirements into comprehensive test specifications and automation strategies.
- Mentor junior QA engineers and facilitate technical discussions on AI testing best practices.
- Contribute to MLOps and LLMOps practices with a focus on quality assurance, continuous testing, and monitoring.
- Act as a thought leader in AI quality assurance, sharing expertise in company-wide forums and representing the organization in emerging testing technologies.
- Implement comprehensive automated testing frameworks and monitoring systems for AI model performance, accuracy, and reliability.
- Create and maintain test automation suites using industry-standard tools and frameworks (Selenium, pytest, etc.).
- Ensure compliance with ethical AI principles, security standards, and regulatory requirements through thorough testing.
- Conduct systems analysis and recommend operational improvements to testing processes.
- Report deviations of quality to development teams and create informative bug reports with effective follow-up.
- Create detailed test reports documenting all issues found during testing iterations.
- Work closely with AI researchers, engineers, designers, and product teams to translate AI quality requirements into effective test strategies.
- Participate in daily stand-ups and iteration meetings in an Agile/Scrum environment.
- Optimize testing processes considering factors like test coverage, execution time, and resource usage.
- Support application feature enhancements by ensuring robust quality assurance for AI capabilities.
About You:
You are a fit for the position of Senior QA Software Engineer - AI if your background includes:
Required Skills and Experience:
- Bachelor's degree in Computer Science or equivalent experience.
- 5+ years of experience in software quality assurance and test automation; at least 2 years focused on testing AI/ML systems.
- 70% automation/development experience, 30% manual testing experience.
- Proficiency in Python and Java with strong coding skills for test automation.
- Experience with test automation tools and frameworks (e.g., Selenium, pytest, JUnit).
- Strong understanding of machine learning principles, evaluation metrics, and AI system testing methodologies.
- Knowledge of MLOps/LLMOps and the end-to-end lifecycle of AI-powered software applications.
- Experience testing AI model integrations in production systems using APIs and data pipelines.
- Good knowledge and experience working with relational databases (SQL, Oracle) and ability to write complex SQL scripts.
- Experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization tools (e.g., Docker, Kubernetes).
- Hands-on experience with source control and CI/CD systems such as Git and Jenkins.
- Good knowledge and experience working with Web Services and RESTful APIs.
- Excellent problem-solving skills and ability to work independently in a fast-paced environment.
- Strong communication skills and experience working in cross-functional teams.
- Ability to work on multiple projects while setting priorities to complete tasks within set timelines.
Preferred Qualifications:
- Experience testing AI-driven systems, agent-based architectures, or AI APIs from providers like OpenAI and Anthropic.
- Knowledge of testing vector databases, embeddings, or search-based AI systems.
- Experience with AI/ML frameworks (e.g., PyTorch, TensorFlow) for understanding model behavior.
- Familiarity with AI model evaluation techniques, including accuracy metrics, performance benchmarks, and bias testing.
- Experience creating effective end-to-end test plans for complex AI systems.
- Knowledge of NoSQL databases.
- Domain knowledge in legal, tax, or accounting.
- A portfolio of projects demonstrating expertise in testing AI/ML solutions and building quality frameworks for LLMs
#LI-VGA1
What’s in it For You?
Hybrid Work Model: We’ve adopted a flexible hybrid working environment (2-3 days a week in the office depending on the role) for our office-based roles while delivering a seamless experience that is digitally and physically connected.
Flexibility & Work-Life Balance: Flex My Way is a set of supportive workplace policies designed to help manage personal and professional responsibilities, whether caring for family, giving back to the community, or finding time to refresh and reset. This builds upon our flexible work arrangements, including work from anywhere for up to 8 weeks per year, empowering employees to achieve a better work-life balance.
Career Development and Growth: By fostering a culture of continuous learning and skill development, we prepare our talent to tackle tomorrow’s challenges and deliver real-world solutions. Our Grow My Way programming and skills-first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future.
Industry Competitive Benefits: We offer comprehensive benefit plans to include flexible vacation, two company-wide Mental Health Days off, access to the Headspace app, retirement savings, tuition reimbursement, employee incentive programs, and resources for mental, physical, and financial wellbeing.
Culture: Globally recognized, award-winning reputation for inclusion and belonging, flexibility, work-life balance, and more. We live by our values: Obsess over our Customers, Compete to Win, Challenge (Y)our Thinking, Act Fast / Learn Fast, and Stronger Together.
Social Impact: Make an impact in your community with our Social Impact Institute. We offer employees two paid volunteer days off annually and opportunities to get involved with pro-bono consulting projects and Environmental, Social, and Governance (ESG) initiatives.
Making a Real-World Impact: We are one of the few companies globally that helps its customers pursue justice, truth, and transparency. Together, with the professionals and institutions we serve, we help uphold the rule of law, turn the wheels of commerce, catch bad actors, report the facts, and provide trusted, unbiased information to people all over the world.
About Us
Thomson Reuters informs the way forward by bringing together the trusted content and technology that people and organizations need to make the right decisions. We serve professionals across legal, tax, accounting, compliance, government, and media. Our products combine highly specialized software and insights to empower professionals with the data, intelligence, and solutions needed to make informed decisions, and to help institutions in their pursuit of justice, truth, and transparency. Reuters, part of Thomson Reuters, is a world leading provider of trusted journalism and news.
We are powered by the talents of 26,000 employees across more than 70 countries, where everyone has a chance to contribute and grow professionally in flexible work environments. At a time when objectivity, accuracy, fairness, and transparency are under attack, we consider it our duty to pursue them. Sound exciting? Join us and help shape the industries that move society forward.
As a global business, we rely on the unique backgrounds, perspectives, and experiences of all employees to deliver on our business goals. To ensure we can do that, we seek talented, qualified employees in all our operations around the world regardless of race, color, sex/gender, including pregnancy, gender identity and expression, national origin, religion, sexual orientation, disability, age, marital status, citizen status, veteran status, or any other protected classification under applicable law. Thomson Reuters is proud to be an Equal Employment Opportunity Employer providing a drug-free workplace.
We also make reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs in accordance with applicable law. More information on requesting an accommodation here .
Learn more on how to protect yourself from fraudulent job postings here .
More information about Thomson Reuters can be found on thomsonreuters.com.
