Job Description:
About ETS:
ETS is a global education and talent solutions organization enabling lifelong learners worldwide to be future-ready. For more than 75 years, we've been advancing the science of measurement to build benchmarks for fair and valid skill assessment across cultures and borders. Our worldwide impact extends through our renowned assessments including TOEFL®, TOEIC®, GRE® and Praxis® tests, serving millions of learners in more than 200 countries and territories. Through strategic acquisitions, we've expanded our global capabilities: PSI strengthens our workforce assessment solutions, while Edusoft, Kira Talent, Pipplet, Vericant, and Wheebox enhance our educational technology and assessment platforms across critical markets worldwide.
Through ETS Research Institute and ETS Solutions, we're partnering with educational institutions, governments, and organizations globally to promote skill proficiency, empower upward mobility, and unlock opportunities for everyone, everywhere. With offices and partners across Asia, Europe, the Middle East, Africa, and the Americas, we deliver nearly 50 million tests annually. Join us in our journey of measuring progress to power human progress worldwide.
Position Summary
The Staff AI Quality Platform Engineer - DevOps, LLMOps & Automation Reliability is responsible for implementing, operationalizing, and supporting the technical platform capabilities that enable AI-first quality engineering across ETS products, platforms, and customer-facing delivery teams. This role helps move AI Quality Automation from proof of concept into reliable operational use by building the deployment, CI/CD, observability, environment, reliability, and support foundations needed for enterprise adoption.
This role works closely with the Principal AI Quality Architect and the Staff Agentic AI Engineer to turn architecture and agentic workflow designs into usable, supportable, and scalable solutions. While the Agentic AI Engineer focuses primarily on building AI agents, workflow logic, prompt patterns, and automation generation capabilities, this role ensures those capabilities can be deployed, executed, monitored, secured, maintained, and supported in real delivery environments.
The role includes hands-on engineering and development work, particularly where platform implementation, integration, automation reliability, pipeline execution, environment readiness, logging, monitoring, and support tooling are required. The role will help implement AI Quality Automation capabilities across Dify, AIQ, GitLab, Jira, Zephyr, Confluence, existing Selenium/Fusion automation assets, and other approved enterprise tools.
This position is critical because ETS is not operating in a clean-sheet environment. Existing Selenium/Fusion automation, AIQ automation, pipeline executions, regression assets, and application-specific dependencies must continue to support delivery while the organization transitions toward AI-first quality engineering. The Staff AI Quality Platform Engineer helps ensure current automation capabilities remain operational while enabling the future-state platform to mature safely and responsibly.
Primary Responsibilities
- Implement and operationalize AI Quality Automation platform capabilities for use by SQA teams, engineering teams, product teams, and other internal customer groups.
- Collaborate with the Principal AI Quality Architect to translate architecture, standards, governance expectations, and roadmap priorities into working platform capabilities.
- Collaborate with the Staff Agentic AI Engineer to deploy, integrate, and support agentic workflows, AI-generated automation outputs, tool integrations, and execution pipelines.
- Build and maintain CI/CD execution patterns that support AI-generated automation scripts, existing automation assets, and future-state quality automation workflows.
- Support integration across Dify, AIQ, GitLab, Jira, Zephyr, Confluence, Selenium/Fusion frameworks, and other approved quality engineering tools.
- Contribute hands-on development where needed to support platform implementation, API integration, automation utilities, pipeline enablement, workflow execution, reporting, and operational support.
- Establish reliable deployment, promotion, rollback, monitoring, and support patterns for AI Quality Automation workflows and related platform components.
- Build observability and reporting capabilities that help teams understand automation execution health, workflow failures, pipeline reliability, test execution outcomes, and platform usage.
- Support the transition of automation assets from legacy automation frameworks toward future-state AI Quality Automation patterns.
- Maintain current automation continuity by supporting active pipeline executions, identifying reliability risks, and helping resolve platform, environment, integration, and execution blockers.
- Partner with DevOps, cloud, infrastructure, InfoSec, application, vendor, and SQA domain teams to ensure platform capabilities are secure, stable, accessible, and ready for operational use.
- Help define and implement LLMOps practices for AI Quality Automation, including workflow versioning, prompt/configuration control, execution logging, model/provider configuration awareness, and operational traceability.
- Develop support runbooks, troubleshooting guides, onboarding materials, operational procedures, and knowledge-base content for teams using the AI Quality Automation platform.
- Monitor platform reliability and support trends, identifying recurring issues that should be addressed through improved automation, tooling, documentation, or architecture changes.
- Ensure that AI-generated quality artifacts and automation outputs can be executed, traced, reviewed, and supported in alignment with ETS quality, security, accessibility, and release-readiness expectations
#LI-AD1
Experience and Skills:
- Strong knowledge of DevOps, CI/CD, platform engineering, automation reliability, and operational support practices.
- Hands-on experience with Git-based development workflows, CI/CD pipelines, automated test execution, environment configuration, and release/deployment processes.
- Experience implementing or supporting quality automation platforms, test execution frameworks, or engineering productivity platforms.
- Working knowledge of AI/LLM platform operations, including workflow versioning, prompt/configuration management, logging, monitoring, traceability, and operational controls.
- Experience with tools and technologies such as GitLab, Jenkins or Azure DevOps, Jira, Zephyr, Confluence, Selenium, Playwright, REST APIs, LLM API specifications, cloud services, and scripting languages.
- Strong scripting or development skills in one or more languages such as Python, JavaScript/TypeScript, Java, Bash, or PowerShell.
- Ability to troubleshoot complex issues across automation code, CI/CD pipelines, test environments, cloud infrastructure, tool integrations, data dependencies, and execution platforms.
- Understanding test automation needs across UI, API, database, desktop, cloud, performance, security, accessibility, functional integration, and end-to-end testing.
- Ability to support both current-state automation platforms and future-state AI Quality Automation capabilities during a multi-platform transition.
- Strong understanding of operational readiness, supportability, observability, reliability, security, and maintainability as platform design principles.
- Strong communication and collaboration skills with the ability to work across architecture, engineering, SQA, DevOps, cloud, security, product, vendor, and customer-supporting teams.
- Ability to document platform patterns, support procedures, implementation guidance, and technical decisions clearly for distributed teams.
Education and Experience
Required
- Bachelor's degree in Computer Science, Engineering, Information Technology, Information Systems, or equivalent practical experience.
- 8+ years of experience in DevOps, platform engineering, SRE, test automation infrastructure, software engineering, cloud engineering, or related technical roles.
- Demonstrated experience implementing and supporting CI/CD pipelines, automation execution frameworks, source-code repositories, and operational platform processes.
- Hands-on experience troubleshooting automation, pipeline, environment, integration, access, infrastructure, or execution reliability issues.
- Experience working with cross-functional teams to implement platform capabilities that support software delivery, quality engineering, or release readiness.
Preferred
- Experience with AI/LLM platform operations, agentic workflow deployment, Dify, LangChain, LangGraph, Semantic Kernel, Azure AI, AWS Bedrock, or comparable AI orchestration platforms.
- Experience with Selenium/Fusion, Playwright, GitLab CI/CD, Jira, Zephyr, Confluence, AWS, BrowserStack, or similar enterprise quality engineering tools.
- Experience supporting AI-assisted test generation, automation script generation, execution feedback loops, or quality engineering platforms.
- Experience modernizing legacy automation platforms or supporting transition from one automation ecosystem to another.
- Experience designing operational support models, dashboards, runbooks, monitoring, and incident response processes for engineering platforms.
- Experience in complex enterprise environments with multiple applications, shared environments, vendor dependencies, and customer-facing delivery obligations.
ETS is mission driven and action oriented
-
We are passionate about hiring innovative thinkers who believe in the promise of education and lifelong learning.
-
We are energized by cultivating growth, innovation, and continuous transformation for the next generation of rising professionals as leaders. Â In support of this ETS offers multiple Business Resource Groups (BRG) for you to learn and advance your career growth!
-
As a not-for-profit organization we will encourage you to lean in to your passion for volunteering. Â At ETS you may qualify for up to an additional 8 hours of PTO for volunteer work on causes that are important to you!
-
The base salary range advertised represents the low and high end of the anticipated salary range for this position. The base pay actually offered will take into account internal equity and also may vary depending on the candidate's geographic region, job-related knowledge, skills, and experience among other factors. The base pay is only one aspect of the Total Rewards Package that will be offered to the successful candidate.
ETS is an Equal Opportunity Employer. We are committed to providing equal employment opportunities to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, marital status, veteran status, sexual orientation, gender identity, or any other characteristic protected by law. We believe in creating a work environment where all individuals are treated with respect and dignity.
From: ETS GCC
