We are looking for a Senior Automation QA Engineer to own end-to-end test automation across API and UI layers for our AI-native platform, including agent workflow validation.
-
Design and execute test strategies for API and UI layers using Pytest and Playwright (Angular).
-
Own CI/CD quality gates in Azure DevOps, ensuring reliable build promotion and release readiness.
-
Build and maintain coverage for agent workflow testing, including non-determinism, agent graph validation, and confidence score assertions.
-
Develop reusable automation frameworks and data setup utilities backed by PostgreSQL.
-
Contribute to LLM evaluation and observability workflows using Langfuse or similar tooling.
-
6+ years of experience in QA automation.
-
Strong hands-on expertise with Pytest and Playwright, with proven depth in both API and UI automation.
-
Solid proficiency in Python and experience working with PostgreSQL for test data and validation.
-
Hands-on experience with Azure DevOps CI/CD pipelines and quality gates.
-
Awareness of LLM/agent testing challenges (non-determinism, agent graph validation, confidence score assertions); exposure to Langfuse or LLM evaluation frameworks is a plus and can be built on the job.
