QA Engineer — AI Evaluation & Automation

We’re looking for a quality-focused engineer who combines a solid manual testing foundation with hands-on automation experience and a growing interest in AI evaluation. You care deeply about how systems behave, you’re not satisfied with surface-level testing, and you’re excited about what modern QA looks like in an AI-driven world.

What you’ll do

• Design and execute evaluation strategies for AI-powered features, including LLM-based flows, recommendation engines, and decision logic.

• Validate AI outputs for correctness, consistency, bias, hallucination risk, and edge cases — defining clear qualitative and quantitative acceptance criteria.

• Perform deep manual, exploratory, and scenario-based testing across the product, from AI-driven flows to core application functionality.

• Build and maintain automated UI and API test suites, and keep them running reliably in CI/CD.

• Build and maintain automated evaluation and regression pipelines for AI-enabled systems.

• Identify non-determinism and reliability risks specific to AI systems and propose practical mitigations.

• Partner closely with engineers and product teams to improve testability, observability, and overall release quality.

• Performance, load, and security testing.

What we’re looking for

• Sharp analytical thinking and attention to detail — you notice what others miss.

• 4+ years of hands-on manual testing experience, including exploratory testing, test planning, and writing acceptance criteria for complex or enterprise systems.

• 4 +years of test automation experience using frameworks such as Playwright, Cypress, or Selenium — covering UI and/or API layers.

• Confident writing and maintaining test code in TypeScript/JavaScript, Python, Java.

• Demonstrated ability to own a full test strategy, not just execute test cases.

• Strong understanding of AI system behavior, including non-determinism, prompt sensitivity, and the unique challenges of validating model outputs.

• Reads backend code comfortably to debug test failures and understand system behavior.

Nice to have

• Experience with AI evaluation frameworks, prompt testing, or model validation techniques.

• Familiarity with agentic QA concepts and AI-assisted testing workflows.

• Hands-on SDET experience — from framework design to internal tooling.

• Exposure to observability tooling and how it supports quality in production systems.

What We Offer

• Competitive compensation and benefits package.

• Remote-first work with a flexible schedule.

• Opportunities for professional growth, training, and certifications.

• A dynamic environment where innovation, security, operational excellence, and cutting-edge ML technologies are highly valued.

• Projects that span cutting-edge AI, UX, fintech, and healthtech domains.