Windmill is a boutique digital product delivery company. Our team of designers, strategists, and engineers love to create great experiences. We design and develop delightful and functional digital products that solve tough problems and enable new opportunities for enterprises in complex industries such as banking & finance, healthcare, and compliance.
3 серпня 2026

Senior AI Verification Engineer (Full-Stack) (вакансія неактивна)

Ukraine, віддалено

We are looking for a senior full-stack engineer who owns quality and verification in an AI-native delivery environment — combining deep engineering judgment with AI tools to review, validate, and test code and AI-driven behavior across the full stack, from generated code to production output.

Key Responsibilities

  • Review and verify code produced by AI coding agents — assessing correctness, architecture, performance, and security to the standard of a senior engineer, not surface-level testing.
  • Own quality end-to-end before releases ship: deep manual, exploratory, and scenario-based testing across both AI and non-AI features, plus fix verification and regression prevention.
  • Evaluate AI-powered features — LLM-based flows, recommendation engines, decision logic — validating outputs for correctness, consistency, bias, hallucination risk, and edge cases, with clear qualitative and quantitative acceptance criteria.
  • Identify non-determinism, prompt sensitivity, and reliability risks specific to AI systems, and propose practical mitigations.
  • Build and maintain automated evaluation and regression pipelines for AI-enabled systems — using AI agents (Cursor, Claude Code) in your own verification workflow.
  • Partner with engineers and product to improve testability, observability, and overall release quality, and to raise the bar on AI-assisted delivery practices.

Required Skills

  • AI-Native Development: Cursor or Claude Code (or equivalent agentic tools); prompt engineering, applied to both building and verifying
  • Engineering depth: 8+ years across the full stack, strong enough to review and validate another engineer’s (or AI’s) code
  • Front-End: React or Angular, with TypeScript / JavaScript
  • Back-End: Java, Node.js, or Ruby
  • AI system behavior: solid understanding of non-determinism, prompt sensitivity, and the challenges of validating model outputs
  • Test automation: Playwright, Cypress, Selenium, or comparable (UI and/or API)
  • Data: SQL and NoSQL (e.g. MongoDB)
  • Infrastructure: Docker, CI/CD, and a major cloud (AWS, Azure, or GCP)
  • Proven ability to own a full verification and test strategy, not just execute test cases
  • Upper-intermediate English (B2+)

Nice to Have

  • AI evaluation frameworks, prompt testing, or model validation techniques
  • Agentic QA concepts and AI-assisted testing workflows
  • Observability tooling and its role in production quality
  • Python (FastAPI / Django), Spring Boot, Next.js
  • Kubernetes, serverless & event-driven architectures
  • High-growth startup or product-led company background

Personal Attributes

  • Sharp analytical thinking and attention to detail — notices what others miss
  • Autonomous & proactive — owns quality without being told what to check
  • Strong engineering judgment and a reviewer’s mindset
  • Strong team player
  • Excellent communication
  • Eager to learn & adapt fast

What We Offer

  • Competitive compensation and benefits package.
  • Remote-first work with a flexible schedule.
  • Opportunities for professional growth, training, and certifications.
  • A dynamic environment where innovation, security, operational excellence, and cutting-edge ML technologies are highly valued.
  • Projects that span cutting-edge AI, UX, fintech, and healthtech domains.