Engaige Technologies B.V. is a Dutch company building an AI customer-service platform for e-commerce. Millions of customer cases flow through our infrastructure every year, across more than 30 countries. Our agents answer customer questions and take actions on the merchant's behalf, such as looking up orders, creating returns, paybacks, changing subscriptions, etc. through 50+ integrations such as Shopify, Zendesk, Gorgias, Intercom, Klaviyo, PostNL and Sendcloud.
About the team
The engineering team consists of 4 engineers. You will work closely with the Head of Product and the CEO, and engineers own their work from initial thought through production.
About the role
As a Senior AI Engineer, you will be responsible for the quality and reliability of the platform and agents that handle these customer cases. This includes how we evaluate them, how we retrieve and classify information, how agents follow merchant policies, and how changes reach production safely.
A wrong promise or a wrong action toward a customer is a refund and a merchant at risk, so measurement, precision and safe rollout are part of the job. The core of this role is classical production ML depth: you have trained, evaluated and shipped models before LLMs became widely available, and you know how models fail once they meet real data.
Responsibilities
Own the evaluation methodology for our agents, including our benchmark that replays real production conversations with simulated tools, and LLM judges that check for unsupported promises, customer language and correct outcomes
Design offline evaluations that predict online behavior, and close the gap between benchmark scores and production results
Build live guardrails, monitoring and staged rollout for model, prompt and policy changes across multiple model providers
Improve retrieval and classification, including RAG with vector search and reranking, and embedding-based contact-reason classification
Develop the policy graphs that let agents follow merchant policies step by step instead of relying on a single large prompt
Extend our AI manager, which investigates conversations, identifies knowledge and policy gaps, and runs weekly improvement rounds
Partner with the Head of Product and the CEO to set technical standard for engineering and evaluation across the team
You may be a good fit if you
Have trained and shipped machine learning models to production before 2022
Have a working understanding of offline and online evaluation, A/B testing, data leakage, training/serving skew, and working with noisy or delayed labels
Have hands-on experience with ranking, recommendation or classification systems
Have built monitoring for model quality and drift, and rolled out model changes safely
Are proficient in Python and comfortable owning backend production code
Have high agency and low ego, prefer shipping over planning, and are comfortable working on problems that are not yet fully defined
Strong candidates may also have
Experience with LLMs in production, such as tool calling, structured output, RAG or LLM-as-judge evaluation
Experience with search, retrieval or agent systems
Experience with Weaviate, Langfuse, OpenTelemetry or Google Cloud Platform
Experience with E-commerce or building SaaS tools.
Our stack
Claude, OpenAI and open-weight models (Kimi, GLM, DeepSeek, Qwen), Weaviate, Python (FastAPI), Native Postgres, Cloud Tasks, Cloud Run and Terraform
Location
This role is ideally based full-time in our office in Utrecht. Hybrid from Utrecht or fully remote is possible. Valencia is also an option. We hire within Europe only.
Compensation
We offer a very generous salary and stock options. For the right candidate, this role has a possible growth path toward CTO.
Senior Ai engineer in Utrecht werkgever: Engaige
Engaige is een uitstekende werkgever die een stimulerende en informele werkomgeving biedt in het hart van Utrecht. Met een focus op persoonlijke groei en ontwikkeling, bieden we een duidelijke groeipad naar de rol van Account Executive, ondersteund door actieve coaching. Werken bij Engaige betekent deel uitmaken van een snelgroeiend AI-bedrijf dat innovatieve oplossingen biedt voor e-commerce, met aantrekkelijke beloningsstructuren en de kans om impact te maken in een dynamische sector.
