Senior AI Systems Engineer

About the role

Infrrd, an enterprise AI company, is looking for a Senior AI Systems Engineer to build automated systems that measure, diagnose, and improve document extraction and classification accuracy at scale.

Key responsibilities

• Design and implement agentic evaluation pipelines that automate error detection, root‑cause analysis, hypothesis generation, prompt testing, A/B measurement and production promotion.
• Own and extend the accuracy measurement infrastructure, including error‑analysis, data‑quality pipelines and batch evaluation frameworks across document types and customer configurations.
• Develop internal tooling to transform manual utilities into automated improvement platforms for classification and extraction correction, NTP rule generation and performance reporting.
• Productionise prototype methodologies into reliable, scalable systems and build LLM‑based extraction and classification pipelines using few‑shot and Retrieval‑Augmented Generation strategies.
• Maintain A/B testing infrastructure for prompt and model changes, create live dashboards tracking extraction accuracy, NTP rates and false‑positive rates.
• Optimize LLM costs through prompt compression, token minimisation, model selection and migration strategies.
• Write production‑grade data pipelines with robust error handling, retries, logging and monitoring.
• Collaborate with platform engineering and applied research teams on architecture and methodology translation.
• Mentor junior engineers and provide documentation for independent operation.

Required profile

• Bachelor’s or Master’s degree in Computer Science, AI/ML, Computational Data Science, Computer Science & Automation or a related discipline.
• 8–10 years of total experience, including at least 4–6 years building production LLM or AI systems and 4–6 years in evaluation, quality measurement or accuracy improvement.

Required skills

• Production‑grade Python development (clean, tested, maintainable code).
• Testing frameworks such as pytest.
• Web service frameworks like FastAPI.