AI Operations Engineer

Description

Summary: Join a team responsible for designing, building, and operating end-to-end infrastructure for production AI systems, focusing on reliability and continuous improvement of ML and LLM-based solutions. Highlights: 1. Design and implement scalable ingestion pipelines for ML and LLM data. 2. Build and maintain data processing workflows and define storage strategies. 3. Develop observability and debugging tools for production AI systems. Lisbon, Portugal * Full\-time * Application and Product Development * Hybrid * Tieto Group Job Description Tieto Iberia is part of the Tieto Group and combines local expertise with the global strength of Tieto, a leading software and digital engineering services company with worldwide presence and capabilities. As part of our continued expansion in AI\-driven platforms and data\-intensive systems, we are looking for a **Machine Learning Operations/ Language Model** **Operations Engineer** to join our advanced engineering team. In this role, you will be responsible for designing, building, and operating the end\-to\-end infrastructure that supports production AI systems, ensuring reliability, observability, data quality, and continuous improvement of machine learning and LLM\-based solutions. Key Responsibilities * Design and implement scalable, source\-agnostic ingestion pipelines for production ML and LLM data. * Build and maintain data processing workflows including ingestion, redaction, storage, classification, and slicing of production signals. * Define and implement data storage strategies, including tiering, retention policies, and privacy\-aware data handling. * Develop observability and debugging tools, including dashboards and query systems for production AI systems. * Implement evaluation and monitoring frameworks, including offline evaluation sets, online autoraters, and regression detection systems. * Build automated triage systems to identify, classify, and surface production failures. * Develop and maintain PII redaction mechanisms and enforce data governance and compliance policies at ingestion level. * Design and operate LLM evaluation mining workflows to continuously improve model and prompt performance. * Implement alerting systems to detect regressions across model and prompt deployments. * Collaborate with AI, engineering, and product teams to ensure robustness and reliability of production AI systems. * Evaluate and select tooling, infrastructure, and hosting strategies for ML/LLM platforms. * Own the operational reliability of the entire ML/LLM data and evaluation pipeline. Requirements * Proven experience building and operating production\-grade data platforms or ML systems, including ingestion, storage, access control, monitoring, and on\-call responsibilities. * Hands\-on experience developing ML/LLM evaluation systems (e.g., regression test sets, autoraters, LLM\-as\-a\-judge frameworks, or golden datasets). * Strong understanding of LLM observability, tracing, and debugging tools. * Experience implementing data privacy controls such as PII redaction in production environments. * Deep understanding of failure modes in ML and LLM systems (hallucinations, retrieval failures, agent loops, ASR/TTS degradation, prompt/model regressions). * Strong production\-level Python engineering skills, with a hands\-on mindset. * Solid understanding of data pipelines, system reliability, and distributed data processing. Nice\-to\-Have Skills * Experience in multi\-tenant or SaaS architectures with strict data isolation requirements. * Familiarity with Azure and/or AWS cloud ecosystems. * Experience making infrastructure trade\-off decisions between managed services and self\-hosted solutions. * Knowledge of vector databases, embedding techniques, and clustering or unsupervised failure detection methods. * Experience with data versioning tools such as LakeFS, DVC, or Delta Lake. * Familiarity with GDPR, data deletion workflows, and compliance\-driven data systems. * Exposure to embedded, automotive, or constrained environments. * Experience with non\-English language model evaluation or multilingual datasets. * Experience working with LLM APIs such as Anthropic Claude, OpenAI models, or open\-source alternatives. * Familiarity with CI/CD workflows, GitHub\-based development, and modern DevOps practices. * Experience with dashboard development using TypeScript or similar frontend technologies. Additional Information At Tieto, we believe in the power of diversity, equity, and inclusion. We encourage applicants of all backgrounds, genders (m/f/d), and walks of life to join our team, as we believe that this fosters an inspiring workplace and fuels innovation. Our commitment to openness, trust, and diversity is at the heart of our mission to create digital futures that benefit businesses, societies, and humanity. Diversity, equity and inclusion \| Sustainability \| Tieto We provide customers across different industries with mission\-critical solutions through our specialized software businesses Tieto Caretech, Tieto Banktech and Tieto Indtech as well as Tieto Tech Consulting business. Our around 14 000 talented vertical software, design, cloud and AI experts are dedicated to empowering our customers to succeed and innovate with latest technology. Tieto’s annual revenue is approximately EUR 2 billion. The company’s shares are listed on the NASDAQ exchange in Helsinki and Stockholm, as well as on Oslo Børs.