DevOps/LLMOps Engineer (Lisbon)

Description

Summary: Join a Portuguese IT company focused on turning technology into real impact, optimizing organizational performance, and working closely with clients. Highlights: 1. Lead structured handover of AI solutions and develop in-house AI features. 2. Build and maintain CI/CD pipelines for AI solutions, applying LLMOps practices. 3. Act as L3 escalation for AI solutions and manage backlog with business owners. Since 2016, BITIL has been growing with a clear mission: turning technology into real impact. We are a Portuguese IT company that brings together multidisciplinary professionals with strong experience in business consulting, infrastructure management, systems and database administration, platform integration, Business Intelligence and application development. We work with leading clients across the financial, public and private sectors, always focused on delivering robust, secure and results‑oriented solutions with the goal of optimizing organizational performance. At BITIL, we believe technology only creates value when it is aligned with people and with the strategic goals of the business. That’s why we work closely with our clients and continuously invest in the growth of our teams. **What we're looking for** A **DevOps Engineer** to join a international project. **Working model:** Hybrid **English level:** B2 (mandatory) **Candidates must be living in Portugal** **Main Responsibilities** Solution Takeover \& Knowledge Transfer * Lead structured handover of AI solutions delivered by external vendors, across any in\-scope solution type (agents, copilots, RAG apps, document intelligence services, integrations, fine\-tuned LLMs, etc.). * Review code, prompts, configurations, architecture, and documentation; identify technical debt and define a remediation backlog. * Produce or refresh runbooks, SOPs, prompt and model documentation, and operational documentation for solutions taken into run mode. * In\-House AI Solution Development * Design, develop, and ship AI features and full solutions using authorized AI stack, selecting the most appropriate pattern for each business problem (rule\-based automation,RAG, agentic workflows, fine\-tuning, multimodal AI services, etc.). * Build solutions with Azure OpenAI, Microsoft Copilot Studio, M365 Copilot extensibility, Azure AI Foundry, Azure AI Services (vision, speech, translation, document intelligence), and custom Python/TypeScript services as appropriate. * Apply enterprise patterns for grounding, evaluation, guardrails, and responsible AI consistent with AI governance and Microsoft Copilot Studio governance frameworks. * Contribute reusable components (connectors, prompt templates, agent skills, evaluation harnesses) to the AI platform library. * Collaborate with the AI Solution Lead on architecture reviews, pattern selection, and design validation. DevOps \& Platform Operations (incl. LLMOps) * Build and maintain CI/CD pipelines (GitHub Actions / Azure DevOps) for AI solutions, following GitOps and IaC standards. * Apply LLMOps practices: prompt versioning, evaluation datasets, A/B testing of prompts and configurations, automated quality evaluation, and regression testing. * Implement observability (logs, traces, cost, usage, response quality, grounding quality) and act on alerts. * Manage application lifecycle, secrets, environment promotion, and release management. * Optimize consumption costs (tokens, compute, storage, inference calls) and report on unit economics per solution. User Support (L3\) * Act as the L3 escalation point for in\-scope AI solutions; partner with L1/L2 team per the existing service model. * Investigate and resolve incidents, problems, and quality issues (response quality, hallucinations, grounding failures, integration errors); drive permanent fixes back into the codebase, prompts, or configurations. * Maintain a knowledge base of known issues, workarounds, and FAQs for end users and support partners. Backlog Management \& Business Engagement * Run discovery and refinement sessions with business product owners; translate user needs into well\-formed user stories with acceptance criteria. * Triage, estimate, and prioritize the backlog together with the business; plan iterations and communicate releases. * Demo increments, collect feedback, and steward adoption together with business champions. GitHub Copilot Enablement (Secondary) * Act as a subject\-matter expert for GitHub Copilot, authorized AI coding platform. * Provide best\-practice guidance, internal enablement sessions, and prompt/usage patterns for other ADS developers. * Monitor adoption signals and feed insights back to the AI governance team. **REQUIREMENTS** **Experience** * 5–8 years of professional software development experience. * At least 2 years building or operating generative AI / LLM\-based solutions in production. * Demonstrated breadth across multiple GenAI solution types (e.g., conversational, agentic, RAG, document intelligence, integrations). * Proven track record of taking over and stabilizing vendor\-built code bases. * Hands\-on experience with backlog management and direct business stakeholder engagement. **Technical Skills (Must\-have)** * Strong Python and/or TypeScript; solid software engineering fundamentals (testing, version control, code review). * Azure cloud (App Service / Container Apps / Functions / AKS), Azure OpenAI, Azure AI Foundry, Microsoft Graph, REST API integration. * Practical experience with at least three GenAI solution patterns from the following: conversational agents, agentic/multi\-agent workflows, RAG, document intelligence, AI\-driven integrations. * Prompt engineering, evaluation, and guardrail design for LLM\-based applications. * CI/CD pipelines, IaC (Bicep / Terraform), containerization (Docker), observability tooling. * Microsoft 365 ecosystem awareness (Copilot Studio, SharePoint, Teams as data sources and surfaces). **Technical Skills (Nice\-to\-have)** * Agentic frameworks (Microsoft Agent Framework, LangGraph, Pydantic AI etc.). * Vector stores and knowledge\-graph tooling (Azure AI Search, Qdrant, Neo4j, LlamaIndex etc.). * Azure AI Services (Document Intelligence, Vision, Speech, Translator) for multimodal application development. * Fine\-tuning and model customization on Azure OpenAI / Azure AI Foundry. * GitHub Copilot enablement experience. * Azure AI Engineer (AI\-102\) certification. **Technical Skills (Must\-have)** * Strong Python and/or TypeScript; solid software engineering fundamentals (testing, version control, code review). * Azure cloud (App Service / Container Apps / Functions / AKS), Azure OpenAI, Azure AI Foundry, Microsoft Graph, REST API integration. * Practical experience with at least three GenAI solution patterns from the following: conversational agents, agentic/multi\-agent workflows, RAG, document intelligence, AI\-driven integrations. * Prompt engineering, evaluation, and guardrail design for LLM\-based applications. * CI/CD pipelines, IaC (Bicep / Terraform), containerization (Docker), observability tooling. * Microsoft 365 ecosystem awareness (Copilot Studio, SharePoint, Teams as data sources and surfaces). **Technical Skills (Nice\-to\-have)** * Agentic frameworks (Microsoft Agent Framework, LangGraph, Pydantic AI etc.). * Vector stores and knowledge\-graph tooling (Azure AI Search, Qdrant, Neo4j, LlamaIndex etc.). * Azure AI Services (Document Intelligence, Vision, Speech, Translator) for multimodal application development. * Fine\-tuning and model customization on Azure OpenAI / Azure AI Foundry. * GitHub Copilot enablement experience. * Azure AI Engineer (AI\-102\) certification. **Education** * Bachelor's or Master's degree in Computer Science, Software Engineering, or equivalent. **Behavioral Competencies** * Outcome\-oriented, pragmatic, comfortable balancing build versus run. * Strong written and verbal English communication; able to translate between business and technical audiences. * Continuous improvement mindset; operates well in a hybrid global team with vendors and internal stakeholders. * Curiosity and self\-learning attitude in a rapidly evolving AI landscape; able to pick the right tool/pattern for each problem rather than defaulting to one approach * Bachelor's or Master's degree in Computer Science, Software Engineering, or equivalent. **Behavioral Competencies** * Outcome\-oriented, pragmatic, comfortable balancing build versus run. * Strong written and verbal English communication; able to translate between business and technical audiences. * Continuous improvement mindset; operates well in a hybrid global team with vendors and internal stakeholders. * Curiosity and self\-learning attitude in a rapidly evolving AI landscape; able to pick the right tool/pattern for each problem rather than defaulting to one approach Localização do trabalho: Trabalho remoto híbrido em Lisboa