Artificial Intelligence Operations Engineer

AI Ops Engineer

AI Ops Engineer1

This role requires strong prompt engineering capabilities, the ability to triage AI pipeline issues, and a deep understanding of how LLM-based agents interact with tools, memory, and APIs. You will be expected to diagnose and remediate real-time problems, from prompt quality issues to model behavior anomalies.

Key Responsibilities

  Design, fine-tune, and manage prompts for various LLM use cases tailored to customer's enterprise operations.

  Operate, monitor, and troubleshoot agentic AI applications, including identifying whether issues stem from:

• Prompt quality or structure

• Model configuration or performance

• Tool usage, API failures, or memory/recall issues

  Build diagnostics and playbooks to triage LLM-driven failures, including handling fallback strategies, retries, or re-routing to human workflows.

  Collaborate with architects, ML engineers, and DevOps to optimize agent orchestration across platforms like LangGraph, CrewAI, AutoGen, or similar.

  Support integration of agentic systems with enterprise apps like Jira, ServiceNow, Glean, or Confluence using REST APIs, webhooks, and adapters.

  Implement observability and logging best practices for model outputs, latency, and agent performance metrics.

  Contribute to building self-healing mechanisms and alerting strategies for production-grade AI workflows.

Required Qualifications

• 3–6 years of experience in software engineering, DevOps, or ML Ops with exposure to AI/LLM workflows.

• Strong foundation in prompt engineering and experience with LLMs like GPT, Claude, LLaMA, etc.

• Practical understanding of AIOps platforms or operational AI use cases (incident triage, log summarization, root cause analysis, etc.).

• Exposure to agentic AI architectures, such as LangGraph, AutoGen, CrewAI, etc.

• Familiarity with scripting (Python), RESTful APIs, and basic system debugging.

• Strong analytical skills and the ability to trace issues across multi-step pipelines and asynchronous agents.

Good-To-Have

• Glean

• DevRev 

• Codium 

• Cursor 

• Atlassian AI 

• Databricks Mosaic AI

Experience Level

Senior Level