MLOps Engineer Job Description: Duties, Skills and Template

A fraud model that scored well in a notebook can start missing cases a few months after launch, because customer behaviour moved and nobody was watching. An MLOps engineer builds the machinery that catches that: pipelines that retrain and test models, a registry that records exactly what is live, monitoring that spots drift, and the GPU and cloud infrastructure that keeps serving costs under control. In many companies the same person is called an AI infrastructure engineer.

Below is an MLOps engineer job description you can copy, adapted for both classic machine learning and LLM workloads, followed by how the role shifts with seniority, what to test, what commonly goes wrong in these ads and which hiring route fits. We also explain how the job differs from an ML engineer, a DevOps engineer and a data engineer.

MLOps engineer job description template

Choose whether your models are mostly classic ML, deep learning or LLM-based, then edit the brackets to suit. The infrastructure differs a great deal between those three, particularly on GPUs.

About the role

[Company] is hiring an MLOps Engineer to take our models from experiment to production and keep them healthy. You will build CI/CD for ML, run our model registry and feature pipelines, and manage training and serving infrastructure on [Amazon SageMaker AI / Gemini Enterprise Agent Platform, formerly Vertex AI / Azure Machine Learning / Kubernetes]. You will work with data scientists and ML engineers, report to [manager title] and be based in [city / remote].

Responsibilities

• Build CI/CD pipelines for ML that test code, data and model quality before promotion.
• Run experiment tracking and a model registry with MLflow, so every live model traces back to its code, data and parameters.
• Orchestrate training and batch inference with [Kubeflow / SageMaker Pipelines / Airflow].
• Package models in Docker and deploy them to Kubernetes or managed endpoints with canary or shadow releases.
• Operate a feature store, or an equivalent approach that keeps training and serving features consistent.
• Monitor latency, throughput, data drift and prediction quality, with alerts that trigger review or retraining.
• Manage GPU capacity, scheduling and autoscaling, and reduce cost through batching, spot capacity and right-sizing.
• For LLM workloads, run model gateways or inference servers, and track token usage, latency and evaluation results.
• Control access to models, data and secrets in line with [Company] security and privacy policy.
• Give data scientists templates and documentation so they can ship without rebuilding the path each time.

Requirements

• [X] years in MLOps, ML engineering, or DevOps with sustained ML exposure.
• Strong Python, including packaging and testing production code.
• Hands-on Docker and Kubernetes.
• Production use of MLflow, Kubeflow, SageMaker or an equivalent platform.
• Experience deploying and monitoring models that served real traffic.
• Working knowledge of how models are trained and evaluated, enough to talk to data scientists in their own terms.
• Infrastructure as code and CI/CD experience.

Nice to have

• GPU operations on Kubernetes, including device plugins and sharing strategies.
• LLM serving experience with an inference server such as vLLM.
• Exposure to distributed training with PyTorch.
• Experience with a feature store such as Feast or SageMaker Feature Store.

What we offer

• [Package details].
• [Mode of work] and [on-call for production models, if applicable].
• Access to [GPU environments and conference budget].

How the MLOps engineer role changes by level

Junior MLOps engineer

A junior usually comes from DevOps, software or data work. They maintain pipelines, containerise models built by others, respond to failed training runs and improve dashboards. Look for sound Python and Docker skills, plus a project where they deployed a model and kept it running.

Mid-level MLOps engineer

This engineer owns the journey for several models, from registry to serving, and picks release strategies and monitoring thresholds with the data science team. Ask for production deployments, experience with drift and failed rollouts, and Kubernetes confidence.

Senior MLOps or AI infrastructure engineer

Seniors design the ML platform: how teams share GPUs, how features are governed, when to use a managed service and when to self-host. On generative AI work they decide about serving stacks, batching and caching, and they set the cost model. Their influence on data scientists’ daily workflow is a big part of the job.

ML platform lead

Larger organisations may need one person to run ML platform strategy and a small team. Where the focus turns to model selection and overall solution design, an AI architect is a closer match, so write the post accordingly.

MLOps engineer skills that matter most, and how to check them

• Serving under load. Ask how they would serve a model with a strict latency target on limited GPUs. Listen for dynamic batching, autoscaling signals, cold start time and a fallback plan.
• Drift and monitoring. Ask how they would find out a model is degrading when true labels arrive weeks later. Good answers use input drift, prediction distribution and proxy metrics, and they distinguish data drift from concept drift.
• Reproducibility. Ask what would be needed to rebuild the exact model that served customers last quarter. Versioned data, code, parameters, environment and registry records should all come up.
• Training and serving skew. Describe a model that does well offline and badly online. A capable engineer looks at feature computation differences, time leakage and stale features first.
• Cost control. Ask how they would cut a GPU bill that keeps growing. Look for utilisation data, checkpointing so training can use spot capacity, smaller or quantised models and scale-to-zero for quiet services.

For a practical exercise, hand over a notebook and a trained model and ask for a containerised service with a health check, a simple test and a monitoring plan. How they structure that small project tells you more than their knowledge of any one platform.

Common MLOps engineer job description mistakes

• Asking for a data scientist who also runs Kubernetes. Researching models and running production platforms are distinct strengths. Fix: decide which you need, and describe modelling only as context for the role.
• Calling it DevOps with an ML keyword. Without model registries, drift or retraining in the duties, ML-aware engineers will skip the post. Fix: describe the model lifecycle work directly.
• Treating classic ML and LLM operations as identical. Tabular models on CPUs and large language models on GPUs raise very different problems. Fix: state what you actually serve, and at roughly what traffic.
• Leaving out platform choices. A SageMaker shop and a self-managed Kubeflow team need different people. Fix: name what you run, and say whether changing it is on the table.
• Requiring long tenure with new tools. Many LLM serving tools are young, so few candidates can show lengthy experience with them. Fix: ask for infrastructure depth and evidence of learning new stacks quickly.

Contract, contract-to-hire or permanent: which suits an MLOps engineer

A contract engineer is a good match for bounded work such as standing up a first model registry and deployment path, moving serving onto Kubernetes, or cutting GPU spend on a live service. Ask for templates and documentation your data scientists can use on their own, otherwise the improvement fades once the contract finishes.

Contract-to-hire makes sense when you are unsure how much MLOps your team needs, or when you want to see how someone works with data scientists who have strong opinions about their tools. That working relationship is the main thing an interview cannot show.

Permanent hiring is best when models are core to your product and the platform will keep growing. Zilo AI can run any of these: we hire MLOps engineers in India on contract, contract-to-hire or permanent terms, including AI infrastructure engineers. To staff the rest of an AI team, including GenAI and AI/ML engineers, start at the IT hiring hub.

Frequently asked questions

What should an MLOps engineer job description include?

Describe the models you run, your serving platform and cloud, your current registry and monitoring tools, and the state of your ML pipelines. Say whether LLM or GPU work is part of the job, since that strongly shapes who should apply.

What is the difference between an MLOps engineer and an ML engineer?

An ML engineer builds and improves models and their feature pipelines. An MLOps engineer builds the systems that deploy, version, monitor and scale those models. The roles overlap in small teams, but the first is judged on model performance and the second on reliable delivery.

How does MLOps differ from DevOps?

MLOps uses many DevOps practices but adds things unique to ML. Models depend on data as well as code, they degrade without any code change, and they often need GPUs. A DevOps engineer can move into MLOps, provided they learn how models are trained and evaluated.

Is an AI infrastructure engineer the same as an MLOps engineer?

Mostly, and many companies use the titles interchangeably. AI infrastructure usually leans towards GPU clusters, large-scale training and LLM serving, while MLOps often covers the wider model lifecycle. Use the title that matches your main problem.

Would you rather meet MLOps candidates than write this post yourself? Zilo AI can run the search around your stack. Contact the team and describe the models you serve and the platform you run them on.