A support assistant worked well in the demo, then the model provider released a new version and refund answers quietly got worse. Nobody noticed for a fortnight, because nobody had a set of test cases to rerun. A prompt engineer exists to prevent that: they write the system instructions and examples that shape a model’s output, and, far more than the title suggests, they build the evaluation sets that show whether a change made things better or worse.
Before you post this role, check that it needs to exist. In many companies, prompt engineer is no longer a separate job. The work has been folded into AI engineering, and product teams often expect a generative AI engineer to do it alongside retrieval, tool calling and deployment. A standalone role still makes sense where output quality is the main problem and engineers are already in place. This page gives you a template, explains how levels differ, shows what to test, lists mistakes to avoid and covers hiring terms.
Prompt engineer job description template
Decide first whether this is a separate job or part of an engineering role. If the person will also build retrieval or application code, use “AI Engineer” or “Generative AI Engineer” as the title, and treat this template as one section of that ad.
About the role
[Company] is hiring a Prompt Engineer to improve the quality of [product or internal assistant], built on [Azure OpenAI / Amazon Bedrock / Google Cloud / your model provider]. You will design and test prompts, and own the evaluation sets that tell us whether the system is good enough to ship. You will work with [AI engineers / product managers / domain experts], be based in [city / remote] and report to [AI lead or product manager].
Responsibilities
- Write and refine system prompts, few-shot examples and output formats for [use cases such as classification, extraction, summarisation or support].
- Build and maintain evaluation sets from real user inputs, including difficult and edge cases.
- Define with domain experts what a correct, acceptable and harmful answer looks like, and write that down as scoring criteria.
- Run automated evaluations before every prompt or model change, and report the results in plain language.
- Review samples of production output, label failures and turn each failure pattern into a new test case.
- Specify structured outputs and tool descriptions so downstream code can rely on what the model returns.
- Version prompts in [Git / your prompt management tool], with a review step and a way to roll back.
- Test against prompt injection, unsafe requests and attempts to extract private data, working with [security or governance owner].
- Compare models and prompt lengths on quality, latency and token cost.
Requirements
- [X] years of work with LLM applications in production, or equivalent evaluated project work you can show.
- Clear, precise writing, and the ability to explain why a prompt failed.
- Good working Python for running evaluations, calling model APIs and analysing results.
- Hands-on experience with evaluation methods, including test sets, rubrics and regression checks.
- Understanding of how models behave, including tokens, context limits, temperature, and non-deterministic output.
- Experience with structured outputs and few-shot prompting.
- Comfort working with [legal / support / finance / your domain] specialists who judge quality.
Nice to have
- Experience with RAG systems, and diagnosing whether a bad answer came from retrieval or the prompt.
- Familiarity with LangChain or similar frameworks.
- Experience writing prompts in Indian languages for [Hindi / Kannada / Tamil / your users].
- Experience with human review workflows or labelled data.
What we offer
- [Salary range] and [benefits].
- [Work mode] and [model and tool access].
- [Budget for evaluation runs and experiments].
How the prompt engineer role changes by level
Entry-level prompt engineer
True entry-level roles are rare, and they are often review or labelling work under a more senior person. A beginner runs evaluations, labels outputs and tunes prompts inside a set process. Ask for excellent writing, careful judgement and some Python, plus a small evaluated project.
Mid-level prompt engineer
This person owns one or two use cases. They build the test set, set the pass criteria with domain experts, manage prompt versions and investigate production failures. Ask for an example where the numbers went the wrong way and what they changed.
Senior prompt engineer or AI quality lead
A senior sets how evaluation works across the company. They create shared standards, decide when a system is ready for release, plan for model upgrades and advise engineers on system design. Many people at this level hold an AI engineer title, and that is usually the honest way to advertise the job.
Prompt engineer skills that matter most, and how to check them
- Evaluation thinking. Give them twenty model outputs for a support task and ask them to design a scoring rubric. This is the single most useful test, since prompt craft without measurement is guesswork. See whether they split criteria, catch ambiguity and suggest checking agreement between two reviewers.
- Diagnosis. Show a failing example and ask for the cause before any fix. Strong candidates test whether the issue is the prompt, the retrieved context, the model or the data itself, and resist rewriting everything.
- Prompt writing. Set an extraction task with a tricky schema and a messy document. Check the result on inputs they have not seen, because a prompt tuned to three examples often fails on the fourth.
- Regression awareness. Ask what they would do when a provider changes a model. Look for a rerun of the full evaluation set, a side-by-side review and a staged rollout.
- Security. Ask how text inside a user’s uploaded document could change the assistant’s behaviour. Candidates should understand prompt injection and know that instructions alone will not fully stop it, so controls around tools and data are needed as well.
- Engineering level. Ask for a small script that runs a prompt across a dataset and scores results. It shows quickly whether they can work independently.
Common prompt engineer job description mistakes
- Hiring for a title when you need an engineer. If nobody has built the application, a prompt specialist alone will stall. Fix: hire a generative AI engineer first, and add prompt work to that job.
- Leaving out evaluation. An ad that only mentions writing prompts attracts people who collect clever tricks. Fix: make test sets, scoring and regression checks a large part of the responsibilities.
- Describing a content writer. Some ads really want someone to use chat tools for marketing copy. Fix: if it is content work, call it content work, and do not ask for this title.
- Asking for impossible experience. Long tenure on a very young discipline only rewards inflated CVs. Fix: ask for evaluated projects and production examples, and be flexible on years.
- Forgetting who judges quality. A prompt engineer cannot decide alone whether a legal or medical answer is correct. Fix: name the domain experts and how much of their time is available.
Contract, contract-to-hire or permanent: which suits a prompt engineer
Contract suits this role more often than most. A specialist can build the first evaluation set, tune an assistant before launch or run a model comparison, then leave your team with tests and documentation. It is also a sensible way to find out whether you need the job at all.
Contract-to-hire works where the role may grow into wider AI engineering. It lets you see how the person handles real production failures, and whether they can write code to the level your team needs.
Permanent hiring suits companies with several live LLM products, where quality work never finishes because models and user behaviour keep changing. Zilo AI can help you hire prompt engineers in India on contract, contract-to-hire or permanent terms, screening for hands-on work with real data, modern frameworks and deployment. Where the job is broader, our IT talent hiring page covers AI engineers and related roles.
Frequently asked questions
Is prompt engineer still a real job?
Yes, but the standalone title is less common than the skill. Many companies treat prompting and evaluation as part of AI engineering, while separate roles tend to appear in larger AI teams or domains where quality review is heavy.
What is the difference between a prompt engineer and a generative AI engineer?
A prompt engineer focuses on instructions, examples and evaluation. A generative AI engineer builds the application as well, including retrieval, integrations, tool calling and deployment. Hire the engineer first if the product does not exist yet.
Does a prompt engineer need to know how to code?
For most roles, some Python is needed, because evaluations have to run across large sets of examples. A non-coding version can work inside a good tooling setup, particularly for a domain specialist, but the person will depend on engineers.
Why does evaluation matter so much in this role?
LLM output varies, and a change that improves one case can quietly damage others. Without a test set, nobody can say if a new prompt or model is an improvement, and every decision becomes opinion.
If you are unsure whether you need a prompt engineer or a wider AI engineer, Zilo AI can talk that through with you. Contact the team with a description of your product and where output quality is falling short.