Artificial Intelligence
A production prompt engineering checklist
Turn informal prompts into versioned, testable application assets with clear inputs, outputs, and failure behavior.
Documentation checked: verified against the official references below on October 6, 2026. Product names, preview status, and certification objectives can change.
Treat the prompt as an application interface
A production prompt defines the job, accepted context, boundaries, expected output, and failure behavior between an application and a probabilistic model. Write it with the same care as an API contract: its callers, inputs, outputs, versions, tests, and owners should be visible.
- One named task and intended user outcome
- Known input fields and trusted context
- Allowed and prohibited behavior
- A stable output contract
- Explicit uncertainty and escalation behavior
- An owner, version, and change record
Start with a short task contract
Before tuning wording, write a plain-language contract that the product team can review. If success cannot be described without saying 'the answer feels better,' the prompt is not ready for engineering.
- Role: what responsibility the model has
- Goal: the result a successful response produces
- Audience: who will consume the result
- Evidence: which sources or tools may support claims
- Constraints: scope, length, tone, policy, and latency
- Fallback: what happens when required information is absent
Separate trusted instructions from untrusted data
System or developer instructions, user input, retrieved documents, tool results, and conversation history have different trust levels. Delimit them clearly and never assume content becomes safe because it appears inside a prompt. Retrieved documents and tool output can contain instructions intended to redirect the model.
- Keep stable application rules outside user-editable fields
- Label each context block by source and purpose
- Treat documents, webpages, emails, and tool output as data rather than authority
- Apply authorization before context reaches the model
- Validate proposed actions in application code before execution
Make the output contract enforceable
If downstream code expects structured data, use a supported structured-output or function-calling schema instead of asking for JSON only in prose. Azure OpenAI structured outputs constrain responses to a supported subset of JSON Schema, while the application still needs to validate business rules and handle refusals or incomplete results.
- Keep the schema small and stable
- Name and describe every field precisely
- Reject additional properties when the API requires it
- Validate enums, identifiers, permissions, and domain rules after parsing
- Define a typed error or no-result path
- Test model and API support before deployment
Define boundary and failure behavior
A useful system message says what to do when input is ambiguous, unsupported, unsafe, contradictory, or missing evidence. Do not force the model to answer every request; a clear clarification, abstention, or escalation path is part of the product.
- Ask one focused clarification question when required input is missing
- State when available evidence does not support an answer
- Return a safe refusal for prohibited requests
- Escalate consequential decisions to an authorized person
- Avoid fabricating sources, identifiers, actions, or certainty
- Use a predictable error category the application can display
Use examples to teach difficult boundaries
Few-shot examples are most valuable when they demonstrate distinctions that instructions alone leave ambiguous. Include representative positive, negative, and edge cases rather than several easy variations of the same request.
- A normal successful request
- A request that needs clarification
- A valid no-answer case
- A prohibited or out-of-scope request
- A case with conflicting context
- A case requiring the exact output schema
Adapt techniques to the selected model
Prompt advice is not universal across model families and versions. Microsoft documentation notes that some classic prompt-engineering techniques are not recommended for reasoning models. Keep the task contract stable, but test wording, examples, parameters, and decomposition with the exact deployment used in production.
- Record provider, model, deployment, and version
- Do not transfer parameter settings blindly between model families
- Compare concise instructions with more decomposed workflows
- Re-run the regression set after a model update
- Check current documentation before relying on a model-specific technique
Layer safety outside the prompt
A system message can reduce unwanted behavior, but it cannot guarantee compliance. Pair prompt boundaries with content controls, least-privilege tools, input and output validation, authorization, rate limits, approval gates, and an interface that communicates limitations.
- Threat-model direct and indirect prompt injection
- Constrain tools by allowlist, schema, identity, and consequence
- Require approval before consequential or irreversible actions
- Prevent secrets and unnecessary sensitive data from entering prompts or logs
- Test over-refusal as well as unsafe compliance
- Define incident and rollback procedures
Build a representative regression dataset
Create reusable test cases before optimizing the prompt. Microsoft Foundry evaluation datasets can compare prompt, model, or agent versions and support CI/CD quality gates. Start with a small curated set, then add production failures and reviewed edge cases.
- User input and required context
- Expected behavior or ground truth when one exists
- Prohibited behavior
- Schema and deterministic assertions
- Quality, safety, latency, and cost criteria
- Normal, ambiguous, adversarial, and dependency-failure cases
Score objective checks before subjective quality
Use deterministic tests for requirements such as schema validity, required fields, forbidden phrases, citations, and tool calls. Use rubric or model-assisted evaluators for relevance, completeness, tone, or groundedness, and calibrate those scores against human review.
- Parse and schema success
- Correct refusal, clarification, or escalation category
- Expected evidence and citation presence
- No prohibited action or unauthorized data
- Task-specific quality rubric
- Latency, token use, and cost per successful task
Version the complete behavior
Prompt text is only one input to behavior. Version the system instructions together with model deployment, generation settings, schemas, tool definitions, retrieval configuration, safety settings, and evaluation results so a regression can be reproduced.
- Semantic prompt version and owner
- Model and API version
- Output schema and tool contracts
- Retrieval and grounding configuration
- Evaluation dataset version and score summary
- Approval, release, and rollback decision
Use release gates, not prompt intuition
Run fast deterministic checks on every change and a representative evaluation before promotion. Compare the candidate against an approved baseline, inspect high-impact failures, and block release when critical safety or task-completion scenarios regress.
- No critical policy or authorization failure
- Output contract pass rate meets the target
- Task quality is not worse than the baseline
- Latency and cost remain within budget
- A reviewer signs off high-impact scenarios
- Previous version and configuration remain deployable
Monitor prompts after deployment
Offline tests cannot predict every real request. Monitor aggregate outcomes, schema failures, no-answer rates, safety events, latency, cost, and user corrections. Sample interactions only under an approved privacy and retention policy, then convert reviewed failures into regression tests.
- Prompt, model, and tool version in traces
- Successful task and escalation rates
- Validation and parsing failures
- Safety-filter and approval events
- Latency, tokens, and cost
- Feedback triage, evaluation update, and rollback trigger
Official sources
Use these primary references to verify product capabilities, exam objectives, and implementation details.
Put this into practice
Open the prompt review checklist and apply it to one versioned prompt from your application.
Open resource