Reverse Prompting: Work Backward from the Output, Then Prove the Prompt

Give a model an excellent incident report and ask, “What prompt would produce this?” It will usually return something plausible. That feels like reverse engineering. It is not yet engineering.

The returned prompt may reproduce the visible headings while missing the evidence rules that made the report trustworthy. It may copy details that belonged to one incident into a supposedly reusable instruction. It may work on the model that inferred it and drift on the model that must run it. Most importantly, one attractive reproduction tells you almost nothing about how the prompt behaves on the next hundred cases.

Reverse prompting becomes useful when we stop treating the inferred prompt as an answer and start treating it as a candidate artifact:

desired outputs
  -> infer the latent task contract
  -> generate candidate prompts
  -> run them forward on held-out cases
  -> score behavior, safety, cost, and robustness
  -> promote or reject

That is the production idea in one line: work backward to discover a specification, then work forward to prove it.

The term is overloaded

“Reverse prompting” does not yet name one standardized technique. At least four different activities travel under the label:

These are related inverse problems, but they are not interchangeable.

In formal language-model inversion, the original prompt exists and is hidden. Morris et al. showed that next-token probability distributions can leak substantial information about preceding text; on one Llama-2 7B setup, their method recovered 27% of evaluated prompts exactly [1]. Zhang, Morris, and Shmatikov later introduced output2prompt, a black-box method that learns to infer prompts from ordinary text outputs without model logits [2].

Li and Klabjan’s EMNLP 2025 reverse prompt engineering work made the setting stricter: black-box access, no task-specific training, and only five text outputs. Their LLM-plus-genetic-optimization method beat output2prompt by an average 7.3% in cosine similarity across their reported datasets and embedding models; on system-prompt reconstruction it reported a 5.8% improvement over the tailored baseline [3].

That is much stronger than asking a chat model once, “What prompt made this?” It is also not proof of exact recovery. Semantic similarity between two prompts does not establish that they are textually identical, behaviorally equivalent on every input, or safe to deploy.

The practical reverse brief has a different goal. Usually there is no hidden original prompt to steal. A team has examples of good work—reports, analyses, code reviews, support replies—and wants to recover the invariants that made them good. Exact reconstruction is impossible because there may never have been one prompt in the first place. A great artifact may be the result of domain judgment, several revisions, tools, source material, and editorial review.

The right target is therefore not “the prompt.” It is the smallest testable contract that reliably reproduces the desired properties.

The inverse direction can also be useful when the goal is not reconstruction. Hong et al. learn mappings from evaluation outputs back to model-specific evaluator instructions and report that one evaluation sample can seed the process [8]. The use case is narrower than a general reverse brief, but it reinforces the architectural point: an output can initialize an instruction candidate; the candidate still has to be evaluated for the model and task that will use it.

Why one output is weak evidence

An output is a many-to-one projection. Many prompts, contexts, models, tool results, and sampling paths can produce similar text.

Suppose a postmortem says:

Checkout error rate rose after release 2026.07.18. The team rolled back at 14:32 UTC. No customer payment data was lost.

A reverse prompt may infer: “Write a concise incident summary with timeline, cause, remediation, and customer impact.” That is reasonable. But it cannot know from the output alone whether:

• UTC timestamps were a hard contract or happened to appear in this incident;
• the data-loss statement required a security-system receipt;
• the model was allowed to infer causality or had to cite a confirmed finding;
• “concise” meant 150 words, five sections, or one screen;
• rollback was a required recommendation or merely the action taken this time;
• legal review removed a sentence before publication.

One example entangles invariant structure with incidental content. Multiple outputs from the same process help separate them. The Li and Klabjan study found the same basic pattern in a formal setting: a single answer encouraged the inverter to overfit response-specific details, while five answers better exposed what was stable across generations [3].

For a production reverse brief, collect variation deliberately:

• several successful outputs from the same task class;
• edge cases where the correct output is a refusal, escalation, or request for more evidence;
• examples from different authors, tenants, and time periods;
• rejected outputs with reviewer reasons;
• the source material and receipts used to produce each output;
• counterexamples that look stylistically good but are factually or operationally wrong.

The objective is to infer the intersection of good behavior, not the union of visible phrasing.

Recover a contract, not a magic string

A useful inverse analysis decomposes each example into layers:

That decomposition prevents a common failure: using prose instructions to carry controls that belong elsewhere.

“Never issue a refund above $500 without approval” is not merely a writing preference. In ContextOS it belongs in policy evaluation and an approval gate, outside model discretion. “Use only verified and current evidence” belongs in context admission and evidence gates. The prompt can explain those controls, but it should not be their enforcement boundary.

For an incident report, the recovered artifact might look like this:

task: produce_incident_summary
inputs:
  required:
    - incident_record
    - deployment_events
    - mitigation_timeline
    - customer_impact_evidence
output:
  sections:
    - executive_summary
    - verified_timeline
    - confirmed_cause_or_unknown
    - customer_impact
    - corrective_actions
constraints:
  - every material claim cites an evidence_ref
  - timestamps use UTC
  - do not convert correlation into causation
  - if impact evidence is missing, say unknown and escalate
style:
  audience: engineering_and_operations_leaders
  tone: direct_non_blameful
  target_words: 500

This is already more valuable than a polished mega-prompt. It separates inputs, schema, evidence discipline, uncertainty behavior, and style so each can be tested or enforced at the right layer.

The production workflow

1. Build an example set with lineage

Store each desired output with its source inputs, model and version if known, generation settings, tool transcripts, reviewer edits, and final disposition. If the example was edited by a human, preserve both the generated and accepted forms. Otherwise the reverse process will attribute human judgment to a nonexistent model instruction.

Provenance also protects against circularity. An output generated by the system under evaluation should not quietly enter the gold set as if it were an independent human reference.

2. Extract invariant and variable features

Mark features as:

• required: must appear or hold on every eligible case;
• conditional: applies only when a typed predicate is true;
• preferred: improves utility but may trade off with another goal;
• incidental: belongs to one example and must not enter the reusable contract;
• forbidden: a known failure pattern, even if it appears in an attractive example.

This is where domain reviewers earn their keep. A model can propose the decomposition, but a source owner should decide which property represents real policy, business semantics, or factual quality.

3. Generate diverse candidate prompts

Do not ask for one prompt. Ask for candidates that express the same contract using different levels of detail and structure. Include a deliberately minimal candidate; more instructions are not automatically better.

Automatic Prompt Engineer, OPRO, adversarial in-context learning, and later optimizers all support the broader lesson that prompt candidates should be generated and selected against a metric, not accepted because their wording sounds expert [4, 5]. Reverse prompting provides a good initialization; evaluation chooses what survives.

Candidate diversity should vary meaningful choices:

• explicit schema versus an example-led instruction;
• concise versus highly constrained wording;
• one integrated prompt versus staged planner/writer/reviewer calls;
• model-written rubric versus domain-authored rubric;
• no exemplar, one exemplar, and several contrastive exemplars.

Avoid fake diversity where five candidates only swap synonyms.

4. Run the candidates forward

The defining validation step is behavioral:

candidate prompt + held-out input + pinned model/config
  -> generated output
  -> deterministic checks + rubric scorers + human review
  -> scorecard

Use held-out cases the inference step did not see. Run several generations where the serving configuration is stochastic. Repeat across the model versions you actually plan to support. A prompt that only reproduces its source examples has memorized a surface; it has not recovered a task.

Keep the original desired output out of the runtime context during this test. Otherwise the model may copy the answer instead of following the inferred contract.

5. Score the behavior in layers

One similarity score is not enough.

Treat some checks as floors, not weights. A candidate should not compensate for a policy violation by being beautifully written.

Select on a Pareto frontier when objectives conflict. The shortest prompt may be cheaper but less robust. The most constrained prompt may improve format compliance while reducing task quality on a more capable model. A 2025 preprint used the phrase “prompting inversion” for that latter crossover—strong constraints helped one model generation but hurt a more capable one—which is another reason to pin evaluation to model version [6]. That usage is different from output-to-prompt reconstruction, despite the similar name.

6. Promote the whole evaluated bundle

The deployable artifact is not only text. It is:

prompt or context-pack version
+ supported model/config matrix
+ dataset snapshot
+ evaluators and thresholds
+ scorecard
+ known failure slices
+ rollout and rollback target

In ContextOS terms, the inferred wording can become a candidate prompt fragment inside a Context Pack. Policy manifests, tool manifests, evidence gates, approval modes, and source eligibility remain independently governed. The Improvement Loop replays the candidate and promotes it only after the scorecard clears its gates.

That preserves the useful part of reverse prompting—learning from good artifacts—without turning a model’s reconstruction into production authority.

A compact implementation shape

The following is an illustrative article schema, not a new ContextOS public contract:

type ReversePromptCandidate = {
  candidate_id: string
  instruction: string
  inferred_from_example_ids: string[]
  model_scope: string[]
}
 
type CandidateScorecard = {
  candidate_id: string
  dataset_snapshot: string
  contract_pass_rate: number
  evidence_precision: number
  utility_score: number
  policy_violations: number
  p95_tokens: number
}
 
function eligible(s: CandidateScorecard): boolean {
  return s.policy_violations === 0
    && s.contract_pass_rate >= 0.98
    && s.evidence_precision >= 0.97
}
 
function rank(a: CandidateScorecard, b: CandidateScorecard): number {
  if (a.utility_score !== b.utility_score) return b.utility_score - a.utility_score
  if (a.p95_tokens !== b.p95_tokens) return a.p95_tokens - b.p95_tokens
  return a.candidate_id.localeCompare(b.candidate_id)
}

Notice what the code does not do: it does not ask another model which prompt “feels” best. Eligibility is deterministic, safety violations are disqualifying, and ties have a stable resolution. Model-based judges can contribute signals, but promotion rules should remain explicit.

Failure modes to expect

The prompt copies the answer

The inferred prompt includes names, facts, or turns of phrase that belong to one output. Forward tests on new entities expose this quickly. More varied examples and explicit incidental-feature labeling reduce it.

The prompt recovers style but not causality

Surface similarity rewards headings and vocabulary. It may miss whether claims were grounded or whether a decision rule was applied. Add evidence-level and decision-level checks.

The evaluator rewards its own preferences

If the same model family infers the prompt, generates outputs, and judges them, correlated bias can make a weak candidate look strong. Use deterministic validators, source-grounded checks, human review on important slices, and evaluator diversity.

The prompt is model-relative

Prompts are programs for a changing interpreter. Model upgrades can alter instruction following, tool selection, verbosity, and sensitivity to examples. Re-run the scorecard for every serving change.

The examples encode obsolete policy

Excellent historical outputs may be correct under an old policy. Provenance, valid-time metadata, and source-owner review must precede inference. Reverse prompting should not turn yesterday’s exception into tomorrow’s rule.

The method leaks a protected instruction

Language-model inversion is also a security result. Output-only and logit-based methods show that hidden prompts can leak behavior and, sometimes, text [1, 2, 3, 7]. Do not store secrets in system prompts. Do not expose log probabilities casually. Enforce authorization, data access, and side-effect policy outside the prompt. Assume competitors and attackers can infer at least part of a public-facing system’s operating instructions from repeated outputs.

When reverse prompting is worth using

Use it when:

• the organization has strong examples but a weakly documented process;
• expert reviewers can distinguish structural quality from superficial imitation;
• the output is testable on a meaningful held-out dataset;
• the goal is to bootstrap a contract, rubric, or candidate prompt;
• you can preserve provenance and separate historical policy from current policy.

Avoid it when:

• one artifact is being treated as enough evidence for a general workflow;
• the desired result depends on tacit judgment no evaluator can currently measure;
• reproducing a third party’s hidden system prompt would violate authorization or trust;
• the output contains sensitive data that should not enter an optimization loop;
• the team wants a shortcut around dataset construction and review.

The quality of reverse prompting is capped by the quality of its examples and evaluators. It does not remove specification work. It changes where that work starts.

The durable idea

Forward prompt engineering asks: What instructions might produce good behavior?

Reverse prompting asks: What hidden contract is consistent with the behavior we want?

Production engineering asks the final question: Does the recovered contract keep producing that behavior under new inputs, current policy, bounded authority, and a pinned runtime?

Only the third question closes the loop.

Reverse prompting is therefore not a way to read a model’s mind. It is inverse design: learn a candidate specification from outcomes, make its assumptions explicit, and try to falsify it before release. The prompt is the hypothesis. Replay and evaluation are the proof.

References

[1] John X. Morris et al., “Language Model Inversion”, ICLR 2024.

[2] Collin Zhang, John X. Morris, and Vitaly Shmatikov, “Extracting Prompts by Inverting LLM Outputs”, EMNLP 2024.

[3] Hanqing Li and Diego Klabjan, “Reverse Prompt Engineering: A Zero-Shot, Genetic Algorithm Approach to Language Model Inversion”, EMNLP 2025.

[4] Yongchao Zhou et al., “Large Language Models Are Human-Level Prompt Engineers”, ICLR 2023.

[5] Chengrun Yang et al., “Large Language Models as Optimizers”, ICLR 2024.

[6] Imran Khan, “You Don’t Need Prompt Engineering Anymore: The Prompting Inversion”, 2025 preprint.

[7] Murtaza Nazir et al., “Better Language Model Inversion by Compactly Representing Next-Token Distributions”, 2025 preprint.

[8] Hanhua Hong et al., “Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Prompts”, TACL 2026.

What to read next

• Dataset-First Agent Engineering
• Context Engineering in Production
• Autotune the Harness Improvement Loop