Prompt Versioning & Model Migration
Prompts are code: version them, changelog them, A/B them against a fixed test set, and migrate them safely across models.
Treat prompts as code, because they are. A prompt is a program written in English that controls expensive, user-facing behavior, and a one-word change can shift outputs measurably. So prompts belong in version control, not in a Slack message or a hardcoded string buried in a handler. Give each prompt a file, an id, and a version number, and put it behind an interface your app calls by name and version. The moment two people edit prompts, or you run more than one in production, 'which prompt produced this output?' becomes a question you must be able to answer — and only versioning answers it cleanly.
Keep a changelog and make every change reviewable. When you edit a prompt, record what changed, why, and what you expected it to fix — the same discipline as a code commit. Prompt diffs get code review: a second set of eyes catches the removed constraint or the ambiguous new sentence. Tie each version to its evaluation results so the log reads 'v7 raised billing-category accuracy from 91% to 96% on the test set.' Without this, prompt engineering degrades into folklore — nobody remembers why a weird instruction is there, so nobody dares remove it, and cruft quietly accumulates for years.
Never ship a prompt change on vibes; A/B it against a fixed test set. Build a stable set of representative inputs with known-good outputs — your regression suite — and run both the old and new prompt across it before promoting. Measure what matters for the task: accuracy, format validity, refusal rate, cost, latency. This turns 'it feels better' into 'v8 wins on accuracy, ties on cost.' For subjective tasks, an LLM-as-judge or a small human rating panel over the same fixed set gives you a comparable score. The test set is the asset; guard it and grow it as new failure modes appear in production.
Model migration is when this discipline pays off. You will move prompts across model generations — for cost, capability, or because a model is deprecated — and prompts do not transfer perfectly. A prompt hand-tuned for one model's quirks can underperform on the next in surprising ways: reasoning style, default verbosity, and format adherence all shift. Migrate deliberately: run your existing prompt on the new model against the same test set first to get a baseline, then re-tune only where it regresses, logging changes as a new version. Your gateway from Course 04 makes swapping the model a config change; your test set makes it a safe one.