Prompt Ensembling & Voting

Self-consistency varied the sampling; ensembling varies the prompt. Write the same task three ways — different phrasings, different exemplar sets, even different output formats — run each, and aggregate the answers by majority vote. The premise: every prompt has idiosyncratic failure modes. A word choice that nudges borderline cases, an exemplar that teaches a subtly wrong lesson, a format that invites drift. Those quirks are largely uncorrelated across variants, so a mistake one phrasing induces rarely survives the vote. You're diversifying away prompt-specific risk, exactly the way a portfolio diversifies away single-stock risk.

The research lineage — DiVeRSe is the name to know — pushed the idea further in two directions, both simple in plain language. First, multiply diversity: several distinct prompts times several samples each, giving you a grid of reasoning paths instead of a handful. Second, weight the vote: rather than counting every answer equally, a verifier — a second model or a simple programmatic check — scores each path, and trustworthy paths count for more. You don't need the full apparatus to benefit: even three hand-written phrasings with a plain majority vote measurably steadies a flaky classifier. The industrial version is the same idea with the dials turned up.

Two variants earn their keep in production. Format mixing: ask once for a JSON verdict, once for a prose judgment, once for a table — output format itself biases answers, and mixing hedges that bias. Cross-model panels: pose the question to Claude, GPT, and Gemini and compare. The vote is useful; the disagreement is often more useful. Three models agreeing is a cheap confidence signal; a 2-1 split flags exactly the cases worth human review. Eval pipelines use this as a triage layer — unanimous cases auto-pass, splits get eyes. Disagreement detection is the cheapest uncertainty estimate you can buy.

The honest costs: N variants means N times the spend, so ensembling belongs where stakes justify it — offline batch classification, eval labeling, high-impact decisions — not on every chat turn. You need voteable outputs (the self-consistency rule again). And the subtle limit: votes only cancel uncorrelated errors. A blind spot shared by all your variants — or by all frontier models trained on overlapping data — votes unanimously and wrongly. An ensemble is a variance eraser, not a truth oracle; pair it with the grounding and testing patterns that surround it in this module.