The Refusal Gap

This report publishes a measurement protocol for comparing full response, partial response, refusal, and abstention across models under identical inputs. Publication date: 2026-09-10. Last updated: 2026-09-10.

No measured refusal rates, partial-response rates, or abstention rates are published. Measurement not yet available; methodology published; benchmark results forthcoming.

Refusal behavior is a governance signal: it affects completeness of multi-model comparison, escalation load, and whether a human must proceed without a model recommendation. Related work: agreement/divergence benchmark, disagreement taxonomy, and evaluator methodology. Authority context: multi-model governance, consensus vs. divergence, and human-in-the-loop AI.

Findings are design-level or pending. No measured rates appear in this release.

• A measured public refusal rate is not yet available from the corporate evidence corpus.
• Refusal and abstention must be coded as first-class outcomes; treating them as missing data hides comparison bias.
• Identical prompts and frozen model versions are required before any cross-model refusal comparison is publishable.
• Safety and accountability expectations in NIST AI RMF and OECD AI Principles motivate documenting when systems decline to answer^[1][2][3].
• Security and misuse context from CISA’s AI guidance reinforces treating refusal as an operational control signal, not only a product quirk^[4].

Given a fixed prompt set and frozen model versions, how do eligible models differ in full response, partial response, refusal, and abstention — and how should those outcomes affect comparison validity and human escalation?

Quantitative answer: measurement not yet available.

Dataset size: Not published — measured corpus not yet released.

Methodology document ID: SL-RG-METHOD-2026.1 (methodology only — not a dataset).

Status: methodology published; benchmark results forthcoming. Design measurement protocol is complete; observed rates remain pending.

Define a prompt inventory covering permitted, boundary, and prohibited request classes relative to organizational policy. Freeze model identity and version per run. Capture raw outputs before post-processing that could mask refusals.

Code each output into one primary class: full response, partial response, refusal, or abstention. Secondary tags may record policy category and whether a human proceeded without a model answer. Adjudication rules live in SL-RG-METHOD-2026.1.

Compute rates only after labeling is complete and reproducibility checks pass. Until then: measurement not yet available for all quantitative endpoints.

The table is a field design for future measurement. It does not report observed frequencies.

Build escalation paths for refusal-heavy prompts before publishing any rate comparisons. A model that refuses more often is not automatically “safer” or “worse” without policy context.

Do not cite homepage marketing or informal demos as refusal evidence. Wait for a measured corpus under SL-RG-METHOD-2026.1.

Platform context: AI governance platform and human review expectations in human-in-the-loop AI.

No labeled public refusal corpus is released. Cross-provider refusal rankings cannot be inferred from this document.

Provider policy changes can invalidate frozen comparisons; version freeze and re-baselining are mandatory before any future rate publication.

Cited external sources inform governance expectations; they do not contain Smart Logic AI refusal benchmarks^[1][2][3][4].

SmartSolo retains model path and human authorization so refusal events can be reconstructed alongside accepted outputs when instrumentation is enabled.

Research publishes the protocol first. Rates will appear only after the measured corpus and adjudication process meet SL-RG-METHOD-2026.1.

See also agreement/divergence and evaluator methodology.