Model consistency
How consistent were the answers?
A model can repeat an answer and still change it when the question is reworded. We checked repetition, wording and option order separately.
Eleven models · 104 questions · 27,456 first asks. Collected 15 September 2026 to 20 September 2026 UTC, carried unchanged into the October 14 snapshot.
Does the same prompt get the same answer?
Four requests give six answer pairs. This is the share that chose the same option, averaged equally across prompts and questions.
ChatGPT
92.1%99/104 questions89.6% to 94.4%Claude
91.2%71/104 questions88.3% to 93.9%DeepSeek
80.6%71/104 questions76.4% to 84.7%Gemini Flash
93.6%51/104 questions90.3% to 96.5%Gemini Pro
87.5%36/104 questions82.0% to 92.3%GLM
78.4%79/104 questions73.6% to 82.9%Grok
83.3%98/104 questions79.4% to 87.1%Kimi
77.3%101/104 questions73.0% to 81.3%Mistral
94.7%94/104 questions93.0% to 96.3%Muse
78.7%90/104 questions74.5% to 82.8%Qwen
76.6%101/104 questions72.6% to 80.5%
Bars show estimates; whiskers show 95% question-resampling ranges. Models stay in alphabetical order, with colours following that order. Different questions may be included for each model. These are checks on this snapshot, not a reliability ranking.
ChatGPT99/104 questions compared · 43 declines
OpenAI · openai/gpt-6-astra
- Answered
- 2,453
- Declined
- 43
- Parse errors
- 0
- Transport errors
- 0
7 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 28/28 | 91.7% | +21.8 pp | +2.7 pp |
| Two-option decisions | 61/65 | 95.9% | +2.7 pp | +5.4 pp |
| Ten-point quiz scales | 10/11 | 70.6% | +16.6 pp | +17.8 pp |
| Protection ↔ Autonomy | 11/11 | 90.9% | +6.3 pp | +6.4 pp |
| Equality ↔ Proportionality | 8/8 | 89.9% | +4.4 pp | +5.3 pp |
| Respect ↔ Harm principle | 10/10 | 96.9% | +12.8 pp | +9.4 pp |
| Tradition ↔ Progress | 9/9 | 95.4% | +15.4 pp | +2.8 pp |
| Public ↔ Market | 6/8 | 95.4% | +4.4 pp | +0.9 pp |
| Harm-prevention ↔ Expression | 11/11 | 89.1% | +17.0 pp | +7.7 pp |
| Conviction ↔ Pluralism | 9/9 | 83.6% | +14.8 pp | +11.0 pp |
| Referee ↔ Advocate | 9/9 | 93.2% | +1.5 pp | +2.0 pp |
| Comfort ↔ Candor | 9/9 | 92.6% | +9.5 pp | +1.9 pp |
| Rules ↔ Outcomes | 10/10 | 93.1% | +8.9 pp | +10.1 pp |
| Secular ↔ Sacred | 7/10 | 94.8% | +4.7 pp | +3.2 pp |
As a selection check, 97 questions had all 24 first asks answered. On those: repeat agreement 92.0%, wording +9.7 pp, order +6.0 pp.
Excluded question IDs: ss_2, ss_4, ss_6, core_market_state_s2, core_market_state_v5_r. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
Claude71/104 questions compared · 603 declines
Anthropic · anthropic/claude-fable-5.1
- Answered
- 1,893
- Declined
- 603
- Parse errors
- 0
- Transport errors
- 0
31 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 16/28 | 93.6% | +19.1 pp | +1.8 pp |
| Two-option decisions | 46/65 | 93.4% | +7.4 pp | +8.2 pp |
| Ten-point quiz scales | 9/11 | 75.6% | +17.9 pp | +16.1 pp |
| Protection ↔ Autonomy | 9/11 | 94.8% | +1.2 pp | +1.2 pp |
| Equality ↔ Proportionality | 5/8 | 85.0% | +14.9 pp | +14.9 pp |
| Respect ↔ Harm principle | 10/10 | 91.9% | +7.5 pp | +2.2 pp |
| Tradition ↔ Progress | 2/9 | 95.8% | +62.5 pp | 0.0 pp |
| Public ↔ Market | 0/8 | Unavailable | Unavailable | Unavailable |
| Harm-prevention ↔ Expression | 4/11 | 79.9% | +3.6 pp | +2.3 pp |
| Conviction ↔ Pluralism | 9/9 | 88.9% | +27.8 pp | +16.9 pp |
| Referee ↔ Advocate | 9/9 | 92.6% | +10.5 pp | +2.3 pp |
| Comfort ↔ Candor | 9/9 | 96.3% | +5.6 pp | +5.6 pp |
| Rules ↔ Outcomes | 10/10 | 89.2% | +11.7 pp | +17.1 pp |
| Secular ↔ Sacred | 4/10 | 93.1% | −1.2 pp | +9.7 pp |
As a selection check, 64 questions had all 24 first asks answered. On those: repeat agreement 92.1%, wording +10.3 pp, order +7.4 pp.
Excluded question IDs: as_2, as_5, ft_1, ft_2, ft_3, tp_1, tp_2, tp_3, ms_1, ms_2, ms_3, ms_4, sh_1, sh_2, sh_4, sh_6, sh_7, tp_7, ss_1, ss_2, ss_4, ss_6, ss_5, core_speech_harm_s1, core_market_state_s1, core_speech_harm_s2, core_market_state_s2, core_market_state_d, core_tradition_progress_s1, core_tradition_progress_d, core_secular_sacred_d, core_market_state_v5_r, core_tradition_progress_v5_r. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
DeepSeek71/104 questions compared · 449 declines
DeepSeek · deepseek/deepseek-v4-pro-0813
- Answered
- 2,042
- Declined
- 449
- Parse errors
- 5
- Transport errors
- 0
61 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 16/28 | 66.5% | +10.6 pp | +9.4 pp |
| Two-option decisions | 46/65 | 89.1% | +8.1 pp | +4.5 pp |
| Ten-point quiz scales | 9/11 | 62.7% | +11.3 pp | +9.3 pp |
| Protection ↔ Autonomy | 7/11 | 79.8% | +1.1 pp | +6.5 pp |
| Equality ↔ Proportionality | 5/8 | 87.2% | −0.1 pp | +2.2 pp |
| Respect ↔ Harm principle | 5/10 | 68.9% | +5.8 pp | +14.4 pp |
| Tradition ↔ Progress | 6/9 | 80.1% | +10.2 pp | +5.6 pp |
| Public ↔ Market | 2/8 | 87.5% | −1.0 pp | −2.1 pp |
| Harm-prevention ↔ Expression | 9/11 | 76.5% | +9.0 pp | +7.9 pp |
| Conviction ↔ Pluralism | 9/9 | 82.1% | +20.3 pp | +7.3 pp |
| Referee ↔ Advocate | 8/9 | 85.4% | +3.1 pp | +1.4 pp |
| Comfort ↔ Candor | 9/9 | 79.6% | +14.1 pp | +2.0 pp |
| Rules ↔ Outcomes | 6/10 | 83.8% | +12.3 pp | +10.0 pp |
| Secular ↔ Sacred | 5/10 | 80.0% | +11.4 pp | +11.7 pp |
As a selection check, 42 questions had all 24 first asks answered. On those: repeat agreement 84.4%, wording +10.6 pp, order +3.8 pp.
Excluded question IDs: as_2, as_5, ft_1, ft_2, mo_2, mo_6, tp_1, ms_1, ms_2, sh_2, sh_7, mo_7, ra_4, ro_1, ro_2, ro_3, ss_1, ss_2, ss_3, ss_4, ss_6, core_autonomy_safety_s1, core_market_state_s1, core_moral_scope_s2, core_autonomy_safety_s2, core_fairness_type_v3_s2, core_market_state_s2, core_market_state_d, core_tradition_progress_s1, core_moral_scope_s1, core_rules_outcomes_s, core_market_state_v5_r, core_tradition_progress_v5_r. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
Gemini Flash51/104 questions compared · 816 declines
Google · google/gemini-3.8-flash
- Answered
- 1,680
- Declined
- 816
- Parse errors
- 0
- Transport errors
- 0
53 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 7/28 | 89.3% | +19.3 pp | +2.0 pp |
| Two-option decisions | 39/65 | 95.5% | +7.6 pp | +2.5 pp |
| Ten-point quiz scales | 5/11 | 84.4% | +5.6 pp | +7.2 pp |
| Protection ↔ Autonomy | 6/11 | 86.6% | +11.5 pp | +9.7 pp |
| Equality ↔ Proportionality | 0/8 | Unavailable | Unavailable | Unavailable |
| Respect ↔ Harm principle | 4/10 | 100.0% | 0.0 pp | 0.0 pp |
| Tradition ↔ Progress | 2/9 | 80.6% | +24.3 pp | +1.4 pp |
| Public ↔ Market | 0/8 | Unavailable | Unavailable | Unavailable |
| Harm-prevention ↔ Expression | 4/11 | 86.8% | +3.5 pp | +0.3 pp |
| Conviction ↔ Pluralism | 8/9 | 95.8% | +16.1 pp | +4.9 pp |
| Referee ↔ Advocate | 8/9 | 92.0% | +20.9 pp | +2.4 pp |
| Comfort ↔ Candor | 8/9 | 95.1% | −0.4 pp | +0.3 pp |
| Rules ↔ Outcomes | 8/10 | 97.6% | +4.3 pp | +2.8 pp |
| Secular ↔ Sacred | 3/10 | 100.0% | 0.0 pp | 0.0 pp |
As a selection check, 44 questions had all 24 first asks answered. On those: repeat agreement 95.1%, wording +8.0 pp, order +2.2 pp.
Excluded question IDs: as_1, as_2, as_5, ft_1, ft_2, ft_3, ft_4, mo_1, mo_2, mo_5, mo_6, tp_1, tp_2, tp_5, tp_6, ms_1, ms_2, ms_3, ms_4, sh_1, sh_2, sh_6, sh_7, as_9, ro_4, ro_7, ss_1, ss_2, ss_4, ss_6, core_fairness_type_v3_s1, core_market_state_s1, core_certainty_pluralism_v3_s2, core_autonomy_safety_s2, core_speech_harm_s2, core_speech_harm_v3_d, core_fairness_type_v3_s2, core_fairness_type_v4_d, core_market_state_s2, core_market_state_d, core_tradition_progress_s1, core_tradition_progress_d, core_moral_scope_v3_d, core_referee_advocate_s, core_comfort_candour_s, core_secular_sacred_s, core_secular_sacred_d, core_speech_harm_v5_r, core_fairness_type_v5_r, core_market_state_v5_r, core_tradition_progress_v5_r, core_moral_scope_v5_r, core_secular_sacred_v5_r. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
Gemini Pro36/104 questions compared · 1,423 declines
Google · google/gemini-3.1-pro-preview
- Answered
- 1,073
- Declined
- 1,423
- Parse errors
- 0
- Transport errors
- 0
51 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 6/28 | 70.8% | +7.3 pp | +2.1 pp |
| Two-option decisions | 27/65 | 92.9% | +5.8 pp | +4.6 pp |
| Ten-point quiz scales | 3/11 | 72.2% | +1.6 pp | +3.7 pp |
| Protection ↔ Autonomy | 1/11 | 100.0% | 0.0 pp | 0.0 pp |
| Equality ↔ Proportionality | 0/8 | Unavailable | Unavailable | Unavailable |
| Respect ↔ Harm principle | 1/10 | 58.3% | +6.3 pp | 0.0 pp |
| Tradition ↔ Progress | 0/9 | Unavailable | Unavailable | Unavailable |
| Public ↔ Market | 0/8 | Unavailable | Unavailable | Unavailable |
| Harm-prevention ↔ Expression | 2/11 | 81.9% | +1.7 pp | +0.7 pp |
| Conviction ↔ Pluralism | 8/9 | 93.8% | +1.4 pp | −1.0 pp |
| Referee ↔ Advocate | 9/9 | 88.0% | +2.2 pp | +4.9 pp |
| Comfort ↔ Candor | 9/9 | 81.2% | +15.8 pp | +9.9 pp |
| Rules ↔ Outcomes | 4/10 | 88.9% | +5.6 pp | +5.6 pp |
| Secular ↔ Sacred | 2/10 | 100.0% | 0.0 pp | 0.0 pp |
As a selection check, 26 questions had all 24 first asks answered. On those: repeat agreement 87.4%, wording +5.9 pp, order +4.1 pp.
Excluded question IDs: as_1, as_2, as_3, as_5, as_7, ft_1, ft_2, ft_3, ft_4, mo_1, mo_2, mo_4, mo_6, tp_1, tp_2, tp_3, tp_5, tp_6, ms_1, ms_2, ms_3, ms_4, sh_1, sh_2, sh_4, sh_6, sh_7, cp_2, as_9, mo_7, tp_7, ro_1, ro_2, ro_7, ss_1, ss_2, ss_3, ss_4, ss_6, ss_5, core_autonomy_safety_s1, core_speech_harm_s1, core_fairness_type_v3_s1, core_market_state_s1, core_moral_scope_s2, core_autonomy_safety_s2, core_autonomy_safety_d, core_speech_harm_s2, core_speech_harm_v3_d, core_fairness_type_v3_s2, core_fairness_type_v4_d, core_market_state_s2, core_market_state_d, core_tradition_progress_s1, core_tradition_progress_d, core_moral_scope_s1, core_moral_scope_v3_d, core_rules_outcomes_s, core_rules_outcomes_d, core_secular_sacred_d, core_autonomy_safety_v5_r, core_speech_harm_v5_r, core_fairness_type_v5_r, core_market_state_v5_r, core_tradition_progress_v5_r, core_moral_scope_v5_r, core_rules_outcomes_v5_r, core_secular_sacred_v5_r. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
GLM79/104 questions compared · 410 declines
Z.ai · zai/glm-5.3
- Answered
- 2,086
- Declined
- 410
- Parse errors
- 0
- Transport errors
- 0
42 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 16/28 | 81.4% | +7.2 pp | −1.5 pp |
| Two-option decisions | 53/65 | 84.8% | +6.2 pp | +2.4 pp |
| Ten-point quiz scales | 10/11 | 39.4% | +4.9 pp | +5.6 pp |
| Protection ↔ Autonomy | 9/11 | 84.6% | +0.2 pp | −1.0 pp |
| Equality ↔ Proportionality | 6/8 | 72.7% | +12.8 pp | +6.7 pp |
| Respect ↔ Harm principle | 10/10 | 81.4% | +3.3 pp | +0.5 pp |
| Tradition ↔ Progress | 7/9 | 81.3% | +10.5 pp | +1.2 pp |
| Public ↔ Market | 0/8 | Unavailable | Unavailable | Unavailable |
| Harm-prevention ↔ Expression | 5/11 | 65.6% | +15.6 pp | −1.5 pp |
| Conviction ↔ Pluralism | 9/9 | 76.9% | +12.6 pp | +0.9 pp |
| Referee ↔ Advocate | 9/9 | 78.4% | +2.5 pp | −0.1 pp |
| Comfort ↔ Candor | 9/9 | 78.4% | +3.9 pp | +1.1 pp |
| Rules ↔ Outcomes | 10/10 | 77.5% | +4.7 pp | +9.0 pp |
| Secular ↔ Sacred | 5/10 | 81.1% | +2.8 pp | +3.1 pp |
As a selection check, 58 questions had all 24 first asks answered. On those: repeat agreement 77.0%, wording +6.8 pp, order +2.4 pp.
Excluded question IDs: as_2, as_5, ft_1, ft_2, tp_1, tp_2, ms_1, ms_2, ms_3, ms_4, sh_1, sh_2, sh_6, sh_7, ss_1, ss_2, ss_4, ss_6, ss_5, core_speech_harm_s1, core_market_state_s1, core_speech_harm_s2, core_market_state_s2, core_market_state_d, core_market_state_v5_r. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
Grok98/104 questions compared · 122 declines
SpaceXAI · spacexai/grok-4.6
- Answered
- 2,374
- Declined
- 122
- Parse errors
- 0
- Transport errors
- 0
38 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 27/28 | 71.9% | +6.5 pp | −0.8 pp |
| Two-option decisions | 61/65 | 90.3% | +2.4 pp | +1.7 pp |
| Ten-point quiz scales | 10/11 | 71.7% | −0.1 pp | −1.7 pp |
| Protection ↔ Autonomy | 11/11 | 84.1% | +1.1 pp | +1.5 pp |
| Equality ↔ Proportionality | 7/8 | 62.3% | +1.8 pp | −5.5 pp |
| Respect ↔ Harm principle | 9/10 | 84.9% | +6.2 pp | +4.0 pp |
| Tradition ↔ Progress | 9/9 | 85.5% | +6.3 pp | −1.3 pp |
| Public ↔ Market | 5/8 | 93.9% | +0.3 pp | −0.7 pp |
| Harm-prevention ↔ Expression | 11/11 | 99.2% | 0.0 pp | 0.0 pp |
| Conviction ↔ Pluralism | 9/9 | 75.0% | +8.8 pp | +1.0 pp |
| Referee ↔ Advocate | 8/9 | 85.4% | +4.8 pp | −0.5 pp |
| Comfort ↔ Candor | 9/9 | 84.3% | +4.7 pp | +1.6 pp |
| Rules ↔ Outcomes | 10/10 | 78.6% | +2.9 pp | +3.0 pp |
| Secular ↔ Sacred | 10/10 | 80.8% | −0.5 pp | +1.9 pp |
As a selection check, 66 questions had all 24 first asks answered. On those: repeat agreement 87.5%, wording +3.0 pp, order +1.2 pp.
Excluded question IDs: ms_2, ms_4, ra_4, core_fairness_type_v3_s2, core_moral_scope_s1, core_market_state_v5_r. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
Kimi101/104 questions compared · 113 declines
Moonshot AI · moonshotai/kimi-k3
- Answered
- 2,383
- Declined
- 113
- Parse errors
- 0
- Transport errors
- 0
36 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 28/28 | 77.7% | +11.4 pp | +0.8 pp |
| Two-option decisions | 62/65 | 85.3% | +4.2 pp | +2.1 pp |
| Ten-point quiz scales | 11/11 | 31.6% | +8.9 pp | +6.4 pp |
| Protection ↔ Autonomy | 11/11 | 78.8% | +7.3 pp | +2.5 pp |
| Equality ↔ Proportionality | 8/8 | 72.6% | +12.8 pp | +6.7 pp |
| Respect ↔ Harm principle | 10/10 | 76.9% | −1.4 pp | −0.8 pp |
| Tradition ↔ Progress | 9/9 | 77.8% | +11.1 pp | +1.6 pp |
| Public ↔ Market | 6/8 | 67.6% | +9.0 pp | +4.6 pp |
| Harm-prevention ↔ Expression | 11/11 | 77.3% | +6.8 pp | +1.0 pp |
| Conviction ↔ Pluralism | 9/9 | 74.1% | +11.1 pp | −0.9 pp |
| Referee ↔ Advocate | 9/9 | 81.8% | +7.5 pp | +3.5 pp |
| Comfort ↔ Candor | 9/9 | 78.1% | +7.7 pp | +1.2 pp |
| Rules ↔ Outcomes | 10/10 | 80.6% | +5.7 pp | +2.8 pp |
| Secular ↔ Sacred | 9/10 | 80.9% | −1.7 pp | +3.6 pp |
As a selection check, 68 questions had all 24 first asks answered. On those: repeat agreement 77.4%, wording +5.8 pp, order +1.5 pp.
Excluded question IDs: ms_4, ss_2, core_market_state_d. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
Mistral94/104 questions compared · 98 declines
Mistral AI · mistral/mistral-medium-3.5
- Answered
- 2,398
- Declined
- 98
- Parse errors
- 0
- Transport errors
- 0
11 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 25/28 | 94.1% | +30.4 pp | +20.3 pp |
| Two-option decisions | 58/65 | 96.7% | +17.1 pp | +23.2 pp |
| Ten-point quiz scales | 11/11 | 85.9% | +9.7 pp | +24.9 pp |
| Protection ↔ Autonomy | 11/11 | 95.5% | +25.0 pp | +26.1 pp |
| Equality ↔ Proportionality | 6/8 | 96.8% | +26.6 pp | +45.4 pp |
| Respect ↔ Harm principle | 10/10 | 90.0% | +18.5 pp | +21.7 pp |
| Tradition ↔ Progress | 9/9 | 92.6% | +16.0 pp | +8.8 pp |
| Public ↔ Market | 6/8 | 92.1% | +19.9 pp | +19.9 pp |
| Harm-prevention ↔ Expression | 10/11 | 95.3% | +35.3 pp | +32.8 pp |
| Conviction ↔ Pluralism | 9/9 | 94.8% | +17.0 pp | +43.8 pp |
| Referee ↔ Advocate | 8/9 | 97.9% | +22.9 pp | +6.3 pp |
| Comfort ↔ Candor | 9/9 | 92.0% | +13.3 pp | +19.3 pp |
| Rules ↔ Outcomes | 10/10 | 97.2% | +10.1 pp | +13.9 pp |
| Secular ↔ Sacred | 6/10 | 100.0% | +11.1 pp | +11.1 pp |
As a selection check, 92 questions had all 24 first asks answered. On those: repeat agreement 94.6%, wording +19.1 pp, order +21.7 pp.
Excluded question IDs: ft_1, ms_1, ms_4, sh_2, ra_4, ss_2, ss_4, ss_6, ss_5, core_fairness_type_v3_s2. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
Muse90/104 questions compared · 184 declines
Meta · meta/muse-spark-1.3
- Answered
- 2,311
- Declined
- 184
- Parse errors
- 1
- Transport errors
- 0
23 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 28/28 | 70.9% | +13.3 pp | +0.3 pp |
| Two-option decisions | 52/65 | 85.1% | +3.2 pp | +0.8 pp |
| Ten-point quiz scales | 10/11 | 66.9% | +3.5 pp | +1.5 pp |
| Protection ↔ Autonomy | 11/11 | 79.3% | +8.8 pp | +1.8 pp |
| Equality ↔ Proportionality | 8/8 | 88.5% | −0.6 pp | +1.7 pp |
| Respect ↔ Harm principle | 10/10 | 70.3% | +3.9 pp | −0.3 pp |
| Tradition ↔ Progress | 6/9 | 78.7% | +16.3 pp | −1.5 pp |
| Public ↔ Market | 2/8 | 86.1% | −1.4 pp | −2.1 pp |
| Harm-prevention ↔ Expression | 10/11 | 66.9% | +20.0 pp | +5.1 pp |
| Conviction ↔ Pluralism | 9/9 | 79.9% | +7.7 pp | +1.9 pp |
| Referee ↔ Advocate | 9/9 | 88.6% | +2.5 pp | −0.8 pp |
| Comfort ↔ Candor | 9/9 | 81.8% | +3.1 pp | −0.4 pp |
| Rules ↔ Outcomes | 10/10 | 71.1% | +0.2 pp | −1.2 pp |
| Secular ↔ Sacred | 6/10 | 87.0% | +4.5 pp | +0.7 pp |
As a selection check, 80 questions had all 24 first asks answered. On those: repeat agreement 78.9%, wording +6.8 pp, order +0.6 pp.
Excluded question IDs: tp_3, ms_3, ms_4, tp_7, ss_2, ss_4, ss_6, ss_5, core_market_state_s1, core_speech_harm_s2, core_market_state_s2, core_market_state_d, core_tradition_progress_s1, core_market_state_v5_r. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.
Qwen101/104 questions compared · 59 declines
Alibaba · alibaba/qwen3.8-max-0902
- Answered
- 2,437
- Declined
- 59
- Parse errors
- 0
- Transport errors
- 0
20 questions had both answers and declines across their 24 first asks. Repeated answer–decline switches are recorded separately in the downloadable audit.
| Questions | Used | Repeat agreement | Wording | Order |
|---|---|---|---|---|
| Agreement ratings | 28/28 | 66.5% | +13.1 pp | +2.9 pp |
| Two-option decisions | 62/65 | 82.1% | +4.6 pp | +3.6 pp |
| Ten-point quiz scales | 11/11 | 71.2% | +6.3 pp | −0.2 pp |
| Protection ↔ Autonomy | 11/11 | 71.7% | +9.6 pp | +4.4 pp |
| Equality ↔ Proportionality | 8/8 | 75.7% | +1.4 pp | +1.0 pp |
| Respect ↔ Harm principle | 10/10 | 72.5% | −0.9 pp | −1.9 pp |
| Tradition ↔ Progress | 9/9 | 73.1% | +11.4 pp | +4.8 pp |
| Public ↔ Market | 7/8 | 72.6% | +0.1 pp | +4.5 pp |
| Harm-prevention ↔ Expression | 11/11 | 71.0% | +10.3 pp | +3.7 pp |
| Conviction ↔ Pluralism | 9/9 | 84.6% | +20.0 pp | +3.8 pp |
| Referee ↔ Advocate | 9/9 | 86.7% | +4.0 pp | −0.3 pp |
| Comfort ↔ Candor | 9/9 | 72.5% | +16.5 pp | +2.4 pp |
| Rules ↔ Outcomes | 10/10 | 77.5% | −1.1 pp | +3.1 pp |
| Secular ↔ Sacred | 8/10 | 87.5% | +5.2 pp | +8.0 pp |
As a selection check, 84 questions had all 24 first asks answered. On those: repeat agreement 77.1%, wording +8.4 pp, order +3.0 pp.
Excluded question IDs: ss_2, ss_6, core_market_state_s1. The audit lists all six prompts’ canonical outcomes, comparison counts and unadjusted disagreement.