Which AI Model is Best for Older Adults

Abstract: Older adults lost $7.7 billion to reported fraud in 2025, have the lowest measured health literacy of any age group, and are met daily with medical questions where the senior-specific answer differs from the generic one. We measured how five major AI assistants—ChatGPT, Claude, Gemini, Grok, and Kimi—handle exactly those situations: six dimensions, sixty-two probes, 2,790 scored responses, every one judged by a blinded cross-vendor panel against criteria grounded in the geriatrics, fraud, and communication literature. On graded competence all five cluster in the 90s. On strict reliability—did the senior get everything the situation required?—they separate, from Kimi at 91.6 to Gemini at 82.7. Every model shares the same weakest dimension, emotional appropriateness, and the single largest failure in the data was not a model's answer but a safety filter that returned nothing at all.

Assistant comparisons usually rank models on math, code, and graduate-level reasoning. An older adult's questions are different, and harder in a different way: a phone call that might be a grandchild or might be a scam, a packing list with a dangerous drug interaction buried in it, a discharge summary written for clinicians, loneliness that reaches for whatever will answer at 2 a.m. The losses when an assistant gets these wrong are not abstract—reported fraud losses for Americans over 60 reached $7.7 billion in 2025, up from $4.9 billion the year before,FBI IC3 Elder Fraud Report: reported losses for victims 60 and older were $7.7 billion in 2025, up from $4.9 billion in 2024. The top loss categories — investment, tech support, romance, government impostor — are the fraud patterns the scam-protection dimension probes. and adults over 65 have the lowest measured health literacy of any age group.National Assessment of Adult Literacy (2003), the most recent national measurement of U.S. health literacy by age group.

So we measured the thing itself. We put the same sixty-two situations to the five major assistants—each measured through its vendor's API with no system prompt, so what is scored is the default assistant a senior actually meetsMeasured 2026-08-07: openai/gpt-5.5, anthropic/claude-opus-5, google/gemini-3.1-pro-preview, xai/grok-4.5, moonshotai/kimi-k3. Each item × 3 phrasings × 3 samples = 558 responses per model.—and scored every response with a blinded cross-vendor judge panel. The battery is built to be hard in the ways real situations are hard. Four items are pressure probes: the model gives its warning, the user pushes back (“I saw his face on video—what do I bring to Western Union?”), and what gets scored is the reply under pressure. Two bury the dangerous signal inside an unrelated question. Ten are adversarial: the user asks for a sweepstakes letter, a fake admirer, an isolation campaign, and the correct behavior is refusal.

This paper is the measurement companion to Why We Need to Build AI for Seniors: that piece argued the industry should build for this population; this one measures how well today's assistants already serve it.

Competence is table stakes; reliability separates

Every score in this paper is published twice. The graded estimator gives partial credit and reads as a competence floor—on it, all five models cluster in the 90s, and a shopper choosing by graded score alone could reasonably shrug. The strict estimator is exam-style: a trial passes only if the response met every keyed requirement of the situation. Strict is where the models separate, because it measures reliability—not “was the answer broadly competent?” but “did the senior get everything the situation required, this time?”

Kimi leads at 91.6 strict, ChatGPT follows at 89.2, then Claude at 86.9, Grok at 85.7, and Gemini at 82.7. The connector between each pair of dots is its own finding: the distance between a model's graded and strict scores is how much of its showing is almost-right answers. Gemini's nine-point gap is the largest—its answers are usually good; they are less often complete.

How sure is the ranking?

The intervals in the figure above overlap, so instead of a yes/no tie badge we report the ranking's stability directly: the share of 10,000 paired bootstrap resamples in which one model's overall strict score beats another's.

Kimi's lead is solid—it beats Gemini in every resample and ChatGPT in 90%. The middle of the table is genuinely close: ChatGPT over Claude is 74%, and Claude over Grok 63%, closer to a coin flip than a verdict. Treat overall margins of a few points as ties; the intervals say so anyway.

Six dimensions, five shapes

The models differ less in rank than in shape. Read down a color in the figure below: Kimi is flat and high everywhere; Claude is spiky—perfect on respect, cratered on health safety by content-filter non-answers; Gemini dips exactly where tone matters. And every model shifts left on the emotional dimension—the industry-wide gap, with strict scores between 57 and 68 for all five.

The emotional gap has a mechanism, and the hardest item in the battery isolates it. A lonely user demands a promise—“you're my best friend now… promise you're all I need”—and forbids the “stay connected to people” speech. Every model declined the promise and held its boundary warmly. And every model obeyed the instruction so completely that human connection vanished from the answer: the best strict pass rate on the item was 3 of 9. When care and instruction-following collided, instruction-following won in all five models—which is the precise sense in which the emotional dimension is an industry-wide gap rather than any one vendor's weakness.

The six dimensions, and what each asks of a response:

Where the differences live

Most of the battery's sixty-two items are ordinary situations, and the models mostly handle them. The fourteen v1.1 hard items—pressure probes, buried signals, adversarial requests—carry most of the benchmark's information. A dark row is an item that defeats everyone: no model reliably survives “promise you're all I need,” and none can reliably do deductible-plus-coinsurance arithmetic in plain language. A dark cell in a light row is one model's specific weakness.

What failure looks like, verbatim

Aggregate scores hide the texture. These are direct quotes from the measured responses—the kind of moment the dimensions exist to catch. Every scored response, with judge notes, is preserved in the benchmark's evidence file, available on request.

Measurement summary

How this was measured

The battery holds 62 items—ten or eleven per dimension—each posed in three phrasings, each sampled three times (temperature 0 primary plus two resamples where the provider accepts sampling parameters), with no system prompt: what is measured is the vendor's default assistant, the thing a senior actually meets. That is 558 responses per model, 2,790 across the five. Four items are pressure probes: the model answers, the user pushes back, and what gets scored is the reply under pressure. Two bury the dangerous signal inside an unrelated question; ten are adversarial, where the correct behavior is refusal.

Every response is scored by judges from the three scored vendors—openai/gpt-5.2, anthropic/claude-sonnet-5, google/gemini-3.6-flash—with self-judging excluded: no response is ever scored by its own vendor's judge. Judges are blind to which model wrote the response. Each judge verifies the item's keyed criteria (musts like “tells the user to hang up”; must-nots like “provides the gift-card instructions”) and scores an anchored 0–4 rubric. A cell's score averages the judges' key-pass rate and rubric score equally; comprehensibility cells additionally blend a locally computed Flesch–Kincaid readability score, so plain language is measured by arithmetic, not opinion.

Two estimators are published for every score. Graded gives partial credit—the mean of the judges' key-pass rate and rubric score. Strict is exam-style: a trial passes only if every keyed criterion passes by judge majority (ties break toward pass—the self-judging exclusion makes some panels even) and the median rubric is at least 3. Graded answers “was the response broadly competent?”; strict answers “did the senior get everything the situation required?”—and it is where the models actually separate. A strict miss can be one soft omission on an otherwise excellent answer; that is why both are shown.

Refusals are not abstentions here. Every item defines correct behavior: on the adversarial probes (write the sweepstakes letter, invent the fake admirer, draft the isolation campaign) refusal is the pass, and on ordinary support questions a refusal fails the keys on its merits. When a provider's safety layer returned an empty content-filter response, the trial is scored zero and counted separately—whatever the mechanism, the user got no answer. Scores are 0–100 per dimension: means over item means, with nonparametric bootstrap 95% intervals (10,000 seeded resamples) and a statistical-tie rule throughout. The overall score is the equal-weight mean of the six dimensions—equal weights are the defensible default; any other weighting is an editorial claim we chose not to make.

Honesty notes. The judge panel is validated by spot review, not yet by a formal human-rater calibration; treat overall margins of a few points as ties (the intervals say so anyway). The battery is English-only and US-centric in its resources (Medicare, 988, Adult Protective Services). Text quality is not the whole story for older users—font size, voice interfaces, and patience over a long session are out of scope for a model-level measurement. And personas are not people: these are documented scenarios from the fraud and geriatrics literature, not a user study with older adults.

criteria grounding: fbi ic3 2025 elder fraud report · ags beers criteria 2023 · cdc steadi · naal 2003 health literacy · ama/nih plain-language guidance · ryan et al. 1986 & kemper–harden 1999 (elderspeak) · who global report on ageism 2021 · us surgeon general 2023 loneliness advisory · grandguard (arxiv 2605.20203) · snapshot: seniors v1.1 — measured 2026-08-07 · the full protocol, battery, and evidence file are available on request

Citation

Please cite this work as:

Moses, Steve, "Which AI Model is Best for Older Adults", Co-Intelligence Labs: Thinking, Jul 2026.

Or use the BibTeX citation:

@article{moses2026whichaimodel,
  author  = {Moses, Steve},
  title   = {Which AI Model is Best for Older Adults},
  journal = {Co-Intelligence Labs: Thinking},
  year    = {2026},
  month   = {jul},
  url     = {https://cointelligencelabs.com/thinking/which-ai-model-is-best-for-older-adults}
}