Introduction to Psychology
Popular Psychology

Back to the directoryFind in the timeline

How large are nudge effects? Meta-analyses and nudge-unit trials

A 2022 meta-analysis of choice-architecture studies reported a pooled effect of d = 0.45. Critics said publication bias inflated it. A separate comparison found academic nudge papers showed larger effects than trials run by two government nudge units.

2022. Marks the year of the Mertens meta-analysis, the DellaVigna and Linos paper (Econometrica, volume 90, issue 1) and the August 2022 PNAS letters. Chronology section: 2010 to 2026.

StudyContestedDecisions

Where it sits
Academic psychologyResearch published for other researchers and students.
Evidence level
ContestedThe evidence conflicts or is disputed, for example after failed replications.
Sources inspected
4 of 43 with the relevant text, company document or archive item checked, 1 as an abstract or reference article, 0 not verified.

What happened

In 2022 two large reviews of nudge research appeared. Mertens and colleagues pooled 455 effect sizes from 214 publications on choice architecture in PNAS. DellaVigna and Linos, in Econometrica, compared 126 trials run by two US government nudge units with nudge papers from academic journals. In August 2022 PNAS printed letters by Maier and colleagues and by Szaszi and colleagues, arguing that publication bias had inflated the pooled result.

The psychological idea

Here a nudge is a change to how choices are presented, without banning options or changing prices. A meta-analysis pools effect sizes across studies. Cohen's d expresses a difference in standard-deviation units. Publication bias arises when small or null results are less likely to be published, which inflates the average of what appears. Correction methods try to estimate the average without that filter.

In practice

Mertens and colleagues report a pooled d of 0.45. Food choices showed the largest effect (d = 0.72) and finance the smallest (d = 0.25). They found moderate publication bias, and adjusting for it gave d = 0.31. Maier and colleagues put the adjusted overall estimate at 0.04. DellaVigna and Linos found an average take-up effect of 8.7 percentage points in academic papers and 1.4 points in nudge-unit trials, and estimate that selective publication explains about 70 per cent of the gap.

Why it mattered

The papers answer the size question in three ways: a pooled average of 0.45, an adjusted average near zero, and a nudge-unit average of 1.4 percentage points. The nudge-unit trials covered all trials the units ran and still showed a positive average. Mertens and colleagues say their overall conclusion holds after adjustment, while Szaszi and colleagues see no reason to expect large and consistent effects.

Limits and qualifications

These are statistical reviews, not new experiments. Mertens and colleagues excluded studies that combined nudges with incentives or education and included only English-language work. Heterogeneity was very high. Correction methods rest on assumptions, and the letters give adjusted values from -0.01 to 0.11 depending on the method. The nudge-unit trials come from two US organizations and measure take-up, so they do not represent all nudges or consumer marketing.

Evidence behind the claims

  • Mertens and colleagues synthesised 455 effect sizes from 214 publications and reported a random-effects estimate of d = 0.45, 95% CI 0.39 to 0.52.

    DocumentedSource: [1]Where: Results, overall effect

  • Mertens and colleagues found food had the largest domain effect (d = 0.72) and finance the smallest (d = 0.25); they reported publication bias, with an adjusted estimate of d = 0.31 under moderate bias, and said their overall conclusion still holds; heterogeneity was very high (I-squared 99.67 per cent); they excluded studies combining nudges with incentives or education and included only English-language publications.

    DocumentedSource: [1]Where: Results by domain, publication bias, heterogeneity and limitations

  • Maier and colleagues re-analyzed the corrected dataset with robust Bayesian meta-analysis and reported an adjusted combined estimate of 0.04 [0.00, 0.14] against an unadjusted 0.43 (0.11 using only the most precise estimate per study), concluding no evidence remained that nudges are effective after correction.

    DocumentedSource: [2]Where: Key findings and conclusion

  • Szaszi and colleagues reported bias-corrected estimates of d = -0.01 (Andrews-Kasy), 0.07 (WAAP) and 0.08 (trim and fill) for the same data, said they saw no reason to expect large and consistent effects, and recommended focusing on when and where nudges produce large effects.

    DocumentedSource: [3]Where: Main text of the letter

  • DellaVigna and Linos analyzed 126 randomized trials from two US nudge units and found an average take-up effect of 1.4 percentage points, against 8.7 points in academic journal papers.

    DocumentedSource: [4]Where: Publisher page summary of the abstractChecked at abstract level

  • DellaVigna and Linos estimate that selective publication, worsened by low statistical power, explains about 70 per cent of the difference between the two samples.

    InterpretationSource: [4]Where: Publisher page summary of the abstractChecked at abstract level

People involved

  • Stephanie Mertens. Lead author of the meta-analysis
  • Maximilian Maier. Lead author of the publication-bias letter
  • Barnabas Szaszi. Lead author of a critical letter
  • Stefano DellaVigna. Co-author of the nudge-unit comparison
  • Elizabeth Linos. Co-author of the nudge-unit comparison

In the Psy101 course

This is course relevance only. It is not evidence for the historical or corporate claims on this page.See the full course map.

Sources and inspection

4 of 4 sources were opened and read at the level shown. A source marked as not verified supports no claim on this page by itself.

  1. [1]
    The effectiveness of nudging: A meta-analysis of choice architecture interventions across behavioral domainsMertens, S; Herberz, M; Hahnel, U; Brosch, T. Proceedings of the National Academy of Sciences (via Europe PMC, PMC8740589), 2022. Journal article. Primary source.Relevant text checkedRead the open-access full text from Europe PMC as returned by the fetch tool (first 100,000 of about 160,000 characters: sample, overall effect, domains, publication bias, heterogeneity, exclusions, language). The remainder was not read.
  2. [2]
    No evidence for nudging after adjusting for publication biasMaier, M; Bartos, F; Stanley, T; Shanks, D; Harris, A; Wagenmakers, E. Proceedings of the National Academy of Sciences 119(31) (via Europe PMC, PMC9351501), 2022. Journal article. Critique or replication.Relevant text checkedRead the open-access letter text from Europe PMC as returned by the fetch tool: methods, adjusted and unadjusted estimates and conclusion. Print date 2 August 2022 (volume 119, issue 31).
  3. [3]
    No reason to expect large and consistent effects of nudge interventionsSzaszi, B; Higney, A; Charlton, A; Gelman, A; Ziano, I; Aczel, B; Goldstein, D; Yeager, D; Tipton, E. Proceedings of the National Academy of Sciences (via Europe PMC, PMC9351519), 2022. Journal article. Critique or replication.Relevant text checkedRead the open-access letter text from Europe PMC as returned by the fetch tool: bias-corrected estimates and conclusion.
  4. [4]
    RCTs to Scale: Comprehensive Evidence from Two Nudge UnitsDellaVigna, S; Linos, E. Econometrica 90(1), 81-116, 2022. Journal article. Primary source.Abstract or publisher record onlyRead the publisher page summary of the abstract. The full paper was not opened, so only abstract-level claims are made.

Access note: DellaVigna and Linos was read at abstract level only. The Mertens, Maier and Szaszi texts were read through the Europe PMC open-access XML as returned by the fetch tool.

Independent source review: Claude review agent, independent of the research agent, 2026-10-21. Verdict: pass-with-edits.