Does Lumosity Work? The FTC Said No, the Evidence Says You Get Better at the Games

You will get better at Lumosity’s games. That is not in dispute and it is not interesting.

The question everyone actually means is whether getting better at the games makes you better at anything else - remembering names, following a conversation, doing your job, staying sharp as you age. That is called far transfer, and it is where the evidence is.

The evidence is not good. And the regulatory record is unusually explicit.

The federal action

On 5 January 2016, the Federal Trade Commission announced a settlement with Lumos Labs, Inc. and two named individuals, its co-founders Kunal Sarkar and Michael Scanlon, in the Northern District of California, case number 3:16-cv-00001-SK.

The stipulated final judgment entered $50,000,000.00 as equitable monetary relief, required payment of $2,000,000.00 within seven days, and provided that “Upon such payment, the remainder of the judgment is suspended.”

The FTC’s charge was deceptive advertising: that the company had claimed its games would improve everyday performance and reduce cognitive decline without adequate evidence.

The order records that “Defendants neither admit nor deny any of the allegations in the Complaint, except as specifically stated in this Order.” A settlement is not a finding of fact against the company, and we are not going to describe it as one.

What the research finds

Owen and colleagues (2010), in Nature, ran the largest single test. 52,617 people registered; 11,430 completed both benchmarking assessments and at least two full training sessions over six weeks.

Their conclusion, verbatim: “These results provide no evidence to support the widely held belief that the regular use of computerized brain trainers improves general cognitive functioning in healthy participants beyond those tasks that are actually being trained.”

We are printing both numbers, because a 78% attrition rate is a real limitation of the study most often cited as decisive, and this article is not going to apply one standard to the industry and another to its critics.

Melby-Lervåg, Redick and Hulme (2016), in Perspectives on Psychological Science, meta-analysed 87 publications containing 145 experimental comparisons of working-memory training.

Against active control groups - the comparison that matters, because it controls for expectation and engagement - the far-transfer effects were:

Outcome	Hedges’ g	95% CI
Nonverbal ability	0.05	[-0.02, 0.13]
Verbal ability	0.05	[-0.07, 0.17]
Word decoding	0.08	[-0.09, 0.24]
Arithmetic	0.06	[-0.08, 0.19]
Reading comprehension	0.15	[0.03, 0.27]

Four of the five are null. Reading comprehension is nominally significant, and the authors attribute it to measurement artefacts. We are printing it anyway, because leaving it out would be exactly the selective reporting this article is criticising.

Their p-curve analysis concluded that “there is no evidential value from the studies of working memory training using active controls.”

Simons and colleagues (2016), in Psychological Science in the Public Interest, reviewed 132 journal articles and reported that “none of the cited studies conformed to all of the best practices we identify as essential to drawing clear conclusions about the benefits of brain training for everyday activities.”

The word “cited” is doing important work in that sentence. Those 132 papers were not a neutral literature sample. They were the articles the brain-training industry itself cited as its evidence base on its own websites. Simons and colleagues reviewed the industry’s own best case and found that none of it met the full methodological bar.

Sala and colleagues (2019), in Intelligence, found far transfer against active controls in older adults at ḡ = -0.008.

The trial that found a benefit

There is one, and it is the most instructive document in the whole area.

Hardy, Nelson, Thomason, Sternberg, Katovich, Farzin and Scanlon (2015), in PLOS ONE, randomised 4,715 participants, used an active control (crosswords), and found a positive effect of d = 0.255.

That is a large, randomised, actively controlled trial with a positive result. On design alone it is better than most of what is discussed above.

Now the disclosure, which PLOS ONE’s policy required the authors to make, verbatim:

“Lumos Labs, Inc. funded the research through the development of its software tools. JLH, DAS, KK, FF and MS are employed at Lumos Labs, the company that produces the cognitive training program Lumosity that is used in this study. These authors hold stock options in the company. RAN works as a consultant for Lumos Labs. MET is on the Scientific Advisory Board of Lumos Labs and holds stock options in the company.”

Five of the seven authors were Lumos Labs employees holding stock options. The sixth was a paid consultant. The seventh sat on the scientific advisory board and also held stock options. Every author had a financial interest in the outcome.

And one detail closes the loop. Co-author Michael Scanlon is the same Michael Scanlon named personally as a defendant in the FTC action four months later - the company’s co-founder and former Chief Scientific Officer. The author of the flagship positive trial became a named defendant in a federal deceptive-advertising case concerning the claims that trial supported.

We are not alleging that the study is wrong. We are saying that a reader deciding what to believe should know who ran it.

The strongest evidence that training can transfer

To be fair to the field, there is a genuinely independent trial with a positive long-term result, and it deserves to be represented precisely.

The ACTIVE trial, reported at ten years by Rebok and colleagues (2014) in the Journal of the American Geriatrics Society, randomised 2,832 older adults to reasoning training, speed training, memory training, or control. Retention at ten years was 44%.

What persisted:

• Reasoning training: 73.6% of trained participants were still performing above their pre-trial baseline, against 61.7% of controls.
• Speed training: 70.7% were performing at or above baseline, against 48.8% of controls.

What did not:

• Memory training: no difference from control after ten years.

And the caveat that matters most. Participants in the training groups reported less difficulty with everyday tasks than controls - but standard performance-based tests of function conducted by the researchers showed no difference between the groups. The everyday-life benefit exists in what people said about themselves and disappears on objective measurement. The researchers’ own position was that the self-reports are interesting but that it is not yet known whether they would let older people live independently for longer.

There is one more thing to say about ACTIVE, and it is the crucial one for this article’s question: it trained specific cognitive abilities under research conditions. It did not test a commercial product. It is not evidence that Lumosity works.

The two open letters

In October 2014, the Stanford Center on Longevity and the Berlin Max Planck Institute for Human Development published a consensus statement from the scientific community expressing scepticism about brain-training claims. In December 2014, a rebuttal letter appeared at cognitivetrainingdata.org.

Both letters remained open for signature and their counts grew over time, so any figure needs a date attached. Simons and colleagues, writing in 2016, record the consensus letter as “more than 70 psychologists and neuroscientists” and the rebuttal as “133 scientists and therapists.” The rebuttal was announced in December 2014 with 127 signatories.

One thing worth noting about the rebuttal: its most visible public defender is the chief executive of a company that sells a competing brain-training product, which hosts a response citing its own product’s peer-reviewed papers. The letter defending the industry is amplified by the industry.

What this leaves you with

You will improve at the trained tasks. Nobody disputes this, and it is what the practice effect predicts.

Far transfer to untrained abilities is not supported against active controls, across 87 publications and 145 comparisons, with a p-curve indicating no evidential value.

The largest single test, with 11,430 completers, found no transfer - with the honest caveat that most registrants dropped out.

The one strong positive trial had seven authors and seven financial interests.

The best independent evidence, ACTIVE, is real but narrower than it sounds: reasoning and speed training persisted at ten years, memory training did not, and the everyday-function benefit was self-reported only and vanished on objective testing.

None of this is a claim about any medical condition, and nothing here is medical advice. No study discussed above tested treatment or prevention of any diagnosis.

For what spacing actually delivers and where its popular rules break down, see spaced repetition. For an instrument that turned out to measure its own questionnaire, see nomophobia. For a memory technique that did not survive replication, see Sans Forgetica.

Limitations of this article

We report a settlement, not a judicial finding. The defendants neither admitted nor denied the allegations.

We could not confirm the exact signatory count of the 2014 consensus letter. Secondary sources give “nearly 70” and “more than 70.” We have used the range with its date rather than a precise number. See the withheld-figures record.

We could not establish who organised the rebuttal letter.

Formal published replies exist between the meta-analysis authors and their critics in Psychonomic Bulletin & Review. These are scientific disputes, not corrections or expressions of concern, and we found no correction, retraction or expression of concern attached to any paper cited here.

We did not search PubPeer, whose access policy blocked automated retrieval.