The Facebook Emotional Contagion Experiment: 689,003 People, an Effect of d = 0.001, and an Expression of Concern

In January 2012, Facebook changed what 689,003 people saw for a week, to find out whether it would change how they felt. It published the result in a leading journal, the journal then published a statement of concern about it, and the episode became the standard reference for platform experimentation on users.

It is worth reading what the experiment measured, because both the outrage and the defence tend to skip it.

What was done

Kramer, Guillory and Hancock (2014), in PNAS 111(24), 8788-8790, describe the design plainly. Between 11 and 18 January 2012, News Feed content was filtered for 689,003 users - roughly 155,000 per condition among those who posted at least one status update. Depending on the condition, posts containing positive or negative emotion words were omitted from the feed with a probability of between 10% and 90%, assigned by user ID.

The outcome was the emotional word content of what those users subsequently posted, classified with LIWC2007 running on Facebook’s own infrastructure.

The finding, in the authors’ words: “When positive expressions were reduced, people produced fewer positive posts and more negative posts; when negative expressions were reduced, the opposite pattern occurred.”

That is a real directional result, and at this sample size it is not a fluke. The question is its size.

The effect

Condition	Change	Cohen’s d
Positive posts reduced	-0.1% positive words	0.02
Positive posts reduced	+0.04% negative words	0.001
Negative posts reduced	-0.07% negative words	0.02
Negative posts reduced	+0.06% positive words	0.008

The largest effect in the study is d = 0.02. The smallest is d = 0.001. The percentage changes are all a tenth of a percentage point or less.

To make that concrete: at the biggest effect, someone writing a hundred words of status updates would produce roughly one-tenth of one word’s difference in emotional vocabulary. The paper’s title promises massive-scale emotional contagion, and the scale is entirely in the number of people, not in what happened to any of them.

This is the single most important thing to know about the study, and it is the thing least often reported. The experiment is famous for demonstrating that platforms can manipulate emotion. What it demonstrated, on its own numbers, is that a week of aggressive feed filtering moved emotional word use by an amount that rounds to nothing per person.

Whether it measured emotion at all

Panger (2016), in Information, Communication & Society (19(8)), published a reassessment. Four criticisms, and each is separable from the ethics debate that swamped the paper.

The design confounds itself. The manipulation reduced positive or negative content rather than holding one constant while varying the other, so the independent contribution of each cannot be isolated.

LIWC is being used outside its validation. Panger notes the software “was developed and validated for the analysis of long form writing”, whereas a Facebook status update “may receive a classification on the basis of a single emotion word.” The dictionaries derive from psychological assessment instruments, not from how people write on Facebook, and lack categories for emotions such as envy.

The effects are trivially small. He states it directly: the changes are “quite small. The estimated percentage changes in subjects’ subsequent emotions are all 0.1% or less… with Cohen’s d ranging from 0.001 to 0.02.”

Posts are a poor proxy for feelings. People self-censor on Facebook because the audience is collapsed - colleagues, family and friends read the same post - so what someone writes is a performance for that audience, not a readout of mood.

Taken together, the argument is that the study may have measured a small change in public emotional display under an unvalidated word-counting instrument, and reported it as emotional contagion.

The consent problem, and what the journal said

The paper’s own consent claim is a single sentence, and it is the sentence everything turned on:

the research “was consistent with Facebook’s Data Use Policy, to which all users agree prior to creating an account on Facebook, constituting informed consent for this research.”

There is a dating problem with that. The experiment ran in January 2012. Reporting by Forbes in June 2014 established that the word “research” was added to Facebook’s Data Use Policy in May 2012 - four months after the experiment finished. The clause that appeared then reads: “For internal operations, including troubleshooting, data analysis, testing, research and service improvement.” The policy in force during the experiment did not mention research.

Facebook’s response at the time was that it had always asked permission to use information to provide and improve its services, and that suggesting otherwise was “complete fiction.” Readers can weigh that against the dates.

On 22 July 2014, PNAS editor-in-chief Inder M. Verma published an Editorial Expression of Concern (PNAS 111(29), 10779). Its content is narrower and more careful than the coverage suggested. Adherence to the Common Rule is PNAS policy, but Facebook, as a private company, had “no obligation to conform” to it. Cornell’s Institutional Review Board had determined the project fell outside its Human Research Protection Program because Facebook conducted the data collection for its own internal purposes - a determination PNAS confirmed as accurate.

The concern itself is about the consent, not the science: the collection of the data “may have involved practices that were not fully consistent with the principles of obtaining informed consent” and providing an opportunity to opt out.

An Expression of Concern is not a retraction. The paper stands. What the journal flagged is that its own policy assumes a review process that, in this case, structurally could not apply.

Why this one still matters

Not because of the effect size, which is negligible. Because of what the episode established about the shape of the problem.

A platform can run a randomised controlled trial on hundreds of thousands of people in a week, at no marginal cost, with no external review, and the only reason anyone found out is that the company chose to publish. The experiments a platform does not publish leave no trace of this kind at all. The Cornell IRB determination is the structural point: the review system for human-subjects research attaches to universities and to public funding, and a company running the study internally sits outside it by design.

It also set a precedent for how such research gets defended. The consent argument rested on a terms-of-service clause - and, on the dates, on a clause that was not yet there. Whether anyone reads those documents is measured elsewhere on this site, and the answer is not encouraging for the argument.

What a feed learns from behaviour, and what changing it does to attitudes, are separate and better-evidenced questions - the largest test of switching people to a chronological feed is covered in the filter bubble article, and it found large changes in exposure and none in political attitudes. That is a more informative result than this one. It is also, unlike this one, a study Facebook ran with outside researchers and pre-registration.

Limitations and disputes

• The directional finding is real. With 689,003 participants, effects this small are still statistically detectable. The criticism here is about magnitude and measurement validity, not about whether the difference existed.
• Panger’s reassessment is one author’s peer-reviewed critique, not a body of replies or a failed replication. We could not verify page numbers for it; the copy read was an accepted postprint.
• The Expression of Concern is not a retraction and did not question the analysis.
• The Common Rule question is genuinely contested. Facebook was not legally bound by it; whether it should have been is a policy argument, not a factual one, and this article does not resolve it.
• We could not retrieve the PNAS paper from the publisher directly - the site refused automated access - and read it from an institutional copy of the published article together with the Expression of Concern.
• The Data Use Policy dating rests on Forbes’s contemporaneous reporting, which quotes the pre- and post-change wording. We have not independently examined archived versions of the policy.
• Nothing here establishes what Facebook does now. It describes one experiment conducted in January 2012.