Back to the directory Find in the timeline

Boeker and Urman's sock-puppet audit of TikTok personalization

Researchers ran scripted accounts on the TikTok website to test what changes the For You feed. Following a creator had the strongest effect. The abstract and main text disagree on the order of liking and watch time. This is a limited web audit, not a full test of the algorithm.

January 2022. Date of the arXiv preprint (v1, 28 January 2022), the version read. The arXiv page says it was accepted for the Web Conference 2022. The experiments ran in 2021. Chronology section: 2010 to 2026.

StudyIndependent experimentPlatforms

Where it sits: Academic psychologyResearch published for other researchers and students.
Evidence level: Independent experimentIndependent research, with its design and limits stated.
Organizations: TikTokAn organization is listed where the sources concern it. The text says exactly how.
Sources inspected: 2 of 21 with the relevant text, company document or archive item checked, 1 as an abstract or reference article, 0 not verified.

What happened

Maximilian Boeker (University of Zurich and Technical University of Munich) and Aleksandra Urman (University of Zurich) posted their audit as a preprint on 28 January 2022, and its arXiv page says it was accepted for the Web Conference 2022. Between late June and mid-August 2021 they ran scripted accounts on the TikTok website and compared each account's For You feed with a twin that only scrolled. They collected data on 30,436 posts across 39 completed scenarios.

The psychological idea

A sock-puppet audit uses programmed accounts to isolate one factor at a time. Each scenario paired an active bot, which followed, liked or watched longer, with a control bot that scrolled for the same time. Differences larger than the noise between identical controls were read as personalization. The authors varied language, location, likes, follows and how long a video was watched, from 25 to 400 percent of its length.

In practice

Every tested factor changed the feed. The authors rank following a creator as the strongest influence, then watching for longer, then liking, and say the gap between the last two is small. Location mattered more than language. Watching posts chosen to fit a persona changed feeds more than watching random posts, and watching 25 percent of videos changed them more than watching 75 percent, which the authors could not fully explain. Identical control accounts already saw feeds in which about two thirds of posts differed.

Why it mattered

It is an early test, run from outside the company, of which user actions change TikTok's feed. The authors say their data do not confirm the Wall Street Journal's finding, because following a creator influenced the feed more than the other factors tested. They argue that explicit actions such as following give users more control than lingering does.

Limits and qualifications

This is not a full audit of the algorithm. The authors call it a first step, say findings may hold only for a short window because TikTok changes its system, and note that it tested only the web version, with few bots and only explicit factors, not commenting or sharing. Scripted accounts are not real viewers. The abstract ranks likes above view rate; the body ranks view rate above likes.

Evidence behind the claims

The authors ran a sock-puppet audit with a custom bot on the TikTok website in incognito mode, pairing an active user with a control user, with about 20 runs per scenario, between late June and mid-August 2021, collecting data on 30,436 posts across 39 successfully completed scenarios.
Experimental findingSource: [1]Where: Sections 3.1, 3.2 and 4
In 11 control scenarios, where neither of two identical users interacted with the feed, the feeds still differed: roughly 66 to 70 percent of posts differed, depending on whether three or five batches were collected.
Experimental findingSource: [1]Where: Section 4.1, Controlling Against Noise
The authors report that all tested factors (language, location, likes, follows, video view rate) influenced recommendations, with following the strongest, then video view rate, then likes, and location a stronger factor than language; the arXiv abstract orders likes before view rate.
Experimental findingSources: [1] [2]Where: Sections 4.2 and 4.6 and 6; arXiv abstract page
Video view rate was varied between 25 and 400 percent of video length; watching posts matching a set persona changed feeds more than watching random posts, and feeds of the user who watched only 25 percent of posts differed more than those of the user who watched 75 percent (average difference 0.85 percent against 0.56 percent), which the authors explain only tentatively.
Experimental findingSource: [1]Where: Section 4.5, Video View Rate
The authors write that they cannot confirm the Wall Street Journal's findings because their data show following specific creators influences the feed more than the other tested factors.
Experimental findingSource: [1]Where: Section 4.6, Concluding Results
The authors state their analysis is not exhaustive and is a first step, that findings may be accurate only for a small time window because TikTok changes its algorithm, that they used only a few agents on the web version, and that it includes only the most explicit factors, not commenting or sharing.
Experimental findingSource: [1]Where: Introduction, Section 3.4 and Section 6, Conclusion
The arXiv page for the preprint shows submission on 28 January 2022 and the note that it was accepted for publication at the Web Conference 2022.
DocumentedSource: [2]Where: arXiv abstract page, submission history and commentsChecked at abstract level

People involved

Maximilian Boeker. Researcher, University of Zurich and Technical University of Munich
Aleksandra Urman. Researcher, University of Zurich

In the Psy101 course

OpenStax Psychology 2e, section 2.2: Approaches to ResearchChapter 2, Psychological Research. Method. The section discusses this topic.Opened the page: it says experiments give a high degree of control over variables, and that correlational data cannot show cause and effect. It does not mention control groups by name.

This is course relevance only. It is not evidence for the historical or corporate claims on this page.See the full course map.

Sources and inspection

2 of 2 sources were opened and read at the level shown. A source marked as not verified supports no claim on this page by itself.

[1]
An Empirical Investigation of Personalization Factors on TikTokBoeker, M; Urman, A. arXiv 2201.12271 (v1), 2022. Preprint. Primary source.Relevant text checkedReviewer opened the arXiv PDF (v1) and read pages 1 to 9 as page images: abstract, introduction, related work, methods, all experiment sections, discussion and conclusion. The earlier review did not inspect supplementary tables. A Codex follow-up found appendices A to C in the current 12-page v1 PDF; their availability is no longer described as missing.
[2]
An Empirical Investigation of Personalization Factors on TikTok (arXiv abstract page)Boeker, M; Urman, A. arXiv, 2022. Preprint. Primary source.Abstract or publisher record onlyReviewer read the arXiv abstract page: abstract, submission date and the accepted-at-Web-Conference-2022 comment. The abstract orders the factors follow, like, then video view rate, which differs from the paper body.

Access note: The Web Conference version was not opened. The arXiv v1 PDF currently includes appendices A to C; the earlier review covered the main text, not an independent recalculation from those tables.

Independent source review: Claude review agent, independent of the research agent, 2026-10-21. Verdict: pass-with-edits.