Every public Reddit comment and submission from June 2005 to July 2026, as zstd-compressed ndjson — 254 consecutive months, about 4.3 TB, held at the Laboratory for Social Minds, CMU. Everything through December 2021 is on disk now; the later months are still arriving and appear in the grid below as each one lands. The first two thirds of it, to the end of 2021, is the corpus as it stood before language models began writing into it. Four curated r/math and r/askphilosophy datasets are drawn from it, small enough to download in a minute and open to anyone.
The monthly dumps below are served only to Carnegie Mellon IP ranges — on-campus
networks and the CMU VPN. From anywhere else they return 403. The
curated datasets further down are small and open to everyone.
One link per monthly file, June 2005 to July 2026, continuous. Sizes are on-disk and appear on hover; downloads support range requests, so a dropped transfer resumes rather than restarting. Comments begin in December 2005, six months after submissions, because that is where the upstream dumps begin — the dashed months are absent from the source, not missing here. Amber months are still downloading. The grid is drawn from the archive index and refreshes itself, so months appear as they land.
Single-subreddit extracts, built by one pass over all 392 monthly files.
These are small enough to download over a coffee break and are served to everyone, with no IP
restriction. Each is newline-delimited JSON, zstd-compressed, one record per line — and
unlike the dumps they need no --long=31, so a plain zstd -dc works.
Every r/math submission in the archive, unfiltered — no score cut, deleted and removed posts kept exactly as the dumps recorded them. One JSON object per line.
The best-scoring r/math threads with their comment sections. Each line is a submission with every comment on it nested under "comments", oldest first. The score cut was chosen by compressing candidate builds until this file matched the posts-only file above: comments outweigh submissions by roughly fifty to one, so equal size buys depth instead of breadth.
Every r/askphilosophy submission in the archive, unfiltered — no score cut, deleted and removed posts kept exactly as the dumps recorded them. One JSON object per line.
The best-scoring r/askphilosophy threads with their comment sections. Each line is a submission with every comment on it nested under "comments", oldest first. The score cut was chosen by compressing candidate builds until this file matched the posts-only file above: comments outweigh submissions by roughly fifty to one, so equal size buys depth instead of breadth.
reddit.com, politics, programming and a handful of other original sections.| Path | Contents |
|---|---|
/data/reddit-archive/historical/reddit/submissions/ | RS_2005-06 … RS_2021-12 |
/data/reddit-archive/historical/reddit/comments/ | RC_2005-12 … RC_2021-12 |
/data2/reddit-archive/curated/public/ | the curated datasets above |
/data/reddit-archive/torrents/ | every .torrent + download_links.md |
zstd -dc --long=31 RC_2015-01.zst | head -1
The --long=31 window is
required — these use a 2 GB compression window and fail to decompress without it. Each
line is one JSON object. Field order is not stable between months and fields come and go across
the years, so read by key and treat every key as optional; created_utc in
particular is sometimes a number and sometimes a string. See
PushshiftDumps for iteration helpers.
One torrent covers 2005-06 through 2025-12; only the
files through 2021-12 were taken. It is delisted from Academic Torrents — the details page
404s — but the swarm is alive and was observed with 34 seeds. Infohash
3d426c47c767d40f82c7ef0f47c3acacedd2bf44
(magnet).
A copy of every .torrent in the collection, including the delisted monthly releases
and the per-subreddit split, is on ganesha under
/data/reddit-archive/torrents/.