You may be interested in an SQLite version of the dataset that is 553 GB vs. the 908 GB JSON: https://archive.org/details/2015_reddit_comments_corpus_sqlite htt

You may be interested in an SQLite version of the dataset that is 553 GB vs. the 908 GB JSON: https://archive.org/details/2015_reddit_comments_corpus_sqlite https://archive.org/details/2015_reddit_comments_corpus_sqli... The storage format of a dataset can make a big difference in memory usage.