3 ms·
This ties directly into the comment dataset, made by the same person and via the same method[1]. Working on pulling down the submission data now, will have to s
by tomtoise 11y ago
This ties directly into the comment dataset, made by the same person and via the same method[1]. Working on pulling down the submission data now, will have to spin up another server if I want to do the comment/submission correlation.
>>"This dataset will go nicely with the full Reddit Comment Corpus that I released a couple months ago. The link_id from each comment corresponds to the id key in each of the submission objects in this dataset."
- laumars 11y agoIs the Comment Corpus still available? I have looked but couldn't see anything obvious (my apologies if I'm being thick here)
- tomtoise 11y agoSure, here you go [1] - It unpacks to over 1tb though, hence me considering spinning up another server just for the comments. Might be a good use for my test Hadoop cluster. [1] https://www.reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment/ https://www.reddit.com/r/datasets/comments/3bxlg7/i_have_eve...
- laumars 11y agoThank you. I'll be unpacking this on a FreeBSD server, so hopefully ZFS's lz4 file system compression will take some of the bite out of that 1TB file.