33 ms·
This would likely make a great natural language data set for compression algorithms.
by JoshTriplett 12y ago
This would likely make a great natural language data set for compression algorithms.
- pipeep 12y agoTrue, but doesn't Twitter already provide an API for access to a fraction of the firehose? Surely that would be enough data. If Twitter doesn't have a good API, Reddit allows full access to all comments through their API (although Reddit has orders of magnitude less data).
- minimaxir 12y agoTwitter's API is too limited for historical data. (you'll hit the rate limits quickly for any meaningful volume). Reddit's rate limits, however, let you process a million comments every day.
- hyperbovine 12y ago@JoshTriplett tweets r alrdy #compressed. hth
- loceng 12y agoUpvoted for much needed humour on HN.