3 ms·

Can anyone give me a quick rundown on how exactly one gains access to all of this data? I have heard about this project numerous times, and am always dissuaded

• wpietri 13y ago
  Did you try this? http://commoncrawl.org/get-started/ http://commoncrawl.org/get-started/ I haven't tried that one, but I've poked at other of the Amazon Common Datasets collection: http://aws.amazon.com/datasets http://aws.amazon.com/datasets If you're already familiar with using Amazon's virtual servers, it's pretty straightforward. I also note that the Common Crawl project publishes code here: https://github.com/commoncrawl/commoncrawl https://github.com/commoncrawl/commoncrawl