3 ms·
If you can crawl flickr and get 50TB of data, do it.... it is more than a "were I so inclined" situation. I have had a very hard time crawling and indexing larg
by kastnerkyle 12y ago
If you can crawl flickr and get 50TB of data, do it.... it is more than a "were I so inclined" situation. I have had a very hard time crawling and indexing large datasets like this - companies tend to protect their data!