3 ms·

Ask HN: What is the largest dataset you know of?

• wizzerking 5y ago
  The mass spectral dataset of today is 12 petabytes
• melissalobos 5y ago
  C4 uncleaned is ~ 2.3 TB (https://huggingface.co/datasets/c4 https://huggingface.co/datasets/c4) which is a huge amount of only text. Otherwise I have worked with a video hosting site that had more than 100's of TBs of data. It occurs to me that there are several measures of biggest. Raw size, number of rows, etc. which did you mean?
  • pizza 5y ago
    > It occurs to me that there are several measures of biggest. Raw size, number of rows, etc. which did you mean? Hm good point. I guess my cop-out answer is "the most informative one." Maybe "the largest description of some aspect of the world." If you've heard of the saying "the map is not the territory" then I guess I am asking, in a general sense, what is the "biggest map"?