5 ms·

45GB of data could be like a dozen employees' or less Outlook PST files. For this to be astounding we would need to know the quality of the data. Otherwise it i

45GB of data could be like a dozen employees' or less Outlook PST files. For this to be astounding we would need to know the quality of the data. Otherwise it is a bunch of hype and hoopla.

campbel 3y ago
You better pay up or we'll delete all of Marge and Victors email backups!
anitil 3y ago
I'm not sure about the legality and ethics of training models on stolen data, but for reference, but so far as I can tell the Enron email data set is about 1.5GB (much lower than I expected to be honest!). And I believe some of the more interesting things found in that data set (outside of the fraud) were people cheating on their partners. https://www.kaggle.com/datasets/wcukierski/enron-email-dataset/data https://www.kaggle.com/datasets/wcukierski/enron-email-datas...
- justsocrateasin 3y ago
  I do believe that 1.5GB is tarred and gzipped though, so it is a fair bit bigger. That's also supposedly half a million emails, so 45gb is quite a bit.
- primax 3y ago
  Sure, but that was leaked in 2004. A very different time for email, and where attachments were generally smaller.
  - jmalicki 3y ago
    The Enron dataset was not leaked - it was made public by the US government as evidence from an investigation by the Federal Energy Regulatory Commission. https://www.ferc.gov/electric/industry-activities/addressing-2000-2001-western-energy-crisis/information-released-enron-investigation https://www.ferc.gov/electric/industry-activities/addressing...
    - anitil 3y ago
      Yes my understanding is that this is why it's such a common training set - it's interesting, large(ish) and free and legal with no licence requirements. I think it's public domain? But I'm not from the US, so don't quote me.
- freedude 3y ago
  It depends upon the size of the individual email. In the early 2000's email attachments were less common and on average smaller in size. Today, 20 years later, it is common in my line of work to see email attachments in the 5-10MB range. Pass them back and forth a couple of dozen times for changes and approvals and it doesn't take long for it to balloon. Yesterday, I worked on an outlook email issue on a PST file that contained 40,000 emails and was over 10 GB in size. His PST was new less than two years ago. It really depends upon how it is used, misused or abused.
  - anitil 3y ago
    You're absolutely right. That's one thing that I didn't follow up on, I only briefly looked at the csv in the kaggle page and didn't see anything that looked like images.
deleted 3y ago
[deleted]
seventytwo 3y ago
I was just gonna say, there’s probably single CFD simulation files that are larger than 45GB.
justinclift 3y ago
Could be the porn stash of the Boeing directors. That could make some of them pretty nervous. ;)
JCharante 3y ago
or it could be a single finite element analysis file

45GB of data could be like a dozen employees' or less Outlook PST files. For this to be astounding we would need to know the quality of the data. Otherwise it is a bunch of hype and hoopla.

campbel 3y ago

You better pay up or we'll delete all of Marge and Victors email backups!

anitil 3y ago

I'm not sure about the legality and ethics of training models on stolen data, but for reference, but so far as I can tell the Enron email data set is about 1.5GB (much lower than I expected to be honest!). And I believe some of the more interesting things found in that data set (outside of the fraud) were people cheating on their partners. https://www.kaggle.com/datasets/wcukierski/enron-email-dataset/data https://www.kaggle.com/datasets/wcukierski/enron-email-datas...

justsocrateasin 3y ago
I do believe that 1.5GB is tarred and gzipped though, so it is a fair bit bigger. That's also supposedly half a million emails, so 45gb is quite a bit.
primax 3y ago
Sure, but that was leaked in 2004. A very different time for email, and where attachments were generally smaller.
- jmalicki 3y ago
  The Enron dataset was not leaked - it was made public by the US government as evidence from an investigation by the Federal Energy Regulatory Commission. https://www.ferc.gov/electric/industry-activities/addressing-2000-2001-western-energy-crisis/information-released-enron-investigation https://www.ferc.gov/electric/industry-activities/addressing...
  - anitil 3y ago
    Yes my understanding is that this is why it's such a common training set - it's interesting, large(ish) and free and legal with no licence requirements. I think it's public domain? But I'm not from the US, so don't quote me.
freedude 3y ago
It depends upon the size of the individual email. In the early 2000's email attachments were less common and on average smaller in size. Today, 20 years later, it is common in my line of work to see email attachments in the 5-10MB range. Pass them back and forth a couple of dozen times for changes and approvals and it doesn't take long for it to balloon. Yesterday, I worked on an outlook email issue on a PST file that contained 40,000 emails and was over 10 GB in size. His PST was new less than two years ago. It really depends upon how it is used, misused or abused.
- anitil 3y ago
  You're absolutely right. That's one thing that I didn't follow up on, I only briefly looked at the csv in the kaggle page and didn't see anything that looked like images.

deleted 3y ago

[deleted]

seventytwo 3y ago

I was just gonna say, there’s probably single CFD simulation files that are larger than 45GB.

justinclift 3y ago

Could be the porn stash of the Boeing directors. That could make some of them pretty nervous. ;)

JCharante 3y ago

or it could be a single finite element analysis file