5 ms·

I'm not sure about the legality and ethics of training models on stolen data, but for reference, but so far as I can tell the Enron email data set is about 1.5G

by anitil 3y ago

I'm not sure about the legality and ethics of training models on stolen data, but for reference, but so far as I can tell the Enron email data set is about 1.5GB (much lower than I expected to be honest!). And I believe some of the more interesting things found in that data set (outside of the fraud) were people cheating on their partners. https://www.kaggle.com/datasets/wcukierski/enron-email-dataset/data https://www.kaggle.com/datasets/wcukierski/enron-email-datas...

justsocrateasin 3y ago
I do believe that 1.5GB is tarred and gzipped though, so it is a fair bit bigger. That's also supposedly half a million emails, so 45gb is quite a bit.
primax 3y ago
Sure, but that was leaked in 2004. A very different time for email, and where attachments were generally smaller.
- jmalicki 3y ago
  The Enron dataset was not leaked - it was made public by the US government as evidence from an investigation by the Federal Energy Regulatory Commission. https://www.ferc.gov/electric/industry-activities/addressing-2000-2001-western-energy-crisis/information-released-enron-investigation https://www.ferc.gov/electric/industry-activities/addressing...
  - anitil 3y ago
    Yes my understanding is that this is why it's such a common training set - it's interesting, large(ish) and free and legal with no licence requirements. I think it's public domain? But I'm not from the US, so don't quote me.
freedude 3y ago
It depends upon the size of the individual email. In the early 2000's email attachments were less common and on average smaller in size. Today, 20 years later, it is common in my line of work to see email attachments in the 5-10MB range. Pass them back and forth a couple of dozen times for changes and approvals and it doesn't take long for it to balloon. Yesterday, I worked on an outlook email issue on a PST file that contained 40,000 emails and was over 10 GB in size. His PST was new less than two years ago. It really depends upon how it is used, misused or abused.
- anitil 3y ago
  You're absolutely right. That's one thing that I didn't follow up on, I only briefly looked at the csv in the kaggle page and didn't see anything that looked like images.

by anitil 3y ago

justsocrateasin 3y ago

I do believe that 1.5GB is tarred and gzipped though, so it is a fair bit bigger. That's also supposedly half a million emails, so 45gb is quite a bit.

primax 3y ago

Sure, but that was leaked in 2004. A very different time for email, and where attachments were generally smaller.

jmalicki 3y ago
The Enron dataset was not leaked - it was made public by the US government as evidence from an investigation by the Federal Energy Regulatory Commission. https://www.ferc.gov/electric/industry-activities/addressing-2000-2001-western-energy-crisis/information-released-enron-investigation https://www.ferc.gov/electric/industry-activities/addressing...
- anitil 3y ago
  Yes my understanding is that this is why it's such a common training set - it's interesting, large(ish) and free and legal with no licence requirements. I think it's public domain? But I'm not from the US, so don't quote me.

freedude 3y ago

It depends upon the size of the individual email. In the early 2000's email attachments were less common and on average smaller in size. Today, 20 years later, it is common in my line of work to see email attachments in the 5-10MB range. Pass them back and forth a couple of dozen times for changes and approvals and it doesn't take long for it to balloon. Yesterday, I worked on an outlook email issue on a PST file that contained 40,000 emails and was over 10 GB in size. His PST was new less than two years ago. It really depends upon how it is used, misused or abused.

anitil 3y ago
You're absolutely right. That's one thing that I didn't follow up on, I only briefly looked at the csv in the kaggle page and didn't see anything that looked like images.