5 ms·

She is probably liable for that data egress bill. Also, how does some randomer have ten of terabytes of disk lying around?

Nextgrid 6y ago
Maybe the data compresses well and/or a lot of it is redundant (so instead of storing it raw you store it in a database and use relationships to link related pieces of data)?
- johnchristopher 6y ago
  That's not how you usually measure a database size.
asiando 6y ago
Given the speed I’m guessing it was towards S3 so there’s plenty of terabytes.
- secondcoming 6y ago
  Yes probably. Wherever it is, that provider is in an interesting legal position by holding this data. Also, I assume there is content from EU citizens in there, and so GDPR violations galore.
  - roywashere 6y ago
    So people post it on Parlor themselves, it is publicly available, but it would be illegal to download for me? That does not make much sense.
    - secondcoming 6y ago
      My understanding is that not all of it was public posts. As an EU citizen, I can request that a company deletes my data. Unless this data dump is being treated as a crime scene or something, then the holders of this data will need some way to comply with these requests. Also, now AWS (or whatever cloud provider it is) is holding content that contains racist and/or illegal content. Are they not effectively now just another Parler?
      - sedev 6y ago
        No, they're not. Holding a static, private archive is not the same thing as hosting a live, public site.
        secondcoming 6y ago
        How so? Aren't people freely perusing all this data, extracting GPS metadata and reading both public and private conversations?
        ilovefood 6y ago
        You cannot be more wrong with this claim, you might not had to deal with this privately or professionaly I presume in which case it is understandable. Parler is liable for taking absolutely no precaution with their users data with whom they had a terms & conditions agreement (in short: users agreed to upload their data for a specific purpose) Now, ripping that _user_ generated data from the website this way without justified (justified = platform user agreement + legitimate interest) purpose or intent or even agreement and storing / distributing / processing it is the epitome of a GDPR transgression and borderline criminal at least in the EU (saying this as a EU citizen). They are liable. I wouldn't touch that dataset with a 10 foot pole. And I would even less brag about it on Twitter, things we do for clout I guess... :) I have no stake in this thing, it's just to emphasize that statements like this are what get people and businesses in big trouble. Stay safe! Archive only your own data or data you gathered legitimately. Take the rest up with a lawyer or ... read the laws.
        sedev 6y ago
        The claim I'm making is "they are not, in fact, 'just another Parler.'" You are pointing out many correct things about their potential liability and problems with holding that data! However, those things are, at most, a subset of the problems with Parler and at worst disjoint problems from Parler's problems. I stand by the argument of "a static, private data archive is not the same as a live, public web service." I did not make the argument of "there are no problems with the static, private data archive."
    - BrandonMarc 6y ago
      If you were to scape someone's private messages from Facebook - or their private posts - and then post them online en masse ... that may or may not be illegal in Europe, but it ain't ethically white. Grey at best.
      - sellyme 6y ago
        Nothing being discussed was private. It was all globally visible.
    - rndgermandude 6y ago
      It might be illegal for you to possess such data after you were asked to destroy it, yes, at least in the EU. The GDPR talks about "data controllers", and citizens have the right to demand such controller remove their personal data. A "data controller" in this context means you knowingly possess the data and are in the position to make decisions about it. You're not a data controller, tho, if e.g. you run some service that allows users to upload data, without your involvement and direction and also do not decide how to use such data. E.g. amazon would not be liable if people put a data dumb in their cloud (unless amazon used the data themselves, instead of just storing/hosting the data at the behest of their customers). Even before the GDPR there have been related laws and court cases, like the case that culminated in the "right to be forgotten" based on a decision by the European Court of Justice, which may well come into play here. I also remember a case in Germany, where a women allowed her partner to take intimate pictures of her, then after the relationship ended had a court order him to destroy the material (not a revenge porn case, there was no allegation he ever shared any of those pictures), meaning it's not always about what's public. I don't know how California's mini-GDPR compares. Then of course there is still the avenue of copyright law if the stuff is put online. Just because a parler user gave parler the permission to distribute a certain piece of content doesn't mean that everybody else has the same permission. I'm pretty sure Parler didn't make people assign them the copyright (which isn't even possible in some jurisdictions), therefore the people who posted on parler still retain the rights to their content. They can therefore use the DMCA or other similar jurisdictions around the world to demand takedowns.
  - r0m4n0 6y ago
    Agree with the legal implications. I would not go anywhere near this data. I don’t think this relates to GDPR though. There are some exemptions for personal use which I think there are arguments this could fall into (IANAL). But my opinion is this isn’t in the spirit of GDPR. There are many other laws that are broad enough in most countries to cover gray area scraping sadly. CFAA in the US for example. This sounds similar to the AT&T weev case.
- AndrewUnmuted 6y ago
  Didn't Parler just get booted from AWS though? Seems odd it'd go right back to Amazon.
gerikson 6y ago
Wasn't this an ArchiveTeam Warrior project? https://archiveteam.org/index.php?title=ArchiveTeam_Warrior https://archiveteam.org/index.php?title=ArchiveTeam_Warrior If so it's distributed among many volunteers. But the data still has to end up somewhere... Archive.org?
- philshem 6y ago
  the data is being processed by archive team https://www.archiveteam.org/index.php?title=Parler https://www.archiveteam.org/index.php?title=Parler Grab status: https://tracker.archiveteam.org/parler/ https://tracker.archiveteam.org/parler/ and will be hosted by archive.org
ClumsyPilot 6y ago
Have you ever been to r/Datahoarder or r/Homelab? 10TB fits on one desktop drive, it's completely pedestrian
- secondcoming 6y ago
  I have not. I have one 4TB HD in my machine and I've never come close to filling it!
  - ghaff 6y ago
    It's really a matter of multimedia: video, raw format photos, music, ripped movies. That's the vast bulk of the probably 8TB or so I have.
  - dijit 6y ago
    Many people have <1TiB on their machines and are content with it. Others, like me, have home NAS's which have 10-20TiB, and they're usually close to full. Fewer people, but a non-zero amount, hoard data, they have 20-100TiB or even more, full on homelabs. If you're a person collecting datadumps or running rainbow tables, you probably have such a unit. It's not even that expensive really, you can get a 24TiB Pegasus Thunderbolt raid array for <1,000 USD
  - mkr-hn 6y ago
    The raw files off my D5600 are roughly 30MB each. If I took 100 photos a day, I would need +1TB every year to keep all of them. The battery is good for about 600 shots. Someone who does photography professionally can easily blow right past that on one job before accounting for their second shooter's files.
DanBC 6y ago
> some randomer They're not a randomer, they're a person interested in data dumps.
- ersii 6y ago
  donk_enby wants to be called she/her according to Twitter, unless you're speaking of someone else
blaser-waffle 6y ago
I'm not a huge data fiend but I've got maybe ~3-4 2TB drives in my house: a Synology NAS + a spare drive. Including old desktops and a couple of random external HDDs, I could probably hit 8-10TB easily. And it's not like it's hard to get more. If I hit a montherlode and I need keep it, it's a 20 minute drive to Target / Best Buy / Walmart for a drive or three. Not as cheap as bulk orders off of Newegg but cost-effective enough to store these dumps.

Nextgrid 6y ago

Maybe the data compresses well and/or a lot of it is redundant (so instead of storing it raw you store it in a database and use relationships to link related pieces of data)?

johnchristopher 6y ago
That's not how you usually measure a database size.

asiando 6y ago

Given the speed I’m guessing it was towards S3 so there’s plenty of terabytes.

secondcoming 6y ago
Yes probably. Wherever it is, that provider is in an interesting legal position by holding this data. Also, I assume there is content from EU citizens in there, and so GDPR violations galore.
- roywashere 6y ago
  So people post it on Parlor themselves, it is publicly available, but it would be illegal to download for me? That does not make much sense.
  - secondcoming 6y ago
    My understanding is that not all of it was public posts. As an EU citizen, I can request that a company deletes my data. Unless this data dump is being treated as a crime scene or something, then the holders of this data will need some way to comply with these requests. Also, now AWS (or whatever cloud provider it is) is holding content that contains racist and/or illegal content. Are they not effectively now just another Parler?
    - sedev 6y ago
      No, they're not. Holding a static, private archive is not the same thing as hosting a live, public site.
      - secondcoming 6y ago
        How so? Aren't people freely perusing all this data, extracting GPS metadata and reading both public and private conversations?
      - ilovefood 6y ago
        You cannot be more wrong with this claim, you might not had to deal with this privately or professionaly I presume in which case it is understandable. Parler is liable for taking absolutely no precaution with their users data with whom they had a terms & conditions agreement (in short: users agreed to upload their data for a specific purpose) Now, ripping that _user_ generated data from the website this way without justified (justified = platform user agreement + legitimate interest) purpose or intent or even agreement and storing / distributing / processing it is the epitome of a GDPR transgression and borderline criminal at least in the EU (saying this as a EU citizen). They are liable. I wouldn't touch that dataset with a 10 foot pole. And I would even less brag about it on Twitter, things we do for clout I guess... :) I have no stake in this thing, it's just to emphasize that statements like this are what get people and businesses in big trouble. Stay safe! Archive only your own data or data you gathered legitimately. Take the rest up with a lawyer or ... read the laws.
        sedev 6y ago
        The claim I'm making is "they are not, in fact, 'just another Parler.'" You are pointing out many correct things about their potential liability and problems with holding that data! However, those things are, at most, a subset of the problems with Parler and at worst disjoint problems from Parler's problems. I stand by the argument of "a static, private data archive is not the same as a live, public web service." I did not make the argument of "there are no problems with the static, private data archive."
  - BrandonMarc 6y ago
    If you were to scape someone's private messages from Facebook - or their private posts - and then post them online en masse ... that may or may not be illegal in Europe, but it ain't ethically white. Grey at best.
    - sellyme 6y ago
      Nothing being discussed was private. It was all globally visible.
  - rndgermandude 6y ago
    It might be illegal for you to possess such data after you were asked to destroy it, yes, at least in the EU. The GDPR talks about "data controllers", and citizens have the right to demand such controller remove their personal data. A "data controller" in this context means you knowingly possess the data and are in the position to make decisions about it. You're not a data controller, tho, if e.g. you run some service that allows users to upload data, without your involvement and direction and also do not decide how to use such data. E.g. amazon would not be liable if people put a data dumb in their cloud (unless amazon used the data themselves, instead of just storing/hosting the data at the behest of their customers). Even before the GDPR there have been related laws and court cases, like the case that culminated in the "right to be forgotten" based on a decision by the European Court of Justice, which may well come into play here. I also remember a case in Germany, where a women allowed her partner to take intimate pictures of her, then after the relationship ended had a court order him to destroy the material (not a revenge porn case, there was no allegation he ever shared any of those pictures), meaning it's not always about what's public. I don't know how California's mini-GDPR compares. Then of course there is still the avenue of copyright law if the stuff is put online. Just because a parler user gave parler the permission to distribute a certain piece of content doesn't mean that everybody else has the same permission. I'm pretty sure Parler didn't make people assign them the copyright (which isn't even possible in some jurisdictions), therefore the people who posted on parler still retain the rights to their content. They can therefore use the DMCA or other similar jurisdictions around the world to demand takedowns.
- r0m4n0 6y ago
  Agree with the legal implications. I would not go anywhere near this data. I don’t think this relates to GDPR though. There are some exemptions for personal use which I think there are arguments this could fall into (IANAL). But my opinion is this isn’t in the spirit of GDPR. There are many other laws that are broad enough in most countries to cover gray area scraping sadly. CFAA in the US for example. This sounds similar to the AT&T weev case.
AndrewUnmuted 6y ago
Didn't Parler just get booted from AWS though? Seems odd it'd go right back to Amazon.

gerikson 6y ago

Wasn't this an ArchiveTeam Warrior project? https://archiveteam.org/index.php?title=ArchiveTeam_Warrior https://archiveteam.org/index.php?title=ArchiveTeam_Warrior If so it's distributed among many volunteers. But the data still has to end up somewhere... Archive.org?

philshem 6y ago
the data is being processed by archive team https://www.archiveteam.org/index.php?title=Parler https://www.archiveteam.org/index.php?title=Parler Grab status: https://tracker.archiveteam.org/parler/ https://tracker.archiveteam.org/parler/ and will be hosted by archive.org

ClumsyPilot 6y ago

Have you ever been to r/Datahoarder or r/Homelab? 10TB fits on one desktop drive, it's completely pedestrian

secondcoming 6y ago
I have not. I have one 4TB HD in my machine and I've never come close to filling it!
- ghaff 6y ago
  It's really a matter of multimedia: video, raw format photos, music, ripped movies. That's the vast bulk of the probably 8TB or so I have.
- dijit 6y ago
  Many people have <1TiB on their machines and are content with it. Others, like me, have home NAS's which have 10-20TiB, and they're usually close to full. Fewer people, but a non-zero amount, hoard data, they have 20-100TiB or even more, full on homelabs. If you're a person collecting datadumps or running rainbow tables, you probably have such a unit. It's not even that expensive really, you can get a 24TiB Pegasus Thunderbolt raid array for <1,000 USD
- mkr-hn 6y ago
  The raw files off my D5600 are roughly 30MB each. If I took 100 photos a day, I would need +1TB every year to keep all of them. The battery is good for about 600 shots. Someone who does photography professionally can easily blow right past that on one job before accounting for their second shooter's files.

DanBC 6y ago

> some randomer They're not a randomer, they're a person interested in data dumps.

ersii 6y ago
donk_enby wants to be called she/her according to Twitter, unless you're speaking of someone else

blaser-waffle 6y ago

I'm not a huge data fiend but I've got maybe ~3-4 2TB drives in my house: a Synology NAS + a spare drive. Including old desktops and a couple of random external HDDs, I could probably hit 8-10TB easily. And it's not like it's hard to get more. If I hit a montherlode and I need keep it, it's a 20 minute drive to Target / Best Buy / Walmart for a drive or three. Not as cheap as bulk orders off of Newegg but cost-effective enough to store these dumps.