10 ms·

Saving of public Google+ content by the Archive Team has begun

170 points by danso 8y ago

mavhc 8y ago
In other news, don't bother putting your content on 3rd party websites, it'll just be deleted and you'll have wasted your time. Set up a server, and a donation button, with a countdown timer for when the server will be deleted if no one donates, if it's useful people should pay to keep it going
- ghaff 8y ago
  And eventually people will stop donating (if they ever did) or you'll lose interest and the content will be deleted anyway. So basically don't bother. Everything will get deleted someday anyway. (Unless it's saved by the Internet Archive which may or may not be around forever (as in a very long time). Which is a bit worrying itself.)
  - dredmorbius 8y ago
    "Over time, all data approaches deleted, or public." -- Quinn Norton https://medium.com/message/hello-future-pastebin-readers-39d9b4eb935f https://medium.com/message/hello-future-pastebin-readers-39d...
    - ghaff 8y ago
      Even public is no guarantee over the long term. But it's better than assuming private will be.
  - the_pwner224 8y ago
    This is a great selling point of distributed systems like IPFS <ipfs.io> and BitTorrent. With most online services, some entity X (Google in this case) says 'I will host this data.' Eventually it stops being worth it for X to host that data, and the data is lost forever (unless someone archives it like this case, which just changes the X to some other entity). With distributed content sharing networks, the data is retained as long as anyone is willing to bear the cost of storing and distributing it. After I upload a file I may eventually decide that it is no longer worth sharing, but as long as someone thinks it is a useful file to share they can have it seeding from their computer. The file is only permanently gone once nobody thinks it's worth sharing. And with a large population on these networks, that is equivalent to the file having no actual value - except for historical research perhaps. This is actually better than what the parent (mavhc) suggested, because with the donation method there is still one entity in charge of sharing the data. Others can archive and share it (if allowed by the primary entity), but there is still a discoverability problem that is solved by IPFS/BT.
    - 0815test 8y ago
      Hoarding lots of data is the kind of thing that becomes really, really cheap to do (per GB) once you do it at scale, and is not-so-cheap otherwise. I think we need both, the big centralized hosters like the Archive, and the distribution that IPFS provides. But the latter mostly as a last resort, to keep lots of copies around of the most socially valuable stuff, that we wouldn't want to lose even in a disaster scenario.
    - Avamander 8y ago
      Actually BitTorrent suffers torrent-rot thanks to the DRM that are private trackers, not a single mainstream client offers you the ability to just DHT/PEX/LSD(LPD) a torrent which means if the trackers go down the torrents die. I find it a big concern, even more so in the future. If ipfs suffers the same problem - noone knows which hash to look up - the files will be irrevocably lost.
      - dvtrn 8y ago
        Isn't tracker "rot" what magnet links were meant to solve or have I misunderstood that?
        Avamander 8y ago
        Magnet links can't solve it if the torrent is marked private. There's no DHT, PEX and LSD(LPD).
kalleboo 8y ago
Cool! I have the Archive Warrior docker image running on my Synology NAS. I just checked and it seems I've archived 160 GB of Google+ already.
- deleted 8y ago
  [deleted]
tyingq 8y ago
I wonder if Google will take note and make a specific donation. Might be a good PR move.
- wrs 8y ago
  If Google wanted to help, they could skip all this drama by just rendering everything themselves and shipping some Transfer Appliances to the archive.
  - tyingq 8y ago
    Or just freeze it as-is, read only...
  - lostmyoldone 8y ago
    I'm not an expert, but unless there are G+ specific exceptions I've missed, I believe that the current wording of their terms-of-service would prohibit them from engaging in the efforts to archive G+ pages. This it was it says about the rights you give Google by/if you are using their services: "The rights you grant [to Google] in this license are for the limited purpose of operating, promoting, and improving our Services, and to develop new ones." The only rubric archival could possibly fall under would be "promotion", but that feels marginal at best.
    - wrs 8y ago
      I see your point, but it’s hard to believe the Internet Archive has more ability to do this than Google does, since they were granted no rights whatsoever. Seems like whatever legal theory the Internet Archive is using must apply to everyone, including Google, unless Google waived whatever that is.
- lostmyoldone 8y ago
  PR is probably the only way it could be possible because their (Google's) terms-of-service effectively forbids them to distribute the user generated content without everybodys consent. But unless there is a lot of public support it could easily backfire, as when everyone realize how the "promotion" clause can be interpreted, the TOS suddenly look quite a bit more onerous.
sodosopa 8y ago
I've made personal copies of rare bits of info for my Asus tablet/laptop but shuttered my pages down. But that was the needle in the haystack of g+ content, most everything else seem to have slacked off shortly after the novelty wore off.
kentonv 8y ago
I'm not sure how I feel about this. Over the last couple years it has become increasingly clear that public personal social media history is a liability. People's histories have often been weaponized against them, and mass processing of social media has been used as a weapon against society as a whole (Cambridge Analytica, etc.). I recently deleted my Facebook account, but didn't take the time to delete Google+ because I figured it was about to disappear anyway. But now it turns out some people are building a public archive of my content, and presumably they aren't going to give me a way to delete it. This makes me uncomfortable. I have now deleted my Google+ history; I hope they didn't get much of it. To be clear, I think the Internet Archive is good people with noble intentions. But given what we've learned recently about social media possibly being a huge mistake... this effort worries me.
- ocdtrekkie 8y ago
  Yeah, I'll probably put in a removal request for my G+ profile as well. I hadn't the heart to delete it myself, but I was looking forward to the social history purge that the shutdown represented. Obviously I have a Takeout download with my own content, but I don't really see any need for the Internet Archive to maintain a copy of it.
  - cromwellian 8y ago
    They can only archive your public posts. Maybe if people were afraid their posts might be read in the future and tied to them they’d stop shitposting so much and only post content they’d be proud of. Internet social media is full of some the most vile verbal diarrhea to come out of the human species, enabled by actual anonymity or beliefs in obscurity that others won’t find what they said. It’s reached a head now with all of this radicalization leading to neofascist terrorism and what’s being asked is for all online platforms to start censoring shitposting. I’d say this removes the primary social feedback mechanism that has existed in human culture: the fear of tit for tat repercussions in the future based on past bad behavior. Perhaps there’s something to be said about holding people To their words. I’ve been in the internet posting since 1986, I’ve said some crazy shit during my teenage years as a dick headed devotee of Ayn Rand, but I don’t want to delete them, I don’t think it’s honest or brave. There are legit reasons for anonymity like violence of course but this isn’t the norm. We need to find ways to promote and protect people who need anonymity while also not encouraging the worst human behavior and abuse online as it seems the alternatives are either self censorship based civility or actual censorship and I prefer The former.
tyingq 8y ago
There are a few gems on Google+ that would be a shame to lose, like: https://plus.google.com/101960720994009339267/posts/R58WgWwN9jp https://plus.google.com/101960720994009339267/posts/R58WgWwN... Archived copy: http://archive.is/4kpvd http://archive.is/4kpvd
- dredmorbius 8y ago
  Indeed. Also infrastructure-like breakage, through APIs, and particularly images: https://philip.greenspun.com/blog/2019/03/07/less-than-a-month-to-go-before-google-breaks-hundreds-of-thousands-of-links-all-over-the-internet/ https://philip.greenspun.com/blog/2019/03/07/less-than-a-mon...
- ardy42 8y ago
  > There are a few gems on Google+ that would be a shame to lose, like: Public posts by notable people like this are a really good thing to archive.
- eneveu 8y ago
  Also the Steve Yegge posts: https://plus.google.com/110981030061712822816 https://plus.google.com/110981030061712822816
criddell 8y ago
I've never understood the urge to archive all this old stuff. If you are somebody that thinks it's important to save as much of this stuff as we can, I'd love to have my opinion changed. The cost to save everything isn't trivial, but often it feels like the value is.
- hanniabu 8y ago
  It's better to have this information and it not be needed than to need it and not have it.
- ourmandave 8y ago
  Next week, on an all new episode of Data Hoarders... #A&EruListening?
- nat8265639392 8y ago
  I agree in that I don’t understand what is important about this. But as someone interested in history knowing that stuff has been thrown out because of this same mindset hundreds of years ago is upsetting. People 2000 years from now might study the google+ years at university lol.
- etiam 8y ago
  "Deep Learning" group. https://plus.google.com/communities/112866381580457264725 https://plus.google.com/communities/112866381580457264725 Discussion as methods and questions emerged, often involving the pioneers of the field. LeCun, Bengio, Krizshevsky, Goodfellow, etc. Ranging from the days when people were starting to realize you could implement theses thing for a GPU instead of just CPU, to days when it's arguably rehabiliteted "A.I." as credible in the popular imagination and spawned a multi-billion dollar industry. Admittedly it's the only thing I'm aware of on G+ which I'm anxious to see archived, but that's probably mostly for lack of looking. These are interesting times. It would be a shame if the records of them are lost just because we put them on a medium which will self-destruct in absence of active maintenance and then failed to put in the maintenance.
  - criddell 8y ago
dredmorbius 8y ago
Author and one of the folks engaged in the G+ Migration / Exodus movement here.
deleted 8y ago
[deleted]
numbfall 8y ago
I am logged in to so many applications and services I think I can never undo my trace on the internet. And then there is this archiving of deleted past. I feel congested.

170 points by danso 8y ago

In other news, don't bother putting your content on 3rd party websites, it'll just be deleted and you'll have wasted your time. Set up a server, and a donation button, with a countdown timer for when the server will be deleted if no one donates, if it's useful people should pay to keep it going

ghaff 8y ago
And eventually people will stop donating (if they ever did) or you'll lose interest and the content will be deleted anyway. So basically don't bother. Everything will get deleted someday anyway. (Unless it's saved by the Internet Archive which may or may not be around forever (as in a very long time). Which is a bit worrying itself.)
- dredmorbius 8y ago
  "Over time, all data approaches deleted, or public." -- Quinn Norton https://medium.com/message/hello-future-pastebin-readers-39d9b4eb935f https://medium.com/message/hello-future-pastebin-readers-39d...
  - ghaff 8y ago
    Even public is no guarantee over the long term. But it's better than assuming private will be.
- the_pwner224 8y ago
  This is a great selling point of distributed systems like IPFS <ipfs.io> and BitTorrent. With most online services, some entity X (Google in this case) says 'I will host this data.' Eventually it stops being worth it for X to host that data, and the data is lost forever (unless someone archives it like this case, which just changes the X to some other entity). With distributed content sharing networks, the data is retained as long as anyone is willing to bear the cost of storing and distributing it. After I upload a file I may eventually decide that it is no longer worth sharing, but as long as someone thinks it is a useful file to share they can have it seeding from their computer. The file is only permanently gone once nobody thinks it's worth sharing. And with a large population on these networks, that is equivalent to the file having no actual value - except for historical research perhaps. This is actually better than what the parent (mavhc) suggested, because with the donation method there is still one entity in charge of sharing the data. Others can archive and share it (if allowed by the primary entity), but there is still a discoverability problem that is solved by IPFS/BT.
  - 0815test 8y ago
    Hoarding lots of data is the kind of thing that becomes really, really cheap to do (per GB) once you do it at scale, and is not-so-cheap otherwise. I think we need both, the big centralized hosters like the Archive, and the distribution that IPFS provides. But the latter mostly as a last resort, to keep lots of copies around of the most socially valuable stuff, that we wouldn't want to lose even in a disaster scenario.
  - Avamander 8y ago
    Actually BitTorrent suffers torrent-rot thanks to the DRM that are private trackers, not a single mainstream client offers you the ability to just DHT/PEX/LSD(LPD) a torrent which means if the trackers go down the torrents die. I find it a big concern, even more so in the future. If ipfs suffers the same problem - noone knows which hash to look up - the files will be irrevocably lost.
    - dvtrn 8y ago
      Isn't tracker "rot" what magnet links were meant to solve or have I misunderstood that?
      - Avamander 8y ago
        Magnet links can't solve it if the torrent is marked private. There's no DHT, PEX and LSD(LPD).

kalleboo 8y ago

Cool! I have the Archive Warrior docker image running on my Synology NAS. I just checked and it seems I've archived 160 GB of Google+ already.

deleted 8y ago
[deleted]

tyingq 8y ago

I wonder if Google will take note and make a specific donation. Might be a good PR move.

wrs 8y ago
If Google wanted to help, they could skip all this drama by just rendering everything themselves and shipping some Transfer Appliances to the archive.
- tyingq 8y ago
  Or just freeze it as-is, read only...
- lostmyoldone 8y ago
  I'm not an expert, but unless there are G+ specific exceptions I've missed, I believe that the current wording of their terms-of-service would prohibit them from engaging in the efforts to archive G+ pages. This it was it says about the rights you give Google by/if you are using their services: "The rights you grant [to Google] in this license are for the limited purpose of operating, promoting, and improving our Services, and to develop new ones." The only rubric archival could possibly fall under would be "promotion", but that feels marginal at best.
  - wrs 8y ago
    I see your point, but it’s hard to believe the Internet Archive has more ability to do this than Google does, since they were granted no rights whatsoever. Seems like whatever legal theory the Internet Archive is using must apply to everyone, including Google, unless Google waived whatever that is.
lostmyoldone 8y ago
PR is probably the only way it could be possible because their (Google's) terms-of-service effectively forbids them to distribute the user generated content without everybodys consent. But unless there is a lot of public support it could easily backfire, as when everyone realize how the "promotion" clause can be interpreted, the TOS suddenly look quite a bit more onerous.

sodosopa 8y ago

I've made personal copies of rare bits of info for my Asus tablet/laptop but shuttered my pages down. But that was the needle in the haystack of g+ content, most everything else seem to have slacked off shortly after the novelty wore off.

kentonv 8y ago

I'm not sure how I feel about this. Over the last couple years it has become increasingly clear that public personal social media history is a liability. People's histories have often been weaponized against them, and mass processing of social media has been used as a weapon against society as a whole (Cambridge Analytica, etc.). I recently deleted my Facebook account, but didn't take the time to delete Google+ because I figured it was about to disappear anyway. But now it turns out some people are building a public archive of my content, and presumably they aren't going to give me a way to delete it. This makes me uncomfortable. I have now deleted my Google+ history; I hope they didn't get much of it. To be clear, I think the Internet Archive is good people with noble intentions. But given what we've learned recently about social media possibly being a huge mistake... this effort worries me.

ocdtrekkie 8y ago
Yeah, I'll probably put in a removal request for my G+ profile as well. I hadn't the heart to delete it myself, but I was looking forward to the social history purge that the shutdown represented. Obviously I have a Takeout download with my own content, but I don't really see any need for the Internet Archive to maintain a copy of it.
- cromwellian 8y ago
  They can only archive your public posts. Maybe if people were afraid their posts might be read in the future and tied to them they’d stop shitposting so much and only post content they’d be proud of. Internet social media is full of some the most vile verbal diarrhea to come out of the human species, enabled by actual anonymity or beliefs in obscurity that others won’t find what they said. It’s reached a head now with all of this radicalization leading to neofascist terrorism and what’s being asked is for all online platforms to start censoring shitposting. I’d say this removes the primary social feedback mechanism that has existed in human culture: the fear of tit for tat repercussions in the future based on past bad behavior. Perhaps there’s something to be said about holding people To their words. I’ve been in the internet posting since 1986, I’ve said some crazy shit during my teenage years as a dick headed devotee of Ayn Rand, but I don’t want to delete them, I don’t think it’s honest or brave. There are legit reasons for anonymity like violence of course but this isn’t the norm. We need to find ways to promote and protect people who need anonymity while also not encouraging the worst human behavior and abuse online as it seems the alternatives are either self censorship based civility or actual censorship and I prefer The former.

tyingq 8y ago

There are a few gems on Google+ that would be a shame to lose, like: https://plus.google.com/101960720994009339267/posts/R58WgWwN9jp https://plus.google.com/101960720994009339267/posts/R58WgWwN... Archived copy: http://archive.is/4kpvd http://archive.is/4kpvd

dredmorbius 8y ago
Indeed. Also infrastructure-like breakage, through APIs, and particularly images: https://philip.greenspun.com/blog/2019/03/07/less-than-a-month-to-go-before-google-breaks-hundreds-of-thousands-of-links-all-over-the-internet/ https://philip.greenspun.com/blog/2019/03/07/less-than-a-mon...
ardy42 8y ago
> There are a few gems on Google+ that would be a shame to lose, like: Public posts by notable people like this are a really good thing to archive.
eneveu 8y ago
Also the Steve Yegge posts: https://plus.google.com/110981030061712822816 https://plus.google.com/110981030061712822816

criddell 8y ago

I've never understood the urge to archive all this old stuff. If you are somebody that thinks it's important to save as much of this stuff as we can, I'd love to have my opinion changed. The cost to save everything isn't trivial, but often it feels like the value is.

hanniabu 8y ago
It's better to have this information and it not be needed than to need it and not have it.
ourmandave 8y ago
Next week, on an all new episode of Data Hoarders... #A&EruListening?
nat8265639392 8y ago
I agree in that I don’t understand what is important about this. But as someone interested in history knowing that stuff has been thrown out because of this same mindset hundreds of years ago is upsetting. People 2000 years from now might study the google+ years at university lol.
etiam 8y ago
"Deep Learning" group. https://plus.google.com/communities/112866381580457264725 https://plus.google.com/communities/112866381580457264725 Discussion as methods and questions emerged, often involving the pioneers of the field. LeCun, Bengio, Krizshevsky, Goodfellow, etc. Ranging from the days when people were starting to realize you could implement theses thing for a GPU instead of just CPU, to days when it's arguably rehabiliteted "A.I." as credible in the popular imagination and spawned a multi-billion dollar industry. Admittedly it's the only thing I'm aware of on G+ which I'm anxious to see archived, but that's probably mostly for lack of looking. These are interesting times. It would be a shame if the records of them are lost just because we put them on a medium which will self-destruct in absence of active maintenance and then failed to put in the maintenance.
- criddell 8y ago

dredmorbius 8y ago

Author and one of the folks engaged in the G+ Migration / Exodus movement here.

deleted 8y ago

[deleted]

numbfall 8y ago

I am logged in to so many applications and services I think I can never undo my trace on the internet. And then there is this archiving of deleted past. I feel congested.