4 ms·

PSA: Internet Archive "glitch" deletes years of user data and accounts

110 points by reaperducer 2y ago

slater 2y ago
This part is a bit odd, from the uhh... "support": "and, for the last time, the old user name is no longer available" As we all know, things in databases are immutable and can never be changed.
slater 2y ago
Apparently they're working on restoring stuff? also: > This story is still developing, and the press have been notified jeez calm down with the self-aggrandizing
- warkdarrior 2y ago
  You get what you pay for.
deleted 2y ago
[deleted]
tonetegeatinst 2y ago
Feels bad man... Hope this all gets fixed. In other news, welcome to the datahoarder community.
deleted 2y ago
[deleted]
sieabahlpark 2y ago
[dead]
muppetman 2y ago
[flagged]
sqidyyy 2y ago
While I absolutely understand the frustration of losing years worth of "preservation-level quality" data, you have to admit that the Internet Archive is a free service we do not deserve. They (the folks behind The Internet Archive) do not owe us anything.
- bt1a 2y ago
  > The Internet Archive serves millions of people each day and is one of the top 300 web sites in the world. A single copy of the Internet Archive library collection occupies 145+ Petabytes of server space (and we store at least 2 copies of everything). We are funded through donations, grants, and by providing web archiving and book digitization services for our partners. As with most libraries we value the privacy of our patrons, so we avoid keeping the IP (Internet Protocol) addresses of our readers and offer our site in https (secure) protocol. QFT What an unbelievably pissy and deserving tone from the author. If you want to be an archivist and you've permanently lost data that was stored in a single, external location, hopefully it's a learning experience
  - msephton 2y ago
    According to your quote the data was stored in at least 2 places, but Internet Archive are choosing not to restore it? That's a different type of learning experience.
  - refulgentis 2y ago
    Random text hunk? QFT? Quantum Field Theory? > What an unbelievably pissy and deserving tone from the author. n.b. they are interlocuting with "Patron Services" :X Pissy? "My support experience with Internet Archive was frustrating and ultimately futile. They did not adequately address my queries and requests" is a very whitebread version of "pissy" > If you want to be an archivist and you've permanently lost data that was stored in a single, external location, hopefully it's a learning experience Yeah, hopefully!
    - rf15 2y ago
      QFT == Quoted for Truth
worstspotgain 2y ago
The Internet Archive is one of the wonders of the digital age. I get a proud humanistic feeling every time I walk by their building on Funston and Clement, a former temple that totally looks the part. There was a fire there about a decade ago that took out a small addition and a restaurant next door. That addition had a "9/11 in pictures" window display that was interesting and kind of eerie. Luckily they were able to save the main building, or the internet as we know it would be quite a bit worse, and the internet as we knew it might be gone.
Springtime 2y ago
Just re the username thing, from what I've seen in the past even changing an active username just once seems to prevent the same user from reverting back to their original username. So could be their account system is a bit primitive in that regard (given that official replies for this news makes it sound like they're utilizing the existing system for restoring content to new usernames, rather than reverting directly via backups). I don't doubt there will be some official clarification though.
FarMcKon 2y ago
Geeze, Karen take a breath. That is sad news, and it really sucks to lose data At the same time it is a shoestring budget nonprofit, doing tons of good for the world, thanklessly. It is not a right, a government service, or even a service you pay for. You have no right to demand anything, just because they are kindly providing you value. Cut them some slack, and stop acting like they own you anything. Maybe even gasp thank them for doing so much good, with no support, for everyone in the world, for free
- msephton 2y ago
  I donate during their annual fundraisers. I don't imagine for a moment that entitles me to anything. But I'd settle for some common courtesy, peace and love from my fellow humans.
fny 2y ago
> create new accounts and silently relinking their old uploads only if the new account has same email as the old account Why can't they just relink all the content and force password resets? Something doesn't add up. Looking at the forums, it seems like someone is doing manual remediation.
- PenguinRevolver 2y ago
  Every item is linked to the uploader's email address. If someone changes the email of that account, all of the items disappear. This is because, as far as the system knows, the item's email address is not the same as the new email address. What I find strange is that the uploader's email address is made public on every item they upload. This is not good, especially if someone uses an alias as their account name but has their full name as their email address. It would be surprisingly easy to scrape every email address from every uploader and cause a moderate leak of information.
seydor 2y ago
Is there an internet archive archive?
- elliotec 2y ago
  Contrary to popular belief, it’s not turtles all the way down.
- skissane 2y ago
  I hear these folks keep a backup of everything: https://en.wikipedia.org/wiki/Utah_Data_Center https://en.wikipedia.org/wiki/Utah_Data_Center
- internetter 2y ago
  Yes: > Do you backup my files? > Yes. We duplicate/backup all files at various locations
anoncow 2y ago
Do better man with ginger beard.
- deleted 2y ago
  [deleted]
- msephton 2y ago
  I don't actually have a ginger beard. It's a long story. I try my best, and hope others do too.
19h 2y ago
Surely, the loss of some user metadata is a catastrophe on par with the burning of the Library of Alexandria. Perhaps next time, the Internet Archive should consult their vast army of paid engineers and their bottomless coffers to ensure such a calamity never befalls humanity again. After all, what's the point of preserving vast swathes of human knowledge and culture if one can't access their personal bookmarks from 2007?
jamesy0ung 2y ago
Give them a break, they do a lot of work, all funded via donations. If your data is so critical that it cannot ever go down, host it yourself.
- msephton 2y ago
  That's the point, it's their data too.
blindriver 2y ago
Free service loses data and offers poor customer service! News at 11! I love how the author unironically links to a donation page but doesn’t mention anything about donating to Internet archive.
sonorous_sub 2y ago
Profiles over there have been glitchy for a while now. I gave up on favorites years ago and just bookmarked everything of interest in my browser instead. User metadata is probably a low priority item for their staff. Still one of my top five favorite sites on the web.
- Ekaros 2y ago
  Sounds to me that they have some very poorly thought out database schemas with pretty weird foreign key choices... Provenance is not unimportant, but maybe in case of Internet Archive lot less important... And in the end fixing poorly designed accounts and information linked to them can be hard.
skissane 2y ago
Have they actually lost the data these people contributed? Or just the metadata which links it to their account - such that the data is still accessible if you link to it directly (not via the uploader’s user profile)? I think the answer is the second not the first, but this blog post isn’t very clear as to what is going on. Losing archived data - that would be more serious. Losing metadata linking it to the uploader - far from ideal but not as bad.
- msephton 2y ago
  It was as clear as I could make it with the information available. The answer is somewhere between your two hypotheses. Lots of data is no longer accessible, and links are broken. Uploads are safe but some other archived data is gone. There's more to an internet archive profile than uploads, so I'd say it's somewhat serious of a situation. We don't really know the true extent of the problem but we can easily see what's missing (see the lists in the blog posts)
deleted 2y ago
[deleted]
mediumsmart 2y ago
The internet archive where data is kept safe forever has deleted a great many accounts of an unknown number and refuses to map the favorites of these now non existent accounts to new accounts signing up with different emails. They call this a glitch. The press has been informed.

110 points by reaperducer 2y ago

This part is a bit odd, from the uhh... "support": "and, for the last time, the old user name is no longer available" As we all know, things in databases are immutable and can never be changed.

slater 2y ago

Apparently they're working on restoring stuff? also: > This story is still developing, and the press have been notified jeez calm down with the self-aggrandizing

warkdarrior 2y ago
You get what you pay for.

deleted 2y ago

[deleted]

tonetegeatinst 2y ago

Feels bad man... Hope this all gets fixed. In other news, welcome to the datahoarder community.

deleted 2y ago

[deleted]

sieabahlpark 2y ago

[dead]

muppetman 2y ago

[flagged]

sqidyyy 2y ago

While I absolutely understand the frustration of losing years worth of "preservation-level quality" data, you have to admit that the Internet Archive is a free service we do not deserve. They (the folks behind The Internet Archive) do not owe us anything.

bt1a 2y ago
> The Internet Archive serves millions of people each day and is one of the top 300 web sites in the world. A single copy of the Internet Archive library collection occupies 145+ Petabytes of server space (and we store at least 2 copies of everything). We are funded through donations, grants, and by providing web archiving and book digitization services for our partners. As with most libraries we value the privacy of our patrons, so we avoid keeping the IP (Internet Protocol) addresses of our readers and offer our site in https (secure) protocol. QFT What an unbelievably pissy and deserving tone from the author. If you want to be an archivist and you've permanently lost data that was stored in a single, external location, hopefully it's a learning experience
- msephton 2y ago
  According to your quote the data was stored in at least 2 places, but Internet Archive are choosing not to restore it? That's a different type of learning experience.
- refulgentis 2y ago
  Random text hunk? QFT? Quantum Field Theory? > What an unbelievably pissy and deserving tone from the author. n.b. they are interlocuting with "Patron Services" :X Pissy? "My support experience with Internet Archive was frustrating and ultimately futile. They did not adequately address my queries and requests" is a very whitebread version of "pissy" > If you want to be an archivist and you've permanently lost data that was stored in a single, external location, hopefully it's a learning experience Yeah, hopefully!
  - rf15 2y ago
    QFT == Quoted for Truth

worstspotgain 2y ago

The Internet Archive is one of the wonders of the digital age. I get a proud humanistic feeling every time I walk by their building on Funston and Clement, a former temple that totally looks the part. There was a fire there about a decade ago that took out a small addition and a restaurant next door. That addition had a "9/11 in pictures" window display that was interesting and kind of eerie. Luckily they were able to save the main building, or the internet as we know it would be quite a bit worse, and the internet as we knew it might be gone.

Springtime 2y ago

Just re the username thing, from what I've seen in the past even changing an active username just once seems to prevent the same user from reverting back to their original username. So could be their account system is a bit primitive in that regard (given that official replies for this news makes it sound like they're utilizing the existing system for restoring content to new usernames, rather than reverting directly via backups). I don't doubt there will be some official clarification though.

FarMcKon 2y ago

Geeze, Karen take a breath. That is sad news, and it really sucks to lose data At the same time it is a shoestring budget nonprofit, doing tons of good for the world, thanklessly. It is not a right, a government service, or even a service you pay for. You have no right to demand anything, just because they are kindly providing you value. Cut them some slack, and stop acting like they own you anything. Maybe even gasp thank them for doing so much good, with no support, for everyone in the world, for free

msephton 2y ago
I donate during their annual fundraisers. I don't imagine for a moment that entitles me to anything. But I'd settle for some common courtesy, peace and love from my fellow humans.

fny 2y ago

> create new accounts and silently relinking their old uploads only if the new account has same email as the old account Why can't they just relink all the content and force password resets? Something doesn't add up. Looking at the forums, it seems like someone is doing manual remediation.

PenguinRevolver 2y ago
Every item is linked to the uploader's email address. If someone changes the email of that account, all of the items disappear. This is because, as far as the system knows, the item's email address is not the same as the new email address. What I find strange is that the uploader's email address is made public on every item they upload. This is not good, especially if someone uses an alias as their account name but has their full name as their email address. It would be surprisingly easy to scrape every email address from every uploader and cause a moderate leak of information.

seydor 2y ago

Is there an internet archive archive?

elliotec 2y ago
Contrary to popular belief, it’s not turtles all the way down.
skissane 2y ago
I hear these folks keep a backup of everything: https://en.wikipedia.org/wiki/Utah_Data_Center https://en.wikipedia.org/wiki/Utah_Data_Center
internetter 2y ago
Yes: > Do you backup my files? > Yes. We duplicate/backup all files at various locations

anoncow 2y ago

Do better man with ginger beard.

deleted 2y ago
[deleted]
msephton 2y ago
I don't actually have a ginger beard. It's a long story. I try my best, and hope others do too.

19h 2y ago

Surely, the loss of some user metadata is a catastrophe on par with the burning of the Library of Alexandria. Perhaps next time, the Internet Archive should consult their vast army of paid engineers and their bottomless coffers to ensure such a calamity never befalls humanity again. After all, what's the point of preserving vast swathes of human knowledge and culture if one can't access their personal bookmarks from 2007?

jamesy0ung 2y ago

Give them a break, they do a lot of work, all funded via donations. If your data is so critical that it cannot ever go down, host it yourself.

msephton 2y ago
That's the point, it's their data too.

blindriver 2y ago

Free service loses data and offers poor customer service! News at 11! I love how the author unironically links to a donation page but doesn’t mention anything about donating to Internet archive.

sonorous_sub 2y ago

Profiles over there have been glitchy for a while now. I gave up on favorites years ago and just bookmarked everything of interest in my browser instead. User metadata is probably a low priority item for their staff. Still one of my top five favorite sites on the web.

Ekaros 2y ago
Sounds to me that they have some very poorly thought out database schemas with pretty weird foreign key choices... Provenance is not unimportant, but maybe in case of Internet Archive lot less important... And in the end fixing poorly designed accounts and information linked to them can be hard.

skissane 2y ago

Have they actually lost the data these people contributed? Or just the metadata which links it to their account - such that the data is still accessible if you link to it directly (not via the uploader’s user profile)? I think the answer is the second not the first, but this blog post isn’t very clear as to what is going on. Losing archived data - that would be more serious. Losing metadata linking it to the uploader - far from ideal but not as bad.

msephton 2y ago
It was as clear as I could make it with the information available. The answer is somewhere between your two hypotheses. Lots of data is no longer accessible, and links are broken. Uploads are safe but some other archived data is gone. There's more to an internet archive profile than uploads, so I'd say it's somewhat serious of a situation. We don't really know the true extent of the problem but we can easily see what's missing (see the lists in the blog posts)

deleted 2y ago

[deleted]

mediumsmart 2y ago

The internet archive where data is kept safe forever has deleted a great many accounts of an unknown number and refuses to map the favorites of these now non existent accounts to new accounts signing up with different emails. They call this a glitch. The press has been informed.