7 ms·
> It wouldn't even be a significant cost and ads would recoup it anyway I 100% don't get it when websites with lots of content and good SEO just delete everyth
by Scoundreller 3y ago
> It wouldn't even be a significant cost and ads would recoup it anyway
I 100% don't get it when websites with lots of content and good SEO just delete everything. Fatwallet (fuck you Rakuten). Yahoo Answers (hey, I didn't say good content).
Even on my own Drupal blogs that I didn't want to maintain/update anymore, I did a giant curl job, recursively regex'd out the login/comment submission fields and dumped them on s3. Voila. And I'm someone that would take 9 attempts to do FizzBuzz.
A bookmark you had to it 20 years ago still works and I still get some Adsense cheque residuals from it for zero effort.
- naet 3y agoI did some work for a design agency that got bought by Twitter.... we basically converted all their different sites to full static and put them on ice. It would have been expensive to keep hosting how they had been, but with everything flattened to static it's super cheap to keep up (although no publishing new stuff without some by hand work or restoring the old servers). All the good articles and design samples stay online, and maybe the original authors can get a little credit or name recognition from any search traffic that finds it. The one "liability" to doing this would be for visitors to not realize the site isn't being updated anymore. If vice is still publishing content outside their website they'd have to make it clear that the site is archived and new content is elsewhere, or people might get the wrong idea and assume nothing new is coming.
- babypuncher 3y agoThat liability seems really easily solved with a banner.
- Scoundreller 3y agoyeah, newspapers do this all the time. Could also regex the dates to show up really large.
- ewalk153 3y agoCould the internet archive offer this as a paid service? For a reasonable fee, we'll archive your site, and give you back a copy of the assets you can turnkey host on the original domain for cheap with a static hosting solution (s3, cloudflare, etc). Everybody wins
- toomim 3y agoI've been wondering this too. I've even been considering authoring a web standard to allow hosts to specify how their pages can be archived in a standard way (e.g. which scripts to include, etc.) and then pitch the IA to offer a "pay $X to archive this data forever" deal to the universe. I'm really curious what the cost per byte would be to make it worthwhile to offer a "host this byte forever, for one up-front fee" service.
- deleted 3y ago[deleted]
- Intralexical 3y ago> I'm really curious what the cost per byte would be to make it worthwhile to offer a "host this byte forever, for one up-front fee" service. https://help.archive.org/help/archive-org-information/ https://help.archive.org/help/archive-org-information/ > *What are your fees?* > At this time we have no fees for uploading and preserving materials. We estimate that permanent storage costs us approximately $2.00US per gigabyte. While there are no fees we always appreciate donations to offset these costs. There's some discussion about this idea on this thread, including comments by ?id=markjgraham, who manages the Wayback Machine, thoughts from John Carmack: https://news.ycombinator.com/item?id=29639222 https://news.ycombinator.com/item?id=29639222 https://twitter.com/ID_AA_Carmack/status/1473327982605385735 https://twitter.com/ID_AA_Carmack/status/1473327982605385735 https://threadreaderapp.com/thread/1473327982605385735.html https://threadreaderapp.com/thread/1473327982605385735.html
- toomim 3y agoAmazing references! Thank you!
- makeitdouble 3y agoYou still have to maintain the domain, and stay on the hook if the site gets overtaken/hacked/defaced.
- deelowe 3y agoI'm sure there's liability, copyright, and trademark concerns as well.
- lodovic 3y agoIf they would want to monetise their content in the future, they must keep it private before it's gobbled up by search engines and AI and becomes public domain.
- bryanrasmussen 3y agohmm, yeah all sites in the future that find themselves unprofitable take content offline to maintain license to AI potential.
- GuB-42 3y agoSearch engines have already gobbled it up the moment it was published. And it is probably in the training dataset of many AIs.
- simlevesque 3y agoI know that web security is a hard problem but I can't wrap my head around the fact that static content has the same issue as a Wordpress blog.
- Scoundreller 3y agoThat’s correct. Only skip-a-heartbeat moment was when aws sent me an email saying that you I have “one or more S3 buckets that allow read or write access from any user on the Internet” But none of my containers had write access. All of them had public read, but yeah, it’s a website and they know this: their own route53 DNS points to the containers. They just sent the same generic mass email to everyone with any public container.
- dirtybirdnj 3y ago> we basically converted all their different sites to full static and put them on ice. It would have been expensive to keep hosting how they had been, but with everything flattened to static it's super cheap to keep up I got absolutely tarred and feathered at an agency for suggesting this strategy and I just want to say thank you for validating what I wanted to do.
- DamnableNook 3y agoOut of curiosity, what was the argument against it?
- mobilefriendly 3y agoThe old content can become a liability. Maybe the brand has shifted, or norms have shifted and the old content is off-key.
- dirtybirdnj 3y agoIt was not an issue of refining the content on one site, you make a good point though. It was more of an issue that they had a bunch of clients on an old CMS system, and they did not want to make any changes the way that the sites were built or hosted. I can make arguments for and against either side of this idea, it all depends how you want to run your business.
- dirtybirdnj 3y agoI had a proof of concept for us to migrate sites to a static / low / no cost hosting option. They were unwilling to spend time "training" the team to learn react (in 2020). They were unwilling to let their senior FE dev spend any time with me to correct the CSS issues I was struggling with. They used this deception and dishonesty to say "it didn't work" and wasn't worth any more time. I build a prototype in a week. It's not like I spent month(s) on it without any ROI. They just wouldn't look at it because that would mean acknowledging I and or / my ideas had value. The closest thing I got to an answer is that the CMS they preferred, which was chosen 10 years ago by people no longer working there... was the only way they could support client sites. Because that's what they've been using. Turnover means it's so hard for us to support anything new because we have no time... Basically they Brawndo'd me. It was hard to stomach getting fired by the incompetent people driving the business into the ground when I was literally pleading with them to implement money saving measures. The horse sometimes would rather kill itself than drink the clean water you've found... that's just life. It's hard to accept. https://i.redd.it/h2lmsdwqtci81.jpg https://i.redd.it/h2lmsdwqtci81.jpg
- User23 3y agoThis is an interesting way of saying that contemporary web “best practices” are a giant waste of money.
- hanniabu 3y ago> The one "liability" to doing this would be for visitors to not realize the site isn't being updated anymore. Nothing a simple notification bar can't solve
- 8note 3y agoI imagine they still have to organize paying royalties and the like
- DennisP 3y agoI don't understand why publishers take down Kindle books when the paper book goes out of print. It happened to one of my favorite scifi novels. It takes zero effort to keep the book available (I know, I self-published a silly little one), and zero effort to include it in your accounting as long as there's a data feed and a computer.
- mdavidn 3y agoHalting digital sales might be necessary to declare a write-off and recoup a tax benefit that year. That was happening to some streaming shows, anyway. Which sci-fi novel?
- DennisP 3y agoThe Golden Age trilogy by John C. Wright. I read one of his later works and it was horrid, but Golden Age was incredible. The first several chapters were hard to get into, I quit several times and the people I loaned it to never got past that. But after that it takes off like a rocket. On a reread I found that the difficulty at first was just from so much being unfamiliar. It's in a distant future with superintelligent AI, immortality, physical abundance and pervasive virtual reality. And in that setting he finds a deeply human tale of epic heroism.
- zoky 3y agoIt probably has to do with publishing rights. Authors may not want to allow digital publication without an actual print run, and once the initial print run ends they lose the digital rights unless they do another print run. Or the digital publication rights may only be negotiated for a fixed period of time, or else require an ongoing fixed payment to retain, so it costs the publisher money to continue offering a digital version that they may not recoup without sufficient sales.
- freddie_mercury 3y agoUsually it is because the rights have reverted to the author and the publisher no longer has rights to publish the book. Meanwhile, the author doesn't own the rights to the cover art/design so they can't just put it up either. I'm not sure how else you'd want to handle it.
- greenknight 3y agoOr even Digg... Lots of comments and discussions lost.
- Scoundreller 3y agoThis is why I post on slashdot. They've passed the test of time (but not UIs, fuck beta). Looks like their first posts in '97 start here: https://slashdot.org/?page=8582 https://slashdot.org/?page=8582 dunno what their december 31st, 1969 posts are after that (errors? intentional de-ranking?) Newspapers have a bad history of "experimenting" with enabling online comments and then deciding the experiment failed and delete them all. You're a newspaper, you're not supposed to delete history when you don't like it!!! And then they complain when people use social media as their newspaper.
- E39M5S62 3y agoThe last time I looked at Slashdot comments (2021, give or take) they were low-effort trolls, racist/sexist, or just gibberish. Has moderation improved there or is it still a cesspool?
- kys_now 3y ago[dead]
- jasonfarnon 3y agoIt's very depressing that all the comment sections from the late 2000s to mid 2010s are nowhere to be found. Also a lot of live journal type sites. Comment sections seem omitted from Internet Archive snapshots, but I find them in many ways more worthy of archival than the published articles that make the cut.
- ajmurmann 3y agoThat type of data would be so interesting for things like historical sentiment analysis
- jonnyone 3y agoThose Yahoo Groups could be a trove of niche, otherwise uncollected information, especially with regard to vintage or specialty electronics. Removing them was a huge loss.
- kevin_thibedeau 3y agoIt's all on groups.io
- syntheticnature 3y agoGroup owners had to import; it was not all preserved. There was some effort done with public groups by the ArchiveTeam, but I know they didn't get everything as there was no good way to find all groups.
- dpezet 3y ago> "I did a giant curl job, recursively regex'd out the login/comment submission fields and dumped them on s3" That is not zero effort. That is at least 1.2 efforts.
- simlevesque 3y ago1.2 efforts which grants you residual revenue for life. Doing it is a no brainer, but some people lack one.
- krisoft 3y agoIt can be also that some decision makers feel it is too much hassle. Or they don’t even know it is an option. For you and me doing this would be probably an afternoon’s work? (Maybe a bit more, maybe a bit less) For someone less technologically inclined it could be seen as a big project. They need to find someone capable of doing it, they need to ask for a quote, they need to supervise the project otherwise the contractor doing it might just do a half assed job or none at all. Not to mention they can only think about doing this if they have an inkling that it is possible. They might be operating in a mindspace where “maintaining the servers” is a large monthly expense. For example if years ago they were sold a CMS with all the bells and wistles for some $bigbucks recurring cost. If they are savvy business people they might have done their research to figure out if “this” can be done cheaper or not. But they might not realise that it is possible to change the requirements such to achieve a massive reduction in cost. This is especially true if they treat the cost of servers as a kind of black box. And very often the people who are providing them with IT services are not incentivised to tell about this option. Will they tell the business owner that oh by the by for half the monthly recurring cost they are paid the business could find someone who puts the page on ice and for the other half runs it for the next decade? Of course not! Doing so would under cut their income stream. That would be crazy. In fact they might spread all kind of FUD and sabotage attempts at scraping the site.
- HenryBemis 3y agoThe ChatGPTs of this world will solve that. I like to believe that I know a couple of things about a couple of things regarding technology and sometimes I ask ChatGPT or Gemini "how can I do so and so, list/name five pieces of software or technical solutions to do so-and-so". I use it/them as a search engine on steroids. Maybe it is time more people also do so.
- makeitdouble 3y agoIt doesn't sparkle joy to them anymore. I wish I was joking, but reducing the amount of stuff that is owned/managed for the sake of it is a common philosophy. Another way to put it: they're focusing. We think it would have been extremely low effort to keep a static site running, they probably thought not having to think about it at all was worth the loss.
- hughesjj 3y agoThis stuff should go to the national archives imo. Same with reddit and hacker news.
- veggieburrito 3y agoHonest, technical, humorous; Good comment!
- RankingMember 3y agoEver more bummed about Fatwallet as Slickdeals continues to worsen with endless "sponsored deals" and censorship to force people to use their cashback and price tracking products.
- Scoundreller 3y agoI wonder what SuckIsStaples is up to?
- Marsymars 3y agoOne of the relatively rare times being Canadian pays off for online resources - RedFlagDeals.com is still doing well in their forums.
- 2-718-281-828 3y agobut how is babby formed?
- blauditore 3y ago> And I'm someone that would take 9 attempts to do FizzBuzz Hehe
- junon 3y agoHoly shit I totally forgot yahoo answers was a thing. That was indeed a shame, even though most of it was complete garbage.
- jstummbillig 3y agoIt takes continuous effort/money to keep metaphorical company lights on. There is things you need to file periodically, legalities you need to comply with, also when they change in the future, just to exist. So if the website shutdown coincides with the shutdown of a company or a division within the company, that might be why. And since a website will usually not shut down if it's turning a considerable profit to begin with, just deleting everything can often look like the best option.
- micromacrofoot 3y ago> did a giant curl job, recursively regex'd out the login/comment submission fields and dumped them on s3 it's because even this cost outweighs how much they care about the content, which is 0... the people who make decisions like this aren't scrimping pennies or interested in preserving effort... they're looking for the simplest way to get millions of dollars into their pockets doing the bare minimum to maintain a library of content indefinitely isn't it these are the kind of people who would happily set fire to a library if they could get away with insurance fraud
- AbstractH24 3y ago> Yahoo Answers OMG I haven't thought about the site in a number of years. Used to be such a common thing to see in google search results.