5 ms·

> TEXTFILES.COM has been online for nearly 25 years with no ads or clickthroughs. If you feel like donating to its roughly $1200 yearly upkeep: Paypal or Venmo.

> TEXTFILES.COM has been online for nearly 25 years with no ads or clickthroughs. If you feel like donating to its roughly $1200 yearly upkeep: Paypal or Venmo. Performance-wise, this doesn’t seem like a $100/month site, considering it’s only simple HTML and text files.

TehCorwiz 3y ago
It’s probably outgoing bandwidth costs.
- mmcdermott 3y ago
  It probably gets mirrored a lit too, given the target audience.
  - ghaff 3y ago
    As I recall, Jason asks people not to do wholesale site copies in general, but I imagine a great number do.
    - 0xEF 3y ago
      You imagine correctly. I cruise open directories regularly, usually looking for books to add to my growing collection (note: books I will actually read/use), generally taking only 2 or 3 books with me when I leave the site. At least once a month, I head to an OD I like to grab a new book or two only to find the host shut the gates because it got posted on r/opendirectories and a bunch of people did pointless site rips, as though they'll read 40k pdf files or whatever. I often wonder if they do it because data caches like that are small currency on other parts of the Internet, sort of the way you had to maintain an ul/dl ratio to stay on a well-maintained BBS back in the day. Who knows. They're certainly not rehosting the libraries in any useful way, I can say that much.
      - ghaff 3y ago
        I think a lot of it is just that many people have a pretty deeply ingrained data hoarding impulse where collecting files--any files--for themselves is the end goal.
        crazygringo 3y ago
        I don't know if it's always hoarding. A lot of stuff on the internet simply disappears. Especially when it's hosted by a single person, or an org that might not be around in 3 years, or they get takedowns (whether legal or not). If you find something you'd like to keep accessing, downloading it all in advance can be a smart move.
        ghaff 3y ago
        So download specific items as the parent says (and as you suggest). But reflexive "download it all just in case" probably isn't helpful especially if it's just for you. I'm not disagreeing with downloading stuff that you want, even maybe site sub-sections. But a lot of the time it turns into just doing a mass download.
- marginalia_nu 3y ago
  Given it's mostly text, it's curious there's no content-encoding applied to HTTP responses. Would reduce bandwidth by something like 70-90% in most cases. Though it's hard to say what's the best configuration without understanding the hardware context.
  - bombcar 3y ago
    Above it’s noted at being north of 300gb, most cheap cloud providers will give you quite a bit of bandwidth for a 300gb server.
    - marginalia_nu 3y ago
      Might be a point to self-host something like this. The value of the project is its extreme longevity, and the half-life of external hosting services is probably something like 5 years.
surteen 3y ago
That site hosts a lot of historical material that many "budget" providers would probably balk at and kick you off their network without much recourse. Having a good relationship with a provider that knows you and will go to bat for you can cost some money. That said, $100/month is not terribly expensive for dedicated hosting.
- deleted 3y ago
  [deleted]
- Solvency 3y ago
  Couldn't you just host all of these files on Github under the pretense of readme files? Would they throw a fit at this?
LukeShu 3y ago
$ rsync --dry-run -a --stats rsync://rsync.textfiles.com/textfiles ./textfiles.com … Number of files: 866,935 (reg: 826,831, dir: 40,083, link: 21) … Total file size: 319,889,987,473 bytes …
- crazygringo 3y ago
  Ah ha, that explains it. Thanks. Even on Digital Ocean, that would be $50 for the 500 GB of block storage, another $20 for keeping a backup, and ~$6 for the actual droplet for hosting. And then some extra bandwidth costs, if people try to bulk-download. Adds up.
- textfiles 3y ago
  Just as a note, you're only getting some of the data that way. The contents of textfiles.com are roughly 2 terabytes of *.textfiles.com endeavors and another 13 terabytes of mirrors and hosting I do.
deleted 3y ago
[deleted]
textfiles 3y ago
As people are wildly speculating in this thread (it is Hackernews, after all), I'll just say that yes, more than you can imagine, my provider has been instrumental on the site staying up through a number of challenges and controversies and that number is my number to keep the site there with the features and arrangements I have. Regardless of what people think the Internet is like in 2024, there are vanishingly small places that will host a site like TEXTFILES.COM for very long.

TehCorwiz 3y ago

It’s probably outgoing bandwidth costs.

mmcdermott 3y ago
It probably gets mirrored a lit too, given the target audience.
- ghaff 3y ago
  As I recall, Jason asks people not to do wholesale site copies in general, but I imagine a great number do.
  - 0xEF 3y ago
    You imagine correctly. I cruise open directories regularly, usually looking for books to add to my growing collection (note: books I will actually read/use), generally taking only 2 or 3 books with me when I leave the site. At least once a month, I head to an OD I like to grab a new book or two only to find the host shut the gates because it got posted on r/opendirectories and a bunch of people did pointless site rips, as though they'll read 40k pdf files or whatever. I often wonder if they do it because data caches like that are small currency on other parts of the Internet, sort of the way you had to maintain an ul/dl ratio to stay on a well-maintained BBS back in the day. Who knows. They're certainly not rehosting the libraries in any useful way, I can say that much.
    - ghaff 3y ago
      I think a lot of it is just that many people have a pretty deeply ingrained data hoarding impulse where collecting files--any files--for themselves is the end goal.
      - crazygringo 3y ago
        I don't know if it's always hoarding. A lot of stuff on the internet simply disappears. Especially when it's hosted by a single person, or an org that might not be around in 3 years, or they get takedowns (whether legal or not). If you find something you'd like to keep accessing, downloading it all in advance can be a smart move.
        ghaff 3y ago
        So download specific items as the parent says (and as you suggest). But reflexive "download it all just in case" probably isn't helpful especially if it's just for you. I'm not disagreeing with downloading stuff that you want, even maybe site sub-sections. But a lot of the time it turns into just doing a mass download.
marginalia_nu 3y ago
Given it's mostly text, it's curious there's no content-encoding applied to HTTP responses. Would reduce bandwidth by something like 70-90% in most cases. Though it's hard to say what's the best configuration without understanding the hardware context.
- bombcar 3y ago
  Above it’s noted at being north of 300gb, most cheap cloud providers will give you quite a bit of bandwidth for a 300gb server.
  - marginalia_nu 3y ago
    Might be a point to self-host something like this. The value of the project is its extreme longevity, and the half-life of external hosting services is probably something like 5 years.

surteen 3y ago

That site hosts a lot of historical material that many "budget" providers would probably balk at and kick you off their network without much recourse. Having a good relationship with a provider that knows you and will go to bat for you can cost some money. That said, $100/month is not terribly expensive for dedicated hosting.

deleted 3y ago
[deleted]
Solvency 3y ago
Couldn't you just host all of these files on Github under the pretense of readme files? Would they throw a fit at this?

LukeShu 3y ago

$ rsync --dry-run -a --stats rsync://rsync.textfiles.com/textfiles ./textfiles.com … Number of files: 866,935 (reg: 826,831, dir: 40,083, link: 21) … Total file size: 319,889,987,473 bytes …

crazygringo 3y ago
Ah ha, that explains it. Thanks. Even on Digital Ocean, that would be $50 for the 500 GB of block storage, another $20 for keeping a backup, and ~$6 for the actual droplet for hosting. And then some extra bandwidth costs, if people try to bulk-download. Adds up.
textfiles 3y ago
Just as a note, you're only getting some of the data that way. The contents of textfiles.com are roughly 2 terabytes of *.textfiles.com endeavors and another 13 terabytes of mirrors and hosting I do.

deleted 3y ago

[deleted]

textfiles 3y ago

As people are wildly speculating in this thread (it is Hackernews, after all), I'll just say that yes, more than you can imagine, my provider has been instrumental on the site staying up through a number of challenges and controversies and that number is my number to keep the site there with the features and arrangements I have. Regardless of what people think the Internet is like in 2024, there are vanishingly small places that will host a site like TEXTFILES.COM for very long.