5 ms·

It's from this tos page: https://www.cloudflare.com/terms/ https://www.cloudflare.com/terms/ 2.8 Limitation on Serving Non-HTML Content ...Use of the Services

by tyingq 4y ago

It's from this tos page: https://www.cloudflare.com/terms/ https://www.cloudflare.com/terms/ 2.8 Limitation on Serving Non-HTML Content ...Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately... A huge text/plain artifact, requested often, would seem to fall into that category of "disproportionate percentage" compared to text/html served.

bornfreddy 4y ago
Sounds like it is meant to deal with multimedia mostly? But anyway, just rename .txt to .html and you're done.
- Maxburn 4y ago
  Simple but will it will break all sorts of automation down the line? All the other adlists are txt and I don't know how they would handle other file types, even if the content is unchanged.
  - PaulDavisThe1st 4y ago
    Determining file type from the file name suffix is a fool's game and always was.
    - ethbr0 4y ago
      Is it? Seems superior to arbitrary magic numbers or headers, and God forbid full naive parsing, in most ways.
      - mannykannot 4y ago
        I doubt there is any solution that is both robust and simple. In a sense, it is the same problem as that which ad blockers are attempting to solve.
      - semi 4y ago
        Whats wrong with storing and delivering the intended content type as metadata, whether thats headers or filesystem metadata like in Mac OS X?
        ethbr0 4y ago
        Transmitting in-band (headers) seems ripe for arbitrary complexity. Someone out there would write a Turing-complete header DSL. And then someone else would write an incompatible alternative implementation. At least file extension is limited and externally visible (and thus accountable) to third party behavior, which should limit the worst complexity excesses. Is filesystem metadata actually different (theoretically) from extension? Or just data in a different format? Extension seems a nice balance between simplicity / brevity and utility, albeit as a hint, not a commandment.
    - Maxburn 4y ago
      I hear you there. I'm more thinking someone probably hard coded txt file extension somewhere so something is likely to fall apart in simply handling that file.
    - tomschwiha 4y ago
      Fun stuff like embedding data into jpgs or pngs.
- tyingq 4y ago
  I imagine that might help with automated tos rate limiting, but eventually someone at Cloudflare will probably cut them off. It's plain text, but it's basically serving a distributed database. And a hint at their scale is "100TB of “Access Denied” served up monthly. Cloudflare just seems to be trying to limit the free tier to "caching website html for the purpose of showing it to humans". They have pricing and plans for things other than that.
- kenmacd 4y ago
  Then CF replaces the html with their Browser Integrity Check. How does the app deal with the list becoming real 'Checking your browser" html?
tomschwiha 4y ago
Cloudflare can decide whom they want to do business with. But a plain text file is in my opinion sort of HTML. At least it is not "non-html" content. A .pdf file would be non-HTML content. What else is important to note that the client is being abused and not the client abusing the service. That should be taken into consideration, when deciding if someone is breaking the ToS.
- lcnPylGDnU4H9OF 4y ago
  I'd agree that's weird. Seems like if it were simply renamed to .html with no content changes, then it would be okay. > What else is important to note that the client is being abused and not the client abusing the service. That should be taken into consideration, when deciding if someone is breaking the ToS. My understanding has this as moot. The issue from Cloudflare's perspective is only that the content is non-HTML and doesn't have anything to do with the rate of traffic (the abuse).
  - bombcar 4y ago
    > (i) serving web pages as viewed through a web browser or other functionally equivalent applications, including rendering Hypertext Markup Language (HTML) or other functional equivalents, and (ii) serving web APIs subject to the restrictions set forth in this Section 2.8. The key is "as viewed through a web browser" imo, this is not really an API and it's not a webpage; it's a datafile and would fall into R2 or similar things.
    - lcnPylGDnU4H9OF 4y ago
      I see, that makes the position more understandable. I guess the same rule would (should) apply if they did indeed simply change the extension.
    - Spunkie 4y ago
      Why do people keep talking like you can't just navigate to a txt file in your browser and have it serve as any old web content? Which is something I have actually done many years ago to search for a domain in these types of lists. Cloudflare is balancing on a razer for this TOS technicality.
ignoramous 4y ago
This limitation apparently doesn't apply to R2 / Workers [0]. May be EasyList could host them there? That's what we do [1] (and the dashboards show 400TB+ per mo [2], likely rigged by the traffic between Workers and Cloudflare Cache). [0] https://news.ycombinator.com/item?id=20791660 https://news.ycombinator.com/item?id=20791660 [1] https://news.ycombinator.com/item?id=30034547 https://news.ycombinator.com/item?id=30034547 [2] https://nitter.net/rethinkdns/status/1546232186554417152 https://nitter.net/rethinkdns/status/1546232186554417152
Slix 4y ago
This doesn't sound right to me. Cloudflare also protects web APIs. This text file is an extremely simple web API, but it is still a web API.
- tyingq 4y ago
  If the web apis were a disproportionate amount of what was served for some customers specific free CF plan, as compared to the cached HTML, then that doesn't match their TOS.
tomrod 4y ago
If I can read it in Lynx, it is web content.
spatley 4y ago
My best guess is that CloudFlare wrote this to prevent folks from serving big binary files like photo, music, or video and this txt file case was an unintended condition that happens to work to CloudFlare's advantage. text/plain though is decidedly not text/html and I would expect CloudFlare to potentially do some on-the- fly optimizations that are aware of the structure of an html file that save terabytes a day at their scale.
- ignoramous 4y ago
  > My best guess is... Some think its very Oracle of Cloudflare to do so. I do not blame them.
layer8 4y ago
The solution seems simple, just wrap it in a trivial HTML envelope. Enclose it in <pre> tags if needed.
arendtio 4y ago
From a legal perspective I can understand such a wording, but I wonder why an engineer simply tells a (non-paying) customer that he violates the ToS, without thinking about it. I mean, one could simply wrap the content in a HTML body and change the extension, but that would actually increase the data load for no good reason. So it is complete non-sense to complain about txt files being served.

by tyingq 4y ago

bornfreddy 4y ago

Sounds like it is meant to deal with multimedia mostly? But anyway, just rename .txt to .html and you're done.

Maxburn 4y ago
Simple but will it will break all sorts of automation down the line? All the other adlists are txt and I don't know how they would handle other file types, even if the content is unchanged.
- PaulDavisThe1st 4y ago
  Determining file type from the file name suffix is a fool's game and always was.
  - ethbr0 4y ago
    Is it? Seems superior to arbitrary magic numbers or headers, and God forbid full naive parsing, in most ways.
    - mannykannot 4y ago
      I doubt there is any solution that is both robust and simple. In a sense, it is the same problem as that which ad blockers are attempting to solve.
    - semi 4y ago
      Whats wrong with storing and delivering the intended content type as metadata, whether thats headers or filesystem metadata like in Mac OS X?
      - ethbr0 4y ago
        Transmitting in-band (headers) seems ripe for arbitrary complexity. Someone out there would write a Turing-complete header DSL. And then someone else would write an incompatible alternative implementation. At least file extension is limited and externally visible (and thus accountable) to third party behavior, which should limit the worst complexity excesses. Is filesystem metadata actually different (theoretically) from extension? Or just data in a different format? Extension seems a nice balance between simplicity / brevity and utility, albeit as a hint, not a commandment.
  - Maxburn 4y ago
    I hear you there. I'm more thinking someone probably hard coded txt file extension somewhere so something is likely to fall apart in simply handling that file.
  - tomschwiha 4y ago
    Fun stuff like embedding data into jpgs or pngs.
tyingq 4y ago
I imagine that might help with automated tos rate limiting, but eventually someone at Cloudflare will probably cut them off. It's plain text, but it's basically serving a distributed database. And a hint at their scale is "100TB of “Access Denied” served up monthly. Cloudflare just seems to be trying to limit the free tier to "caching website html for the purpose of showing it to humans". They have pricing and plans for things other than that.
kenmacd 4y ago
Then CF replaces the html with their Browser Integrity Check. How does the app deal with the list becoming real 'Checking your browser" html?

tomschwiha 4y ago

Cloudflare can decide whom they want to do business with. But a plain text file is in my opinion sort of HTML. At least it is not "non-html" content. A .pdf file would be non-HTML content. What else is important to note that the client is being abused and not the client abusing the service. That should be taken into consideration, when deciding if someone is breaking the ToS.

lcnPylGDnU4H9OF 4y ago
I'd agree that's weird. Seems like if it were simply renamed to .html with no content changes, then it would be okay. > What else is important to note that the client is being abused and not the client abusing the service. That should be taken into consideration, when deciding if someone is breaking the ToS. My understanding has this as moot. The issue from Cloudflare's perspective is only that the content is non-HTML and doesn't have anything to do with the rate of traffic (the abuse).
- bombcar 4y ago
  > (i) serving web pages as viewed through a web browser or other functionally equivalent applications, including rendering Hypertext Markup Language (HTML) or other functional equivalents, and (ii) serving web APIs subject to the restrictions set forth in this Section 2.8. The key is "as viewed through a web browser" imo, this is not really an API and it's not a webpage; it's a datafile and would fall into R2 or similar things.
  - lcnPylGDnU4H9OF 4y ago
    I see, that makes the position more understandable. I guess the same rule would (should) apply if they did indeed simply change the extension.
  - Spunkie 4y ago
    Why do people keep talking like you can't just navigate to a txt file in your browser and have it serve as any old web content? Which is something I have actually done many years ago to search for a domain in these types of lists. Cloudflare is balancing on a razer for this TOS technicality.

ignoramous 4y ago

This limitation apparently doesn't apply to R2 / Workers [0]. May be EasyList could host them there? That's what we do [1] (and the dashboards show 400TB+ per mo [2], likely rigged by the traffic between Workers and Cloudflare Cache). [0] https://news.ycombinator.com/item?id=20791660 https://news.ycombinator.com/item?id=20791660 [1] https://news.ycombinator.com/item?id=30034547 https://news.ycombinator.com/item?id=30034547 [2] https://nitter.net/rethinkdns/status/1546232186554417152 https://nitter.net/rethinkdns/status/1546232186554417152

Slix 4y ago

This doesn't sound right to me. Cloudflare also protects web APIs. This text file is an extremely simple web API, but it is still a web API.

tyingq 4y ago
If the web apis were a disproportionate amount of what was served for some customers specific free CF plan, as compared to the cached HTML, then that doesn't match their TOS.

tomrod 4y ago

If I can read it in Lynx, it is web content.

spatley 4y ago

My best guess is that CloudFlare wrote this to prevent folks from serving big binary files like photo, music, or video and this txt file case was an unintended condition that happens to work to CloudFlare's advantage. text/plain though is decidedly not text/html and I would expect CloudFlare to potentially do some on-the- fly optimizations that are aware of the structure of an html file that save terabytes a day at their scale.

ignoramous 4y ago
> My best guess is... Some think its very Oracle of Cloudflare to do so. I do not blame them.

layer8 4y ago

The solution seems simple, just wrap it in a trivial HTML envelope. Enclose it in <pre> tags if needed.

arendtio 4y ago

From a legal perspective I can understand such a wording, but I wonder why an engineer simply tells a (non-paying) customer that he violates the ToS, without thinking about it. I mean, one could simply wrap the content in a HTML body and change the extension, but that would actually increase the data load for no good reason. So it is complete non-sense to complain about txt files being served.