4 ms·

Good article, but this not a “trick”, it’s a core part of the HTTP protocol. Worthy of an article non the less, judging by how misunderstood the topic is among

Good article, but this not a “trick”, it’s a core part of the HTTP protocol. Worthy of an article non the less, judging by how misunderstood the topic is among the commenters here.

pletnes 4y ago
My impression is that many power users (not web developers) want to download datasets. Sometimes they’ll use a browser to grab a csv or json file, then stuff it into excel/R/python/… to do some sort of analysis work. This kind of knowledge is helpful for that crowd. They often don’t know about the non-url parts of a request.
Semaphor 4y ago
Yeah, asp.net for example, supports both JSON and XML and relies on the accept header to decide what to send.
jchw 4y ago
Content negotiation itself is not a trick, but this usage of it is fair to characterize as a sort of trick. Most uses of content negotiation are about serving the same content in different formats, like different image formats; in this case though, it's actually negotiating a different thing entirely, where one is a machine-readable CSV and the other a human-readable hypertext document. The fact that it works is unexpected to the uninitialized, because even just reading the RFC will not necessarily make it apparent why cURL and the browser would return such different results. There's a lot of ways a server could detect cURL, like the user agent, and it might come as a surprise to some that cURL and other user agents that are not web browsers often send */* for the accept header. It is not a terribly new way to use content negotiation, but arguably still a clever one. It's a deliberate behavior.
- Joker_vD 4y ago
  > serving the same content in different formats, like different image formats; in this case though, it's actually negotiating a different thing entirely, where one is a machine-readable CSV and the other a human-readable hypertext document Huh? No, it's also serving the same content, just in different formats: one is a machine-readable CSV and the other a human-readable hypertext document.
  - jchw 4y ago
    Disagree: the table inside the HTML is the "content" that comprises the entirety of the CSV. Related but absolutely not "same". (It'd be more arguable if all the HTML had was the table, but it's actually just a normal web page with a table.)
    - Joker_vD 4y ago
      Well, neither a picture is really "the same" if it's encoded in different formats, say, JPEG or PNG.
      - jchw 4y ago
        Actually, that's a pretty good point. The thing is that the URL itself refers to some conceptual resource, and the response is ideally a representation of that resource, potentially one of multiple. If you take the same source image and encode it multiple ways, although the two resulting images are different from eachother, they are representations of the same underlying image/resource. But if you were to provide a different image, or alter the image in other ways, I think this would be pretty tricky actually, even if the modification was something trivial. You can imagine a simple use case like a WebP image with the text "Your browser supports WebP" and a PNG image with the text "Your browser does not support WebP." The point is that content negotiation being used to present different logical resources that are not necessarily interchangable representations of the same data feels tricky. I think that is still compatible with the fact that it's all within specifications.
    - lkitching 4y ago
      Arguably the resource is the dataset of stock exchanges, and the CSV representation is forced to omit all the metadata but the HTML representation isn't.
      - jchw 4y ago
        I understand what people are getting at, as it's not really that big of a logical leap. I think the fact that it is somewhat of a stretch, but still "in the lines," is exactly why it is a "trick": it isn't doing anything particularly invalid or hacky, it's just not necessarily what you'd imagine when reading the RFC. Content negotiation to me is more about serving the optimal content to a given agent, not really about selecting modalities for different use cases based on different types of user agents. I think both cases are "valid" although I think it is inherently less tricky if the document talking about and previewing the dataset is referenced via a separate URL from the dataset itself. (Which, of course, entirely mitigates problems like Apache Spark having HTML in the Accept header.)
agloeregrets 4y ago
This. lol. I opened the article and was ready for a cool trick and then they showed it returning csv over Curl and html over a browser and I was like "So the accept header? Maybe there is another trick...nope, just the accept header." Our API already does this at my company, where it returns an HTML formatted, human readable response over a browser nav but returns application/json over anything requesting only that.
- 0xbadcafebee 4y ago
  I didn't realize you could assign a preference hierarchy... to me that's the trick (well, that browsers do it, anyway)
opportune 4y ago
You would be surprised how little people understand HTTP despite working with it for years.
bazoom42 4y ago
I’m just happy they didn’t call it “a hack”.
antifa 4y ago
I was expecting something less obvious other than "101 class on the HTTP Accept Header" lol.

Good article, but this not a “trick”, it’s a core part of the HTTP protocol. Worthy of an article non the less, judging by how misunderstood the topic is among the commenters here.

pletnes 4y ago

My impression is that many power users (not web developers) want to download datasets. Sometimes they’ll use a browser to grab a csv or json file, then stuff it into excel/R/python/… to do some sort of analysis work. This kind of knowledge is helpful for that crowd. They often don’t know about the non-url parts of a request.

Semaphor 4y ago

Yeah, asp.net for example, supports both JSON and XML and relies on the accept header to decide what to send.

jchw 4y ago

Content negotiation itself is not a trick, but this usage of it is fair to characterize as a sort of trick. Most uses of content negotiation are about serving the same content in different formats, like different image formats; in this case though, it's actually negotiating a different thing entirely, where one is a machine-readable CSV and the other a human-readable hypertext document. The fact that it works is unexpected to the uninitialized, because even just reading the RFC will not necessarily make it apparent why cURL and the browser would return such different results. There's a lot of ways a server could detect cURL, like the user agent, and it might come as a surprise to some that cURL and other user agents that are not web browsers often send */* for the accept header. It is not a terribly new way to use content negotiation, but arguably still a clever one. It's a deliberate behavior.

Joker_vD 4y ago
> serving the same content in different formats, like different image formats; in this case though, it's actually negotiating a different thing entirely, where one is a machine-readable CSV and the other a human-readable hypertext document Huh? No, it's also serving the same content, just in different formats: one is a machine-readable CSV and the other a human-readable hypertext document.
- jchw 4y ago
  Disagree: the table inside the HTML is the "content" that comprises the entirety of the CSV. Related but absolutely not "same". (It'd be more arguable if all the HTML had was the table, but it's actually just a normal web page with a table.)
  - Joker_vD 4y ago
    Well, neither a picture is really "the same" if it's encoded in different formats, say, JPEG or PNG.
    - jchw 4y ago
      Actually, that's a pretty good point. The thing is that the URL itself refers to some conceptual resource, and the response is ideally a representation of that resource, potentially one of multiple. If you take the same source image and encode it multiple ways, although the two resulting images are different from eachother, they are representations of the same underlying image/resource. But if you were to provide a different image, or alter the image in other ways, I think this would be pretty tricky actually, even if the modification was something trivial. You can imagine a simple use case like a WebP image with the text "Your browser supports WebP" and a PNG image with the text "Your browser does not support WebP." The point is that content negotiation being used to present different logical resources that are not necessarily interchangable representations of the same data feels tricky. I think that is still compatible with the fact that it's all within specifications.
  - lkitching 4y ago
    Arguably the resource is the dataset of stock exchanges, and the CSV representation is forced to omit all the metadata but the HTML representation isn't.
    - jchw 4y ago
      I understand what people are getting at, as it's not really that big of a logical leap. I think the fact that it is somewhat of a stretch, but still "in the lines," is exactly why it is a "trick": it isn't doing anything particularly invalid or hacky, it's just not necessarily what you'd imagine when reading the RFC. Content negotiation to me is more about serving the optimal content to a given agent, not really about selecting modalities for different use cases based on different types of user agents. I think both cases are "valid" although I think it is inherently less tricky if the document talking about and previewing the dataset is referenced via a separate URL from the dataset itself. (Which, of course, entirely mitigates problems like Apache Spark having HTML in the Accept header.)

agloeregrets 4y ago

This. lol. I opened the article and was ready for a cool trick and then they showed it returning csv over Curl and html over a browser and I was like "So the accept header? Maybe there is another trick...nope, just the accept header." Our API already does this at my company, where it returns an HTML formatted, human readable response over a browser nav but returns application/json over anything requesting only that.

0xbadcafebee 4y ago
I didn't realize you could assign a preference hierarchy... to me that's the trick (well, that browsers do it, anyway)

opportune 4y ago

You would be surprised how little people understand HTTP despite working with it for years.

bazoom42 4y ago

I’m just happy they didn’t call it “a hack”.

antifa 4y ago

I was expecting something less obvious other than "101 class on the HTTP Accept Header" lol.