5 ms·

Hello HN. In this article I try to explain HATEOAS in its original context, hypermedia/HTML, rather than using JSON. JSON isn't a natural hypermedia and it ma

Hello HN. In this article I try to explain HATEOAS in its original context, hypermedia/HTML, rather than using JSON. JSON isn't a natural hypermedia and it makes HATEOAS (and the entire uniform interface of REST) difficult to understand and of frustratlingly little benefit to end users. I don't expect to change the language around REST-ful APIs, where darn near anything over HTTP is called "REST-ful", but I hope that this article will help people who have struggled with understanding what HATEOAS is or why on earth it might be an interesting and useful technology. In its natural environment, HTML, it is much easier to understand.

bityard 5y ago
First let me say I love htmx. It's nice to be able to write a little web app with minimal javascript and fuss. It's all well and good to have your API return HTML when your only client is a web browser. And for many applications (like many of mine) that is enough. But HTTP APIs are most powerful when they are client-agnostic. JSON is terrible in so many ways but it is at least neutral ground: it can be consumed by any HTTP client (which need not be a web browser) and every programming language that matters. For my day job, I deal with a great many HTTP APIs that are never seen or used by a browser. Authentication systems, data storage, schedulers, etc I can write a Python script that turns a JSON API response into a native Python object with a single line of code. How can you do _that_ with a HATEOAS response that is basically guaranteed to contain arbitrary HTML?
- littlecosmic 5y ago
  With htmx the size and complexity of web client should be radically reduced compared to a heavy JS client app consuming the JSON API. I reckon this could free one up to make an API that was specialised for native clients and possibly even the web server itself… maybe no JSON in sight, unless you were keen.
- zczc 5y ago
  If the response is xhtml then de-serialization as easy as with json
  - acdha 5y ago
    I think XHTML is largely a distraction here since the 2000s. HTML5 standardized parsing and the mechanisms for including non-display data. Since the decline of IE8 or so, almost all of the work I’ve had working with data in HTML has been due to differences in the data models, which is also a problem using JSON or XML.
- hliyan 5y ago
  What if we simply move the JSON -> HTML translation layer from the client, to a layer just before egress from the server? Maybe we provide two output options for each endpoint: /foo/1/bar (HTML by default) and /foo/1/bar?format=json
  - klibertp 5y ago
    ...that's a mechanism already built into HTTP in the form of Accept header. There's something called "content negotiation"[1], which seems to be there since the 90s. Not sure about other frameworks, but the Django REST[2] Framework at least implements this correctly and can serialize to XML or JSON transparently. [1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_ne... [2] Of course, the framework has nothing to do with REST at all. I gave up telling people this many years ago - it just doesn't work... Mentioning it here since it's relevant to the topic.
    - acdha 5y ago
      Note however that content negotiation has historically been complicated by things like caching or client oddities setting headers. Many of the projects where I’ve used it on have ended up adding a way to specify the format in the URL, although that has become less of a concern over time.
    - deleted 5y ago
      [deleted]
  - dsego 5y ago
    What about /bar.html and /bar.json ?
- dragonwriter 5y ago
  > I can write a Python script that turns a JSON API response into a native Python object with a single line of code. How can you do _that_ with a HATEOAS response that is basically guaranteed to contain arbitrary HTML? HATEOAS can be (and often is) done with JSON or JSON-based (e.g., JSON+JSONSchema) representation for “backbone” resources (the same resources might also be available in HTML; HATEOAS and HTTP both are designed to accommodate representation being flexible for the same resource. But, sure, you can also do a one-line parse of raw HTML into a python object. Just as with raw JSON, you’ll probably lose some intended semantics, with either you’ll need to use a format based on the root format but with more tightly defined semantics to do a one-line client that doesn't lose meaning. For JSON, JSON+JSONSchema may be enough for many applications. For HTML, Microformats2 may be enough for many applications.
  - ElectricalUnion 5y ago
    > Microformats2 Not only that, those metadata are basically enforced by the big closed garden internet players if you want to use any nice embedding; so you need those anyways.
- jfrunyon 5y ago
  Frankly, I think that HTML can likely be consumed in more languages than JSON by sheer virtue of age. And there are any number of ways you could standardize the format of your HTML response - such that people who know the formatting ahead of time get the exact same benefits as they would with JSON.
- naasking 5y ago
  > I can write a Python script that turns a JSON API response into a native Python object with a single line of code. How can you do _that_ with a HATEOAS response that is basically guaranteed to contain arbitrary HTML? These aren't equivalent scenarios. The JSON response is a programmatic endpoint so of course it only contains programmatically-relevant data. Typically the HTML is not from a programmatic endpoint, but it could be. Strip out all styling and non-programmatic content from an endpoint that returns HTML and you basically have pseudo-XML document that can just easily be mapped to a native object as JSON, with the added benefit that you can actually identify links without needing a separate schema as you do with JSON. This isn't typically done but there's no reason it can't be done. In fact, it would be considerably more powerful than JSON because you can actually transmit functions. For instance, a <form> input and its associated inputs can be translated into a native function with parameters whose names correspond to the input names that makes a remote call. So theoretically, hypertext is JSON + first-class functions. This is why HATEOAS is more flexible.
- recursivedoubts 5y ago
  RPC has its place in life, but hypermedia is extremely powerful for application development. I believe many systems would be best served by providing both: a hypermedia-based web application and a JSON-based data API. This essay on the topic may be of interest to you: https://htmx.org/essays/splitting-your-apis/ https://htmx.org/essays/splitting-your-apis/
- ElectricalUnion 5y ago
  > How can you do _that_ with a HATEOAS response that is basically guaranteed to contain arbitrary HTML? https://pypi.org/project/mechanize/ https://pypi.org/project/mechanize/
badbotty 5y ago
Have you looked into JSON-ld? Sure it is an extension of json but it has a formal specification which solves all the problems you have with your json examples, specifically the out-of-band issue you have with your json and how to link to other things. FWIW I maintain a API and frontend which use JSON-ld and has the backend passing the information I need to render forms based of backend permissions like you are describing. What your html isn't doing for me is give proper type information. Should I look for the text "Status: x" in some div to find the status, and how is this better than having "status" as a key in some json object? I would not enjoy developing a consumer which had to interface with an api nesting data in divs.
- brabel 5y ago
  > What your html isn't doing for me is give proper type information. Should I look for the text "Status: x" in some div to find the status, Oh wow, you seem to have come out of reading this blog post still extremely confused. In HATEOAS, the client doesn't care at all about whether the page contains a "status". It displays the hypermedia the way it is given without knowing the semantics at all - that's what makes it de-coupled from the server. It only knows the semantics of the hypermedia, in this case HTML. In cases where you want a single-purpose client that can automatically take action depending on your domain's concepts (like account status) you simply can't use HATEOAS, go with RPC, which is a much better choice, or, as most people today are doing, stateless JSON over HTTP (which is not REST as most developers today seem to believe).
  - badbotty 5y ago
    > Oh wow, you seem to have come out of reading this blog post still extremely confused. Yea, guilty of not reading the article properly sorry. I now get what you are saying about not using the status to drive what forms appear in UI / actions can be performed. Using JSON-ld you might use the https://schema.org/potentialAction https://schema.org/potentialAction property to describe what actions the resources allows which provides the equivalent information as the HTML form tag. After reading Roy Fielding's blog a bit I do find my understanding of REST to be lacking. My reading of the principles agrees with your statement: "It displays the hypermedia without knowing semantics at all" but then he seems to also endorse RDF and N3 which are more like JSON-ld from my understanding and I'm not sure how they cater to display purposes. > When representations are provided in hypertext form with typed relations (using microformats of HTML, RDF in N3 or XML, or even SVG), then automated agents can traverse these applications almost as well as any human. There are plenty of examples in the linked data communities. More important to me is that the same design reflects good human-Web design, and thus we can design the protocols to support both machine and human-driven applications by following the same architectural style. https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...

bityard 5y ago

First let me say I love htmx. It's nice to be able to write a little web app with minimal javascript and fuss. It's all well and good to have your API return HTML when your only client is a web browser. And for many applications (like many of mine) that is enough. But HTTP APIs are most powerful when they are client-agnostic. JSON is terrible in so many ways but it is at least neutral ground: it can be consumed by any HTTP client (which need not be a web browser) and every programming language that matters. For my day job, I deal with a great many HTTP APIs that are never seen or used by a browser. Authentication systems, data storage, schedulers, etc I can write a Python script that turns a JSON API response into a native Python object with a single line of code. How can you do _that_ with a HATEOAS response that is basically guaranteed to contain arbitrary HTML?

littlecosmic 5y ago
With htmx the size and complexity of web client should be radically reduced compared to a heavy JS client app consuming the JSON API. I reckon this could free one up to make an API that was specialised for native clients and possibly even the web server itself… maybe no JSON in sight, unless you were keen.
zczc 5y ago
If the response is xhtml then de-serialization as easy as with json
- acdha 5y ago
  I think XHTML is largely a distraction here since the 2000s. HTML5 standardized parsing and the mechanisms for including non-display data. Since the decline of IE8 or so, almost all of the work I’ve had working with data in HTML has been due to differences in the data models, which is also a problem using JSON or XML.
hliyan 5y ago
What if we simply move the JSON -> HTML translation layer from the client, to a layer just before egress from the server? Maybe we provide two output options for each endpoint: /foo/1/bar (HTML by default) and /foo/1/bar?format=json
- klibertp 5y ago
  ...that's a mechanism already built into HTTP in the form of Accept header. There's something called "content negotiation"[1], which seems to be there since the 90s. Not sure about other frameworks, but the Django REST[2] Framework at least implements this correctly and can serialize to XML or JSON transparently. [1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_ne... [2] Of course, the framework has nothing to do with REST at all. I gave up telling people this many years ago - it just doesn't work... Mentioning it here since it's relevant to the topic.
  - acdha 5y ago
    Note however that content negotiation has historically been complicated by things like caching or client oddities setting headers. Many of the projects where I’ve used it on have ended up adding a way to specify the format in the URL, although that has become less of a concern over time.
  - deleted 5y ago
    [deleted]
- dsego 5y ago
  What about /bar.html and /bar.json ?
dragonwriter 5y ago
> I can write a Python script that turns a JSON API response into a native Python object with a single line of code. How can you do _that_ with a HATEOAS response that is basically guaranteed to contain arbitrary HTML? HATEOAS can be (and often is) done with JSON or JSON-based (e.g., JSON+JSONSchema) representation for “backbone” resources (the same resources might also be available in HTML; HATEOAS and HTTP both are designed to accommodate representation being flexible for the same resource. But, sure, you can also do a one-line parse of raw HTML into a python object. Just as with raw JSON, you’ll probably lose some intended semantics, with either you’ll need to use a format based on the root format but with more tightly defined semantics to do a one-line client that doesn't lose meaning. For JSON, JSON+JSONSchema may be enough for many applications. For HTML, Microformats2 may be enough for many applications.
- ElectricalUnion 5y ago
  > Microformats2 Not only that, those metadata are basically enforced by the big closed garden internet players if you want to use any nice embedding; so you need those anyways.
jfrunyon 5y ago
Frankly, I think that HTML can likely be consumed in more languages than JSON by sheer virtue of age. And there are any number of ways you could standardize the format of your HTML response - such that people who know the formatting ahead of time get the exact same benefits as they would with JSON.
naasking 5y ago
> I can write a Python script that turns a JSON API response into a native Python object with a single line of code. How can you do _that_ with a HATEOAS response that is basically guaranteed to contain arbitrary HTML? These aren't equivalent scenarios. The JSON response is a programmatic endpoint so of course it only contains programmatically-relevant data. Typically the HTML is not from a programmatic endpoint, but it could be. Strip out all styling and non-programmatic content from an endpoint that returns HTML and you basically have pseudo-XML document that can just easily be mapped to a native object as JSON, with the added benefit that you can actually identify links without needing a separate schema as you do with JSON. This isn't typically done but there's no reason it can't be done. In fact, it would be considerably more powerful than JSON because you can actually transmit functions. For instance, a <form> input and its associated inputs can be translated into a native function with parameters whose names correspond to the input names that makes a remote call. So theoretically, hypertext is JSON + first-class functions. This is why HATEOAS is more flexible.
recursivedoubts 5y ago
RPC has its place in life, but hypermedia is extremely powerful for application development. I believe many systems would be best served by providing both: a hypermedia-based web application and a JSON-based data API. This essay on the topic may be of interest to you: https://htmx.org/essays/splitting-your-apis/ https://htmx.org/essays/splitting-your-apis/
ElectricalUnion 5y ago
> How can you do _that_ with a HATEOAS response that is basically guaranteed to contain arbitrary HTML? https://pypi.org/project/mechanize/ https://pypi.org/project/mechanize/

badbotty 5y ago

Have you looked into JSON-ld? Sure it is an extension of json but it has a formal specification which solves all the problems you have with your json examples, specifically the out-of-band issue you have with your json and how to link to other things. FWIW I maintain a API and frontend which use JSON-ld and has the backend passing the information I need to render forms based of backend permissions like you are describing. What your html isn't doing for me is give proper type information. Should I look for the text "Status: x" in some div to find the status, and how is this better than having "status" as a key in some json object? I would not enjoy developing a consumer which had to interface with an api nesting data in divs.

brabel 5y ago
> What your html isn't doing for me is give proper type information. Should I look for the text "Status: x" in some div to find the status, Oh wow, you seem to have come out of reading this blog post still extremely confused. In HATEOAS, the client doesn't care at all about whether the page contains a "status". It displays the hypermedia the way it is given without knowing the semantics at all - that's what makes it de-coupled from the server. It only knows the semantics of the hypermedia, in this case HTML. In cases where you want a single-purpose client that can automatically take action depending on your domain's concepts (like account status) you simply can't use HATEOAS, go with RPC, which is a much better choice, or, as most people today are doing, stateless JSON over HTTP (which is not REST as most developers today seem to believe).
- badbotty 5y ago
  > Oh wow, you seem to have come out of reading this blog post still extremely confused. Yea, guilty of not reading the article properly sorry. I now get what you are saying about not using the status to drive what forms appear in UI / actions can be performed. Using JSON-ld you might use the https://schema.org/potentialAction https://schema.org/potentialAction property to describe what actions the resources allows which provides the equivalent information as the HTML form tag. After reading Roy Fielding's blog a bit I do find my understanding of REST to be lacking. My reading of the principles agrees with your statement: "It displays the hypermedia without knowing semantics at all" but then he seems to also endorse RDF and N3 which are more like JSON-ld from my understanding and I'm not sure how they cater to display purposes. > When representations are provided in hypertext form with typed relations (using microformats of HTML, RDF in N3 or XML, or even SVG), then automated agents can traverse these applications almost as well as any human. There are plenty of examples in the linked data communities. More important to me is that the same design reflects good human-Web design, and thus we can design the protocols to support both machine and human-driven applications by following the same architectural style. https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...