11 ms·

I think the article is missing some points with regards to the REST. If the API that the author is building is a REST API then the response for a non-existing

by gls2ro 4y ago

I think the article is missing some points with regards to the REST. If the API that the author is building is a REST API then the response for a non-existing resource is 404. In case of REST the main idea is that if you try to GET a resource by ID then you assume that the resource exists. Thus if it does not exists => 404. It does not matter too much which part of the URL is the one that is causing the URL to be wrong. Thus `api/v11/employees/1` and `/api/v1/employees/100` both are wrong resources. In the first case, asking for `/api/v11/employees/1` is not a search or find to see if there is a V11 and if inside there is employee 1. Building an URL is an intentional choise that includes assumptions, like there is an 'api', that has a 'v11' and inside there are employees and one should have the ID 1. The same goes for later case with employee ID 100. If you ask for an employee with an ID that means you know (or assume that you know) that employee exists. Thus if it does not it should be very level clear => 404. In both cases responding with 200 means something like "I kind understand that you want an URL that is similar with some that I have so it is almost okish". But in REST this is not the case. It is like you are serving some static folders and you want to get the file 100 under /api/v1/employee and that file does not exists. Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. That can be added as a debug message for the developer for example. Of course this is IMHO and I am only addressing REST API. If the API is not supposed to be REST then do whatever you want but make sure you document it well and be consistent.

treis 4y ago
>If the API that the author is building is a REST API then the response for a non-existing resource is 404. The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying. >Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. But then we've lost standardization which is the whole point of the error codes to begin with.
- 0xCMP 4y ago
  Exactly the point I bring up to not do this. However, a viable (but often difficult or unsupported) way to solve this would be changing the HTTP Message so it's clearer the intent of the error. Most frameworks like Django hardcode the response messages.
- Feeble 4y ago
  > But then we've lost standardization which is the whole point of the error codes to begin with. Returning 200 and then using response body to denote missing resource is no different. So you have to choose, either 404 can be invalid path and missing employee; or 200 can be valid data or missing employee. Personally I would prefer the former as 200/OK then indicates success.
  - treis 4y ago
    This is getting into semantics, but IMHO a 200 OK with an empty body is the most correct response in this scenario. Everything worked so you get 200 OK and the most accurate representation of a resource that you know doesn't exist is an empty body. 404 Not found is not exactly correct. I argue that the server found a representation of the object with that representation being "It does not exist".
    - recursive 4y ago
      For no content, I'd argue 204 is even more correct.
      - treis 4y ago
        But we do have content to send. Namely, that the server knows that the resource the client requested doesn't exist.
        dragonwriter 4y ago
        That’s not a “representation of the resource”. That’s a fact about the state of the universe, to wit, that it contains no such resource. Which is what is communicated by a 404.
        treis 4y ago
        Again, 404 does not mean that the universe contains no such resource. It means that this specific server did not find the resource.
- dec0dedab0de 4y ago
  >If the API that the author is building is a REST API then the response for a non-existing resource is 404. The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying. As others have mentioned, you can return anything you want in the body of a 404 to clarify. However the other issues you all mentioned should really be in the 500s with errors.
  - treis 4y ago
    >you can return anything you want in the body of a 404 to clarify This isn't how the world works though. Much development happens on COTS platforms or internal frameworks or whatever. Many of those will have generic error handling, logging, alerting, etc. based off the response code. Your pedantically correct 404 with a clarifying body ruins my day (really weeks) because now I have to chase whoever built those things to fix my problem. I am annoyed and curse your name.
    - nrdvana 4y ago
      Use a better framework? In any good framework, returning a json body with a status code should be a one-liner. One topic the article didn't even touch was flagging bad parameters in the request which didn't make sense on the application level. HTTP has 422 for that. I commonly write things like validate_my_param($c->req->params->{format}) or $c->detach(HTTP => 422, ['Invalid format']); (the HTTP view renders that string either as text/plain or json as {"message":"Invalid format"} depending on the accept header of the request.) I've used that for years and been quite happy with the downstream results. Edit: I should elaborate on the downstream results. On the javascript side, the ajax methods often contain separate success and failure callbacks. You often need to handle the failure callback to ensure the user knows that something broke. If you also have an error path in the success callback, it clutters the code. ajax({ success(data, ...) { if (data.wasActuallyAnError) { // lines // of code // to handle // the error } else useTheData(data); }, error(response, ...) { // lines // of code // to handle // the error } }) vs. ajax({ success(data, ...) { useTheData(data); }, error(response, ...) { // lines // of code // to handle // the error } }) This problem is not insurmountable of course. you could wrap up your error code in helper functions and reduce the boilerplate. You could write a completely generic error handler for your front-end framework and automatically include it in all ajax calls. You could also write a custom ajax method that interprets the 200 status error messages and diverts to the error callback. But I still think the 404/422 status code is a better pattern to start from, since most of the client-side frameworks I've used switch code paths based on the status code.
- jehlakj 4y ago
  Maybe I'm not understanding this, but wouldn't network issues take priority naturally? For example, if there's a problem with a database connection, you'd return 500. How would you know if the resource even exists?
  - treis 4y ago
    Network issues between the client and the server.
    - NoGravitas 4y ago
      Those will never return a 404, though.
      - treis 4y ago
        If your DNS server has the wrong IP for a URL it will.
- quesera 4y ago
  > this error then overlaps with server path routing issues, DNS problems, and general network issues. No webserver will ever return a 404 for any of those cases.
- imetatroll 4y ago
  Historically 404 is returned for static websites. Ergo it is the standard.
unregistereddev 4y ago
I agree - though I also agree with the author that this is an area where REST specifications are a little clunky. Yes, `api/v1/employees/100` should return a 404, because that path represents the location of a specific entity and that entity does not exist. Just as the author thinks it's clunky to return HTTP error codes representing application errors, I think it's clunky to apply application logic to HTTP method semantics. GET, POST, and DELETE were designed as instructions telling a web server how to handle static-ish content, and it shows in their design. Why would a GET have a body? It's a request for the server to return a specific file. This leads to breaking REST standards - for example, search endpoints that logically are GET calls (they return entities), but are implemented as POST methods because the search criteria is complex enough that you wanted a request body. Or bulk delete calls that are similarly implemented as POST methods. REST works best in a "rules are meant to be broken" manner in my opinion. It's not a bad system (which is why it is so common), but mixing application logic with HTTP transport logic does lead to oddities.
- bachmitre 4y ago
  this
- dvlsg 4y ago
  Some applications opted for using GET with request bodies as well, for better or worse. Elasticsearch comes to mind. There have been some attempts to extend the list of http verbs to include something that fits those use cases more naturally - SEARCH and QUERY. QUERY got more traction, if I remember correctly, but I haven't heard much about it in a while. https://www.w3.org/2012/ldp/wiki/Proposal_for_HTTP_QUERY_Verb https://www.w3.org/2012/ldp/wiki/Proposal_for_HTTP_QUERY_Ver...
  - ognarb 4y ago
    SEARCH is used by the WebDAV standard, not that I like WebDAV search syntax https://gist.github.com/CarlSchwan/66fde29d52022c0ef76aec362c01a846 https://gist.github.com/CarlSchwan/66fde29d52022c0ef76aec362...
smashed 4y ago
I think the author is aware and I noticed the term REST does not appear anywhere in the article. The main idea of the article is exactly that he does not agree that http error codes be used for application level errors (as REST principles recommend). I think at this point that ship has sailed. Most HTTP based API will use http error codes in various ways. I would be surprised to not receive a 404 when requesting a resource ID that does not exist for example when consuming a new API.
- TimTheTinker 4y ago
  > I think at this point that ship has sailed. If you were talking about TCP/IP, then perhaps - because much of the internet's infrastructure is hard-coded (even burned into silicon) to use current standards. But application semantics aren't frozen in stone by any means. Servers and clients built atop HTTP are still being written. Why not adopt better patterns that apply properly separate semantics for each layer? I liked the REST ideas when they came out, particularly because they provided a much better and simpler alternative to SOAP. But I think improving patterns where we can is always a good idea.
  - smashed 4y ago
    Of course. Its just that in the general mindset, an hypothetical average dev who needs to call your API won't be surprised to receive a 404 for a missing resource. I might be wrong of course. Either way, a proper api doc/spec should make either approach a non-issue. Personnally, I've switched to graphql where it makes sense. Application level error codes and handled on the graphql layer, not on the HTTP layer, so you could say I've adopted that approach!
that_james 4y ago
I very intentionally avoided the REST vs HTTP RPC debate. This is _specifically_ about HTTP APIs. REST is not a synonym for HTTP but there are much better resources out there that rant on about the important of hypertext and URL support etc. This is mostly about the perspective of consumers. >It does not matter too much which part of the URL is the one that is causing the URL to be wrong. Your consumer disagrees, they'd like to know if their URL was fat fingered or if a record was missing. My argument is 404 is inappropriate because the web service exists, but the record doesn't. > Thus `api/v11/employees/1` and `/api/v1/employees/100` both are wrong resources. I can't say this is wrong, but it doesn't feel right. `/api/v11` straight up resolves to nowhere. Maybe this an instance where Gone is better than Not Found? > Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. That can be added as a debug message for the developer for example. 100% agreed. It's just a thought I had while debating with my team. I posted this here for this exact kind of feedback :D good points raised
- inanutshellus 4y ago
  This is a bit weird to me. Your article is entitled "I've been abusing HTTP status codes" ... but... you're not "abusing" them, you're "not using" them for your APIs. (Or, said another way, you're leaving them to their normal usage for HTTP servers.) Thus -- as REST is /the/ canonical "hijack HTTP status codes to mean something clever" paradigm -- your article is /entirely/ in context of REST even if you avoided mentioning it. ... Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted".
  - that_james 4y ago
    > Thus -- as REST is /the/ canonical "hijack HTTP status codes to mean something clever" paradigm -- your article is /entirely/ in context of REST. Oof, that's a hell of a good point. So much for that plan lol > Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted". Seems like, for REST, you'd want to return a 400 (malformed request) or something if your URL was borked rather than overloading 404. Yup, that's the headache I'm trying to muddle my way through. Really it's less "this is how to build APIs" and more "have you considered your consumer when you return data?". But I think even in that context your point stands better. Back to the drawing board it seems. At least I can generate more content now :D
guhcampos 4y ago
I'd say this makes the author's point entirely valid, it only shifts the argument in a broader direction: it's not the application abusing HTTP status codes: it's the REST standard itself. Why, in all fairness, we always knew. It's an overloading of an existing technology, a bit hacky as it is, and stuff like GRPC are the answer to this, but still the simplicity of HTTP and REST are the demise of standardization.
- dragonwriter 4y ago
  > it's the REST standard itself. REST is an architectural style, not a standard. > It's an overloading of an existing technology, a bit hacky as it is REST is not tied to a particular technology or protocol, and is not “overloading” whatever protocol(s) it is used with; the architectural style specifies using the subset of the available features of the underlying protocols that corresponds to REST semantics consistent with the specifications of those protocols.
yencabulator 4y ago
> In case of REST the main idea is that if you try to GET a resource by ID then you assume that the resource exists. Thus if it does not exists => 404. This is actually more than a little scary. If you're configuring the API client, and typo the base URL, your next automation run could decide that no resources exist anymore, and delete all kinds of things from your database. I would really recommend designing APIs that communicate more than just the 404, and writing clients that check that extra part, to differentiate between "something is wrong" and "you requested item of type X from api Y, and that does not exist". If the client got a response that doesn't explicitly state X and Y, it should assume misconfiguration and give a more general error. The response body could be used for this distinction, if you want to stick with 404.

by gls2ro 4y ago

treis 4y ago

>If the API that the author is building is a REST API then the response for a non-existing resource is 404. The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying. >Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. But then we've lost standardization which is the whole point of the error codes to begin with.

0xCMP 4y ago
Exactly the point I bring up to not do this. However, a viable (but often difficult or unsupported) way to solve this would be changing the HTTP Message so it's clearer the intent of the error. Most frameworks like Django hardcode the response messages.
Feeble 4y ago
> But then we've lost standardization which is the whole point of the error codes to begin with. Returning 200 and then using response body to denote missing resource is no different. So you have to choose, either 404 can be invalid path and missing employee; or 200 can be valid data or missing employee. Personally I would prefer the former as 200/OK then indicates success.
- treis 4y ago
  This is getting into semantics, but IMHO a 200 OK with an empty body is the most correct response in this scenario. Everything worked so you get 200 OK and the most accurate representation of a resource that you know doesn't exist is an empty body. 404 Not found is not exactly correct. I argue that the server found a representation of the object with that representation being "It does not exist".
  - recursive 4y ago
    For no content, I'd argue 204 is even more correct.
    - treis 4y ago
      But we do have content to send. Namely, that the server knows that the resource the client requested doesn't exist.
      - dragonwriter 4y ago
        That’s not a “representation of the resource”. That’s a fact about the state of the universe, to wit, that it contains no such resource. Which is what is communicated by a 404.
        treis 4y ago
        Again, 404 does not mean that the universe contains no such resource. It means that this specific server did not find the resource.
dec0dedab0de 4y ago
>If the API that the author is building is a REST API then the response for a non-existing resource is 404. The problem is that this error then overlaps with server path routing issues, DNS problems, and general network issues. Even if it's logically correct it makes dealing with your API annoying. As others have mentioned, you can return anything you want in the body of a 404 to clarify. However the other issues you all mentioned should really be in the 500s with errors.
- treis 4y ago
  >you can return anything you want in the body of a 404 to clarify This isn't how the world works though. Much development happens on COTS platforms or internal frameworks or whatever. Many of those will have generic error handling, logging, alerting, etc. based off the response code. Your pedantically correct 404 with a clarifying body ruins my day (really weeks) because now I have to chase whoever built those things to fix my problem. I am annoyed and curse your name.
  - nrdvana 4y ago
    Use a better framework? In any good framework, returning a json body with a status code should be a one-liner. One topic the article didn't even touch was flagging bad parameters in the request which didn't make sense on the application level. HTTP has 422 for that. I commonly write things like validate_my_param($c->req->params->{format}) or $c->detach(HTTP => 422, ['Invalid format']); (the HTTP view renders that string either as text/plain or json as {"message":"Invalid format"} depending on the accept header of the request.) I've used that for years and been quite happy with the downstream results. Edit: I should elaborate on the downstream results. On the javascript side, the ajax methods often contain separate success and failure callbacks. You often need to handle the failure callback to ensure the user knows that something broke. If you also have an error path in the success callback, it clutters the code. ajax({ success(data, ...) { if (data.wasActuallyAnError) { // lines // of code // to handle // the error } else useTheData(data); }, error(response, ...) { // lines // of code // to handle // the error } }) vs. ajax({ success(data, ...) { useTheData(data); }, error(response, ...) { // lines // of code // to handle // the error } }) This problem is not insurmountable of course. you could wrap up your error code in helper functions and reduce the boilerplate. You could write a completely generic error handler for your front-end framework and automatically include it in all ajax calls. You could also write a custom ajax method that interprets the 200 status error messages and diverts to the error callback. But I still think the 404/422 status code is a better pattern to start from, since most of the client-side frameworks I've used switch code paths based on the status code.
jehlakj 4y ago
Maybe I'm not understanding this, but wouldn't network issues take priority naturally? For example, if there's a problem with a database connection, you'd return 500. How would you know if the resource even exists?
- treis 4y ago
  Network issues between the client and the server.
  - NoGravitas 4y ago
    Those will never return a 404, though.
    - treis 4y ago
      If your DNS server has the wrong IP for a URL it will.
quesera 4y ago
> this error then overlaps with server path routing issues, DNS problems, and general network issues. No webserver will ever return a 404 for any of those cases.
imetatroll 4y ago
Historically 404 is returned for static websites. Ergo it is the standard.

unregistereddev 4y ago

I agree - though I also agree with the author that this is an area where REST specifications are a little clunky. Yes, `api/v1/employees/100` should return a 404, because that path represents the location of a specific entity and that entity does not exist. Just as the author thinks it's clunky to return HTTP error codes representing application errors, I think it's clunky to apply application logic to HTTP method semantics. GET, POST, and DELETE were designed as instructions telling a web server how to handle static-ish content, and it shows in their design. Why would a GET have a body? It's a request for the server to return a specific file. This leads to breaking REST standards - for example, search endpoints that logically are GET calls (they return entities), but are implemented as POST methods because the search criteria is complex enough that you wanted a request body. Or bulk delete calls that are similarly implemented as POST methods. REST works best in a "rules are meant to be broken" manner in my opinion. It's not a bad system (which is why it is so common), but mixing application logic with HTTP transport logic does lead to oddities.

bachmitre 4y ago
this
dvlsg 4y ago
Some applications opted for using GET with request bodies as well, for better or worse. Elasticsearch comes to mind. There have been some attempts to extend the list of http verbs to include something that fits those use cases more naturally - SEARCH and QUERY. QUERY got more traction, if I remember correctly, but I haven't heard much about it in a while. https://www.w3.org/2012/ldp/wiki/Proposal_for_HTTP_QUERY_Verb https://www.w3.org/2012/ldp/wiki/Proposal_for_HTTP_QUERY_Ver...
- ognarb 4y ago
  SEARCH is used by the WebDAV standard, not that I like WebDAV search syntax https://gist.github.com/CarlSchwan/66fde29d52022c0ef76aec362c01a846 https://gist.github.com/CarlSchwan/66fde29d52022c0ef76aec362...

smashed 4y ago

I think the author is aware and I noticed the term REST does not appear anywhere in the article. The main idea of the article is exactly that he does not agree that http error codes be used for application level errors (as REST principles recommend). I think at this point that ship has sailed. Most HTTP based API will use http error codes in various ways. I would be surprised to not receive a 404 when requesting a resource ID that does not exist for example when consuming a new API.

TimTheTinker 4y ago
> I think at this point that ship has sailed. If you were talking about TCP/IP, then perhaps - because much of the internet's infrastructure is hard-coded (even burned into silicon) to use current standards. But application semantics aren't frozen in stone by any means. Servers and clients built atop HTTP are still being written. Why not adopt better patterns that apply properly separate semantics for each layer? I liked the REST ideas when they came out, particularly because they provided a much better and simpler alternative to SOAP. But I think improving patterns where we can is always a good idea.
- smashed 4y ago
  Of course. Its just that in the general mindset, an hypothetical average dev who needs to call your API won't be surprised to receive a 404 for a missing resource. I might be wrong of course. Either way, a proper api doc/spec should make either approach a non-issue. Personnally, I've switched to graphql where it makes sense. Application level error codes and handled on the graphql layer, not on the HTTP layer, so you could say I've adopted that approach!

that_james 4y ago

I very intentionally avoided the REST vs HTTP RPC debate. This is _specifically_ about HTTP APIs. REST is not a synonym for HTTP but there are much better resources out there that rant on about the important of hypertext and URL support etc. This is mostly about the perspective of consumers. >It does not matter too much which part of the URL is the one that is causing the URL to be wrong. Your consumer disagrees, they'd like to know if their URL was fat fingered or if a record was missing. My argument is 404 is inappropriate because the web service exists, but the record doesn't. > Thus `api/v11/employees/1` and `/api/v1/employees/100` both are wrong resources. I can't say this is wrong, but it doesn't feel right. `/api/v11` straight up resolves to nowhere. Maybe this an instance where Gone is better than Not Found? > Nobody is stoppping anyone to add response body to a 404 to indicate which nested resource is invalid. That can be added as a debug message for the developer for example. 100% agreed. It's just a thought I had while debating with my team. I posted this here for this exact kind of feedback :D good points raised

inanutshellus 4y ago
This is a bit weird to me. Your article is entitled "I've been abusing HTTP status codes" ... but... you're not "abusing" them, you're "not using" them for your APIs. (Or, said another way, you're leaving them to their normal usage for HTTP servers.) Thus -- as REST is /the/ canonical "hijack HTTP status codes to mean something clever" paradigm -- your article is /entirely/ in context of REST even if you avoided mentioning it. ... Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted".
- that_james 4y ago
  > Thus -- as REST is /the/ canonical "hijack HTTP status codes to mean something clever" paradigm -- your article is /entirely/ in context of REST. Oof, that's a hell of a good point. So much for that plan lol > Anyway - I'm entirely with you on the foolishness of using 404 to mean both "your URL is messed up" and "I couldn't find the resource you wanted". Seems like, for REST, you'd want to return a 400 (malformed request) or something if your URL was borked rather than overloading 404. Yup, that's the headache I'm trying to muddle my way through. Really it's less "this is how to build APIs" and more "have you considered your consumer when you return data?". But I think even in that context your point stands better. Back to the drawing board it seems. At least I can generate more content now :D

guhcampos 4y ago

I'd say this makes the author's point entirely valid, it only shifts the argument in a broader direction: it's not the application abusing HTTP status codes: it's the REST standard itself. Why, in all fairness, we always knew. It's an overloading of an existing technology, a bit hacky as it is, and stuff like GRPC are the answer to this, but still the simplicity of HTTP and REST are the demise of standardization.

dragonwriter 4y ago
> it's the REST standard itself. REST is an architectural style, not a standard. > It's an overloading of an existing technology, a bit hacky as it is REST is not tied to a particular technology or protocol, and is not “overloading” whatever protocol(s) it is used with; the architectural style specifies using the subset of the available features of the underlying protocols that corresponds to REST semantics consistent with the specifications of those protocols.

yencabulator 4y ago

> In case of REST the main idea is that if you try to GET a resource by ID then you assume that the resource exists. Thus if it does not exists => 404. This is actually more than a little scary. If you're configuring the API client, and typo the base URL, your next automation run could decide that no resources exist anymore, and delete all kinds of things from your database. I would really recommend designing APIs that communicate more than just the 404, and writing clients that check that extra part, to differentiate between "something is wrong" and "you requested item of type X from api Y, and that does not exist". If the client got a response that doesn't explicitly state X and Y, it should assume misconfiguration and give a more general error. The response body could be used for this distinction, if you want to stick with 404.