8 ms·

> Also, if you are literally having HTTP 400s internally, why aren't you using some kind of actual RPC mechanism? Do you like pain? I just had a discussion abo

> Also, if you are literally having HTTP 400s internally, why aren't you using some kind of actual RPC mechanism? Do you like pain? I just had a discussion about this yesterday where we have an internal JSON API that auths a credit card, and if the card is declined it returns a status and a message. Another developer wanted it to return a 4xx error, but that made me uneasy. I think you could make a good argument either way, but to me that isn't a failure you'd present at the HTTP layer. 4xx is better than 5xx, but I was still worried how intermediate devices would interfere. (E.g. an AWS ELB will take your node out of service if it gives too many 5xxs, and IIS can do some crazy things if your app returns a 401.) Also I don't want declined cards to show up in system-level monitoring. But what do other folks think? I believe smart people can make a case either way. EDIT: Btw based on these Stack Overflow answers I'm in the minority: https://stackoverflow.com/questions/9381520/what-is-the-appropriate-http-status-code-response-for-a-general-unsuccessful-req https://stackoverflow.com/questions/9381520/what-is-the-appr...

erikpukinskis 7y ago
I always thought, if there’s a code that officially signals what we’re trying to signal, we should use it, and rely on it as “the signal”. But at the end of the day, your app must target a subset of available APIs. If you use every available API the surface will be too big and you will have snowballing complexity. So if you are already using another API which can signal what you’re trying to signal then feel free to institute a “everything we can expect returns 200” policy. That kind of decision is critical for putting an upper bound on your system complexity. I want to institute a “no default exports” policy in the JavaScript app I’m working on. It’s not that there aren’t places a default export makes sense—there are. It’s that prohibiting them will decrease the number of things our coders have to think about.
- s_kilk 7y ago
  > I always thought, if there’s a code that officially signals what we’re trying to signal, we should use it, and rely on it as “the signal”. This is a line I hear a lot in regards to this general situation, and the interesting part is that the 400 code really does not signal the thing the developer is trying to signal, not at all. But a lot of devs have a weird hang-up about this for some reason, and will go to any lengths to not send a 200 response. I blame the REST cultists.
  - kbenson 7y ago
    Well, 4XX are for client errors, so if the client fails authing to the API, it would be appropriate there. If the API fails authenticating to an API further down the chain, that shouldn't be returned as a 4XX status code, because it could be confused with the immediate API's auth. HTTP status codes can be used to good effect if done thoughtfully, because there's often already a lot of logic in the useragent to handle some cases (such as redirects, etc). Not using anything except for 200 is limiting. Using a bunch of status codes to refer to things that make no sense is confusing. Defining a sane standard for how your application will choose between them that allows for the benefits they offer without confusing clients as to what is actually being communicated is the best of both though.
msluyter 7y ago
Interesting question. My understanding is that a 400 response indicates that there's something malformed about the request itself such that the server can't/won't process it. Given that in order to decline the card the service has to actually process the request, I'd agree that a 400 is inappropriate.
- Shaanie 7y ago
  The previous definition was 400 means the server can't understand the request. Current one is >The 400 (Bad Request) status code indicates that the server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing). Of course, a lot of things could be said to be a client error.
parliament32 7y ago
I think you're right. 4XX means "client made an error" at the application layer. Something like a malformed request (say, bad JSON passed in): absolutely, 4XX is the way to go. However, if the client sent a good message who's contents happen to be rejected, that's a 200 (and whatever error goes in the response body). The whole "did this card work" question happens in a layer above the application layer, and HTTP status codes are only meant for the application layer itself.
- sagichmal 7y ago
  > HTTP status codes are only meant for the application layer itself. You seem to be using a strange definition of "application", but assuming you mean "transport" — I mean, this is obviously not true, right? If you sent a perfectly semantically valid HTTP request for a resource that doesn't exist on the remote, that's a 404. But by your logic, you're saying it should be a 200, because there wasn't a failure in (essentially) nginx or the HTTP library of your application server? HTTP is a transport designed for a specific type of application (document-oriented resources) and its status codes (and lots of other things) reflect that initial coupling, so it's always gonna be a pain when you swap out the application (to a credit card processing service or whatever) and have to think about how the status code mapping will work. And you can certainly "cheat" and say that every request that doesn't crash the server is 200 OK, I guess, but that's pretty clearly not what the Hyper Text Transfer Protocol wants you to do. Its set of status codes clearly reflect details of the underlying application.
  - parliament32 7y ago
    Transport is TCP/UDP/etc. HTTP is part of the application layer, see https://en.wikipedia.org/wiki/OSI_model https://en.wikipedia.org/wiki/OSI_model and https://en.wikipedia.org/wiki/Application_layer https://en.wikipedia.org/wiki/Application_layer. >If you sent a perfectly semantically valid HTTP request for a resource that doesn't exist on the remote Requesting a path (via the application layer) that doesn't exist is a client error, hence the 4XX. But, say, if your website has a search bar and a client searches for "blahblah-thisdoesntexist" and gets no results, the webpage should return a 200 (the request was valid: the client sent form data to an existing resource and it was processed successfully) with something like "no results found" in the body -- because the search function itself is above the application layer. This is what I mean about "is the CC is authorized or not" being above the application layer. >Its set of status codes clearly reflect details of the underlying application. This is where I think you're wrong: HTTP status codes reflect the result of the HTTP request itself, not the next application in the chain, if one exists. This can also be expressed if we look lower down the chain. If my HTTP request gave me a 404, should an error be shown in the TCP stack? Well no, because TCP doesn't know or care about HTTP, as far as it's concerned the TCP connection is established and happy. Likewise, in this example, the API HTTP stack doesn't care about whether the data was a CC or something else or what the result was: as far as the HTTP stack is concerned, some JSON got passed in and parsed properly, and some output got sent back, and everything went well so "200".
citrusx 7y ago
In my opinion, a rejection is an expected outcome, and therefore should have a response code of 200. You're not asking, "Does this card exist?", and sending a 404 if you have no record. You're asking a remote system to do a job for you, and if that job completes successfully, it's a "Success" in the HTTP world. At that point, you rely on the body content to tell you what the service correctly determined for you. A result that the user doesn't like is way different than a result that comes about because something was done wrong at the client side (4xx) or a failure on the server side (5xx).
- meowface 7y ago
  I think this is the exact right way of looking at it. 4xx/5xx are for layers that are below the business logic (like "is the web server able to route this path", not "did the application successfully validate the user's input data and allow them to continue on to the next step in a process"). It can get a little fuzzy for 401 and 403 errors, though; those seem to be returned by APIs pretty often. Not entirely sure how I feel about that, but I think those are a bit more sensible.
  - jacobgreenleaf 7y ago
    The HTTP IETF RFC (2616) states: > The first digit of the Status-Code defines the class of response. The last two digits do not have any categorization role. There are 5 values for the first digit: > - 4xx: Client Error - The request contains bad syntax or cannot be fulfilled What do you think status codes like 404, 405, 406 are for? You say they shouldn't be for "did the application successfully validate the user's input data" but status code 400 is explicitly for bad requests. In your view should a HTTP server ever return 4xx?
    - meowface 7y ago
      It depends how you define validate. If the HTTP headers are malformed, sure. If a JSON or XML API receives malformed JSON or XML, sure. But if it's "register this account" and the username already exists, or "pay with this credit card" but it's not a valid credit card number, or "process this invoice" but the total is missing, then I think it should be a 200 status with an error message. It's validating the request was received and loaded by the application vs. validating the business logic requirements for the request's data payload. Obviously if you send a TAIL method request, you should get a 405, and if you send Accept: eggs/*, you should get a 406. If a route doesn't exist, you should get a 404. If you fail HTTP basic auth, you should get a 403 (but why are you using HTTP basic auth?). If you want certain paths to never be accessed for some reason, you should return a 401.
russelldavis 7y ago
Stripe returns HTTP 402 ("Payment Required") when a card is declined.
paulddraper 7y ago
I think the key is > A "HTTP 400 bad request" is only the sort of thing you get to ignore when it's coming from the great unwashed masses who are lobbing random crazy crap at you, trying to break you. Internally, that's something entirely different. If this is an internal-only API then most people look at 400s as their bug. Sounds like you are in the same boat. If it's an external API, it's not their bug.
munchbunny 7y ago
I would agree with not using the http status codes to return a card processing error. Specifically, you want to be able to easily distinguish between "your url is wrong" or "your authentication credentials are wrong" or "the API endpoint threw an exception" type errors and "credit card processing failed" errors. It's easier long run to put business logic errors somewhere separate from protocol/routing layer (i.e. even a http header would be better) so that you can tell what is Rails/Flask/whatever failing vs. logic failing. This also gives you more flexibility to do stuff at the hardware layer (another commenter mentioned ELB) without interfering with the application layer.
erpellan 7y ago
4xx errors are no better or worse than 5xx. They just mean different things. Generally, 4xx errors means 'Client screwed up'. There's probably no point sending this message again, it isn't going to work until something is fixed. That might mean it will work if, for example, the account is funded or card unlocked. But something needs to be done on behalf of the client. 5xx errors mean 'Server screwed up'. It probably _is_ worth having another attempt at sending the request. Maybe it was a temporary glitch, or maybe a new release of the server has to happen. Regardless, there was probably nothing inherently wrong with the actual request.
z3t4 7y ago
I think the pain comes from not controlling the full stack. But managing the full stack is also painful. Having several abstraction layers will affect performance. Personally I agree that's its not a good idea to reuse (HTTP) error codes in your app layer ala REST
hathawsh 7y ago
For the HTTP API my company has been building, we're returning 400 status codes for most errors. It is intended as a strong suggestion that client code needs to handle errors rather than ignore them. However, we're aware that some client libraries are limited in their ability to handle status codes other than 2xx, so we allow an optional request parameter that asks the server to return 200 instead of 400 for most errors. As you said, smart people disagree about this, so we let them choose. I think we've hit a good balance.
- DreamSpinner 7y ago
  That sounds like an excellent approach. If a consumer chooses "always 200 on error", how do you return the API response status? HTTP Response Header or in a custom value in the response body?
  - hathawsh 7y ago
    Thanks. Error responses from our API always generate a JSON object with top level error and error_description attributes, while non-errors never contain a top level error attribute. It might be interesting to add a response header to disambiguate even more.
xorcist 7y ago
Using HTTP for RPC lends itself to this sort of endless discussion. Authorization is even worse. You could argue either way. What's important is being consistent, at the very least throughout an API but preferably throughout the whole organization. (Personally I'd probably lean towards a 40x of some kind, just make sure it doesn't clash with something that you care about.) Along the same lines, and arguably more important, is how to log the operations where a transaction completed successfully but with a negative answer. If you log expected negatives as errors you can get error blindness.
brown9-2 7y ago
It sounds like a lot of the problems you describe come from using software that assumes normal “HTTP web server” applications when you really have a custom web application.
geezerjay 7y ago
IMHO returning status 400 is only reasonable if the request itself is malformed. A request to check whether a CC is valid does not become malformed depending on whether the CC is valid or not.
neop1x 7y ago
I came to a good analogy - Imagine sending that card number using HTTP form to a HTML target URL. You wouldn't return 4xx code if a card is not found. The user would see your 404 page or browser 404 page and think the page doesn't exist! API shouldn't be different.

erikpukinskis 7y ago

I always thought, if there’s a code that officially signals what we’re trying to signal, we should use it, and rely on it as “the signal”. But at the end of the day, your app must target a subset of available APIs. If you use every available API the surface will be too big and you will have snowballing complexity. So if you are already using another API which can signal what you’re trying to signal then feel free to institute a “everything we can expect returns 200” policy. That kind of decision is critical for putting an upper bound on your system complexity. I want to institute a “no default exports” policy in the JavaScript app I’m working on. It’s not that there aren’t places a default export makes sense—there are. It’s that prohibiting them will decrease the number of things our coders have to think about.

s_kilk 7y ago
> I always thought, if there’s a code that officially signals what we’re trying to signal, we should use it, and rely on it as “the signal”. This is a line I hear a lot in regards to this general situation, and the interesting part is that the 400 code really does not signal the thing the developer is trying to signal, not at all. But a lot of devs have a weird hang-up about this for some reason, and will go to any lengths to not send a 200 response. I blame the REST cultists.
- kbenson 7y ago
  Well, 4XX are for client errors, so if the client fails authing to the API, it would be appropriate there. If the API fails authenticating to an API further down the chain, that shouldn't be returned as a 4XX status code, because it could be confused with the immediate API's auth. HTTP status codes can be used to good effect if done thoughtfully, because there's often already a lot of logic in the useragent to handle some cases (such as redirects, etc). Not using anything except for 200 is limiting. Using a bunch of status codes to refer to things that make no sense is confusing. Defining a sane standard for how your application will choose between them that allows for the benefits they offer without confusing clients as to what is actually being communicated is the best of both though.

msluyter 7y ago

Interesting question. My understanding is that a 400 response indicates that there's something malformed about the request itself such that the server can't/won't process it. Given that in order to decline the card the service has to actually process the request, I'd agree that a 400 is inappropriate.

Shaanie 7y ago
The previous definition was 400 means the server can't understand the request. Current one is >The 400 (Bad Request) status code indicates that the server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing). Of course, a lot of things could be said to be a client error.

parliament32 7y ago

I think you're right. 4XX means "client made an error" at the application layer. Something like a malformed request (say, bad JSON passed in): absolutely, 4XX is the way to go. However, if the client sent a good message who's contents happen to be rejected, that's a 200 (and whatever error goes in the response body). The whole "did this card work" question happens in a layer above the application layer, and HTTP status codes are only meant for the application layer itself.

sagichmal 7y ago
> HTTP status codes are only meant for the application layer itself. You seem to be using a strange definition of "application", but assuming you mean "transport" — I mean, this is obviously not true, right? If you sent a perfectly semantically valid HTTP request for a resource that doesn't exist on the remote, that's a 404. But by your logic, you're saying it should be a 200, because there wasn't a failure in (essentially) nginx or the HTTP library of your application server? HTTP is a transport designed for a specific type of application (document-oriented resources) and its status codes (and lots of other things) reflect that initial coupling, so it's always gonna be a pain when you swap out the application (to a credit card processing service or whatever) and have to think about how the status code mapping will work. And you can certainly "cheat" and say that every request that doesn't crash the server is 200 OK, I guess, but that's pretty clearly not what the Hyper Text Transfer Protocol wants you to do. Its set of status codes clearly reflect details of the underlying application.
- parliament32 7y ago
  Transport is TCP/UDP/etc. HTTP is part of the application layer, see https://en.wikipedia.org/wiki/OSI_model https://en.wikipedia.org/wiki/OSI_model and https://en.wikipedia.org/wiki/Application_layer https://en.wikipedia.org/wiki/Application_layer. >If you sent a perfectly semantically valid HTTP request for a resource that doesn't exist on the remote Requesting a path (via the application layer) that doesn't exist is a client error, hence the 4XX. But, say, if your website has a search bar and a client searches for "blahblah-thisdoesntexist" and gets no results, the webpage should return a 200 (the request was valid: the client sent form data to an existing resource and it was processed successfully) with something like "no results found" in the body -- because the search function itself is above the application layer. This is what I mean about "is the CC is authorized or not" being above the application layer. >Its set of status codes clearly reflect details of the underlying application. This is where I think you're wrong: HTTP status codes reflect the result of the HTTP request itself, not the next application in the chain, if one exists. This can also be expressed if we look lower down the chain. If my HTTP request gave me a 404, should an error be shown in the TCP stack? Well no, because TCP doesn't know or care about HTTP, as far as it's concerned the TCP connection is established and happy. Likewise, in this example, the API HTTP stack doesn't care about whether the data was a CC or something else or what the result was: as far as the HTTP stack is concerned, some JSON got passed in and parsed properly, and some output got sent back, and everything went well so "200".

citrusx 7y ago

In my opinion, a rejection is an expected outcome, and therefore should have a response code of 200. You're not asking, "Does this card exist?", and sending a 404 if you have no record. You're asking a remote system to do a job for you, and if that job completes successfully, it's a "Success" in the HTTP world. At that point, you rely on the body content to tell you what the service correctly determined for you. A result that the user doesn't like is way different than a result that comes about because something was done wrong at the client side (4xx) or a failure on the server side (5xx).

meowface 7y ago
I think this is the exact right way of looking at it. 4xx/5xx are for layers that are below the business logic (like "is the web server able to route this path", not "did the application successfully validate the user's input data and allow them to continue on to the next step in a process"). It can get a little fuzzy for 401 and 403 errors, though; those seem to be returned by APIs pretty often. Not entirely sure how I feel about that, but I think those are a bit more sensible.
- jacobgreenleaf 7y ago
  The HTTP IETF RFC (2616) states: > The first digit of the Status-Code defines the class of response. The last two digits do not have any categorization role. There are 5 values for the first digit: > - 4xx: Client Error - The request contains bad syntax or cannot be fulfilled What do you think status codes like 404, 405, 406 are for? You say they shouldn't be for "did the application successfully validate the user's input data" but status code 400 is explicitly for bad requests. In your view should a HTTP server ever return 4xx?
  - meowface 7y ago
    It depends how you define validate. If the HTTP headers are malformed, sure. If a JSON or XML API receives malformed JSON or XML, sure. But if it's "register this account" and the username already exists, or "pay with this credit card" but it's not a valid credit card number, or "process this invoice" but the total is missing, then I think it should be a 200 status with an error message. It's validating the request was received and loaded by the application vs. validating the business logic requirements for the request's data payload. Obviously if you send a TAIL method request, you should get a 405, and if you send Accept: eggs/*, you should get a 406. If a route doesn't exist, you should get a 404. If you fail HTTP basic auth, you should get a 403 (but why are you using HTTP basic auth?). If you want certain paths to never be accessed for some reason, you should return a 401.

russelldavis 7y ago

Stripe returns HTTP 402 ("Payment Required") when a card is declined.

paulddraper 7y ago

I think the key is > A "HTTP 400 bad request" is only the sort of thing you get to ignore when it's coming from the great unwashed masses who are lobbing random crazy crap at you, trying to break you. Internally, that's something entirely different. If this is an internal-only API then most people look at 400s as their bug. Sounds like you are in the same boat. If it's an external API, it's not their bug.

munchbunny 7y ago

I would agree with not using the http status codes to return a card processing error. Specifically, you want to be able to easily distinguish between "your url is wrong" or "your authentication credentials are wrong" or "the API endpoint threw an exception" type errors and "credit card processing failed" errors. It's easier long run to put business logic errors somewhere separate from protocol/routing layer (i.e. even a http header would be better) so that you can tell what is Rails/Flask/whatever failing vs. logic failing. This also gives you more flexibility to do stuff at the hardware layer (another commenter mentioned ELB) without interfering with the application layer.

erpellan 7y ago

4xx errors are no better or worse than 5xx. They just mean different things. Generally, 4xx errors means 'Client screwed up'. There's probably no point sending this message again, it isn't going to work until something is fixed. That might mean it will work if, for example, the account is funded or card unlocked. But something needs to be done on behalf of the client. 5xx errors mean 'Server screwed up'. It probably _is_ worth having another attempt at sending the request. Maybe it was a temporary glitch, or maybe a new release of the server has to happen. Regardless, there was probably nothing inherently wrong with the actual request.

z3t4 7y ago

I think the pain comes from not controlling the full stack. But managing the full stack is also painful. Having several abstraction layers will affect performance. Personally I agree that's its not a good idea to reuse (HTTP) error codes in your app layer ala REST

hathawsh 7y ago

For the HTTP API my company has been building, we're returning 400 status codes for most errors. It is intended as a strong suggestion that client code needs to handle errors rather than ignore them. However, we're aware that some client libraries are limited in their ability to handle status codes other than 2xx, so we allow an optional request parameter that asks the server to return 200 instead of 400 for most errors. As you said, smart people disagree about this, so we let them choose. I think we've hit a good balance.

DreamSpinner 7y ago
That sounds like an excellent approach. If a consumer chooses "always 200 on error", how do you return the API response status? HTTP Response Header or in a custom value in the response body?
- hathawsh 7y ago
  Thanks. Error responses from our API always generate a JSON object with top level error and error_description attributes, while non-errors never contain a top level error attribute. It might be interesting to add a response header to disambiguate even more.

xorcist 7y ago

Using HTTP for RPC lends itself to this sort of endless discussion. Authorization is even worse. You could argue either way. What's important is being consistent, at the very least throughout an API but preferably throughout the whole organization. (Personally I'd probably lean towards a 40x of some kind, just make sure it doesn't clash with something that you care about.) Along the same lines, and arguably more important, is how to log the operations where a transaction completed successfully but with a negative answer. If you log expected negatives as errors you can get error blindness.

brown9-2 7y ago

It sounds like a lot of the problems you describe come from using software that assumes normal “HTTP web server” applications when you really have a custom web application.

geezerjay 7y ago

IMHO returning status 400 is only reasonable if the request itself is malformed. A request to check whether a CC is valid does not become malformed depending on whether the CC is valid or not.

neop1x 7y ago

I came to a good analogy - Imagine sending that card number using HTTP form to a HTML target URL. You wouldn't return 4xx code if a card is not found. The user would see your 404 page or browser 404 page and think the page doesn't exist! API shouldn't be different.