4 ms·

shouldn't be GET request idempotent?

CodesInChaos 6y ago
GET requests should not have any significant side-effects, unfortunately some do not follow that rule. I once read a story of a whole database getting wiped, because a spider "clicked" on every "delete" link. Idempotence is not enough, it only requires at executing a request a second time (with no other changes in between) has no significant effect, the first time is allowed to have a significant effect. PUT requests are the typical example.
35fbe7d3d5b9 6y ago
Hopping up on my hobby-horse :) No. An idempotent operation is an operation that can be applied any number of times without a change after the initial invocation. This means the first GET would fire and change state, but subsequent GETs would not. GET should be nullipotent: it is an operation which can be applied any number of times without any change.
- tialaramex 6y ago
  Also, this went from being an obscure fact about HTTP that might occasionally bite you to being extremely relevant to production systems with TLS 1.3 (thus some HTTPS and all HTTP/2 and HTTP/3 systems) TLS 1.3's zero round trip resumption by its nature can't be bullet proof against repetition. In toy systems you can defend against this (and you might want to) but in big distributed systems it's going to be possible for an attack to play some HTTP request to your system (perhaps different nodes of it) more than once without you realising. This is obviously unsafe if that HTTP request moves $1000 from your bank account to somebody else's bank account (now they have $1000 extra), and on the other hand obviously safe if the HTTP request is just for the Favicon of Hacker News (now they have another identical copy of the encrypted data) but there's also lots of grey area in between. Translating this human understanding into a rule for machines to follow is the exercise where security holes can occur. The safe option is to just never allow 0RTT resumption (as a client or as a server, either can safely refuse to have any part in this) but in pursuit of performance you can expect other choices to be made.
  - londons_explore 6y ago
    > TLS 1.3's zero round trip resumption by its nature can't be bullet proof against repetition The server side can easily keep track of which nonces have been used, and not allow any re-used nonces to have any side effects. This can be implemented in something like spanner by keeping a database of valid nonces, and tying the marking a nonce as used to the other effects of the HTTP request in the same transaction. Spanners global-consistency will ensure only one transaction will commit. In most cases, the data can be sent back to the user without waiting for commit, preventing any cross-ocean latency penalty too. It's a shame that I don't see a single HTTP/3 server which has any support for a scheme like this.
    - toast0 6y ago
      That's possible, but not really easy. If you don't wait for the commit, you may process the request before getting the notification that it was a duplicate nonce; if you do wait for the commit, there goes your performance boost. The right thing to do in my mind is to only send requests in 0-rtt if they're specifically flagged as eligible.
      - londons_explore 6y ago
        > If you don't wait for the commit, you may process the request before getting the notification that it was a duplicate nonce. Yes, do this. Processing the request isn't bad as long as it doesn't commit. Most evil attackers won't come back to you complaining that your server gave back an HTTP200 response when it should have returned an HTTP400 because they'd reused a nonce.
        tialaramex 6y ago
        The evil attackers don't necessarily care whether in some sense your system "doesn't commit" one of the duplicates. Especially if they can manipulate it to choose whether the "not committed" one is the one they created or one emitted by a real user. I know it was intended flippantly but the attackers do not get the HTTP response code - they're obliged to conduct this attack blind, if they can decrypt the messages then your problem isn't a replay attack but some previously unimagined TLS flaw. So, suppose your system sees three versions of a transaction to transfer $1M to Dr Evil. You're not worried because two of them "won't commit" but do your end users understand that? (hint: they do not). Let's call these versions A, B and C. The real user's web browser believes transaction A succeeded, and is unaware of B and C. Dr Evil complains that the money never arrived, and our user checks, alas they see that eventually transaction A failed (it was judged a duplicate of B, and so was C) and this explains the problem. They initiate a new transfer, let's call that Z. This succeeds and Dr Evil is satisfied. Unfortunately now Dr Evil has $2M instead of $1M.

GET requests should not have any significant side-effects, unfortunately some do not follow that rule. I once read a story of a whole database getting wiped, because a spider "clicked" on every "delete" link. Idempotence is not enough, it only requires at executing a request a second time (with no other changes in between) has no significant effect, the first time is allowed to have a significant effect. PUT requests are the typical example.

35fbe7d3d5b9 6y ago

Hopping up on my hobby-horse :) No. An idempotent operation is an operation that can be applied any number of times without a change after the initial invocation. This means the first GET would fire and change state, but subsequent GETs would not. GET should be nullipotent: it is an operation which can be applied any number of times without any change.

tialaramex 6y ago
Also, this went from being an obscure fact about HTTP that might occasionally bite you to being extremely relevant to production systems with TLS 1.3 (thus some HTTPS and all HTTP/2 and HTTP/3 systems) TLS 1.3's zero round trip resumption by its nature can't be bullet proof against repetition. In toy systems you can defend against this (and you might want to) but in big distributed systems it's going to be possible for an attack to play some HTTP request to your system (perhaps different nodes of it) more than once without you realising. This is obviously unsafe if that HTTP request moves $1000 from your bank account to somebody else's bank account (now they have $1000 extra), and on the other hand obviously safe if the HTTP request is just for the Favicon of Hacker News (now they have another identical copy of the encrypted data) but there's also lots of grey area in between. Translating this human understanding into a rule for machines to follow is the exercise where security holes can occur. The safe option is to just never allow 0RTT resumption (as a client or as a server, either can safely refuse to have any part in this) but in pursuit of performance you can expect other choices to be made.
- londons_explore 6y ago
  > TLS 1.3's zero round trip resumption by its nature can't be bullet proof against repetition The server side can easily keep track of which nonces have been used, and not allow any re-used nonces to have any side effects. This can be implemented in something like spanner by keeping a database of valid nonces, and tying the marking a nonce as used to the other effects of the HTTP request in the same transaction. Spanners global-consistency will ensure only one transaction will commit. In most cases, the data can be sent back to the user without waiting for commit, preventing any cross-ocean latency penalty too. It's a shame that I don't see a single HTTP/3 server which has any support for a scheme like this.
  - toast0 6y ago
    That's possible, but not really easy. If you don't wait for the commit, you may process the request before getting the notification that it was a duplicate nonce; if you do wait for the commit, there goes your performance boost. The right thing to do in my mind is to only send requests in 0-rtt if they're specifically flagged as eligible.
    - londons_explore 6y ago
      > If you don't wait for the commit, you may process the request before getting the notification that it was a duplicate nonce. Yes, do this. Processing the request isn't bad as long as it doesn't commit. Most evil attackers won't come back to you complaining that your server gave back an HTTP200 response when it should have returned an HTTP400 because they'd reused a nonce.
      - tialaramex 6y ago
        The evil attackers don't necessarily care whether in some sense your system "doesn't commit" one of the duplicates. Especially if they can manipulate it to choose whether the "not committed" one is the one they created or one emitted by a real user. I know it was intended flippantly but the attackers do not get the HTTP response code - they're obliged to conduct this attack blind, if they can decrypt the messages then your problem isn't a replay attack but some previously unimagined TLS flaw. So, suppose your system sees three versions of a transaction to transfer $1M to Dr Evil. You're not worried because two of them "won't commit" but do your end users understand that? (hint: they do not). Let's call these versions A, B and C. The real user's web browser believes transaction A succeeded, and is unaware of B and C. Dr Evil complains that the money never arrived, and our user checks, alas they see that eventually transaction A failed (it was judged a duplicate of B, and so was C) and this explains the problem. They initiate a new transfer, let's call that Z. This succeeds and Dr Evil is satisfied. Unfortunately now Dr Evil has $2M instead of $1M.