6 ms·

This isn't remotely a bug, it's not even really anything to do with browsers. The browser will be doing (roughly) the following: while pending_requests():

This isn't remotely a bug, it's not even really anything to do with browsers. The browser will be doing (roughly) the following: while pending_requests(): send_request() read_response() But what send_request and read_response are doing is putting data on the OS's outbound queue and then attempting to get data from the inbound queue. If the data is already in the inbound queue before the request is put on the outbound queue it doesn't matter - the browser is not aware of this fact. So long as the "responses" don't come in faster than the browser is sending requests and causing the queue to overfill, and so long as the responses send out come in the order the browser is sending requests this technique will work. In general this is just an "optimistic strategy".

jacquesm 12y ago
I sure would have been happy to have you on our team in '95, it wasn't obvious to me at all. Afterwards of course it was obvious. But I'm always a little bit suspicious of myself when I think after I am presented with how something works that it is obvious.
- ryanjshaw 12y ago
  Glad to hear you say that. The part of RFC 1945 you quoted seems pretty "clear" to me that the server can't send a response prior to receiving the request. It's pretty "obvious" to me that if the client validates responses and can't find an associated request you're going to have problems, and relying on undocumented behaviour is a bad idea. In my mind the fact it works seems like dumb luck and I would never have thought to try it -- all of which is pretty depressing seeing as how evidently it made you lots of money, which is certainly something I could do with :) In hindsight, it makes perfect sense to exploit a simple solution with low implementation costs even if it has an unknown lifespan or potential risks - if it breaks, you're no worse off than you were before (well maybe not if your clients have come to depend on it and you don't have a backup), if it doesn't, great!
  - jacquesm 12y ago
    I can't tell you how many nights of sleep I lost whenever a new browser release by one of the larger browser manufacturers was announced. Every time I was sure our house of cards would come tumbling down but it never did!
- Sharlin 12y ago
  Yes, I think this is a good example of the fact that "obvious" and "obvious in retrospect" are two very different things. In other words, hindsight is 20:20.
- MichaelGG 12y ago
  How would it even validate the request was sent first? It'd have to keep looking at the receive buffers and make sure they're always zero until it knows it actually sent the last packet. Right? It sounds like it'd be hard to implement and for what benefit?
  - jacquesm 12y ago
    That was the million dollar question and I gambled that I understood enough of the implementation details that it would be impossible to close the hole. That didn't stop me from living in fear of just that :)
- stevejones 12y ago
  Well coming up with the strategy (and actually implementing it, edge cases abound) is the hard part. However the mechanism by which it works isn't a bug.
pbhjpbhj 12y ago
So once the initial request is made you can push anything you like to the browser using this pipelining method? What is stopping the responses from coming in too quickly and "overfilling" the queue? To make it work doesn't seem too hard but aren't there possibilities for exploits if you're loading in to memory unrequested data?
- lallysingh 12y ago
  TCP has flow control.
  - jacquesm 12y ago
    Yes, but if the implementation is indeed as naive as sketched above the buffer will be emptied regardless rendering any flow-control measures moot.
    - colanderman 12y ago
      I'm not sure which implementation you're referring to; in stevejones's example, incoming data remains buffered by the OS until the client specifically requests one response's worth in read_response(). If the OS's buffer ever gets full, it will signal the server's OS to stop sending data; if it continues to get packets it will simply drop them (thus minimizing resource usage). It's of course totally possible for one to make a client that reads responses regardless of whether it send a request, but that's rather silly as giving up flow control like that immediately opens your application up to a DoS attack. (Of course just because it's silly doesn't mean no-one does it!)
      - jacquesm 12y ago
        Ah, I see, you're right, this assumes 'blocking' code. I wonder if that's really how browers work or if they employ an array of open connections that are periodically polled for responses to outstanding requests. Obviously, responding to something that wasn't requested is a bad idea. Flow control only works for large responses (which is good, because that is at least one resource that you can protect), makes you wonder what you could do with multiple answers small enough to fit in the same window and if that would allow you to identify HTTP implementations that have taken 'asynchronous' one step too far.

jacquesm 12y ago

I sure would have been happy to have you on our team in '95, it wasn't obvious to me at all. Afterwards of course it was obvious. But I'm always a little bit suspicious of myself when I think after I am presented with how something works that it is obvious.

ryanjshaw 12y ago
Glad to hear you say that. The part of RFC 1945 you quoted seems pretty "clear" to me that the server can't send a response prior to receiving the request. It's pretty "obvious" to me that if the client validates responses and can't find an associated request you're going to have problems, and relying on undocumented behaviour is a bad idea. In my mind the fact it works seems like dumb luck and I would never have thought to try it -- all of which is pretty depressing seeing as how evidently it made you lots of money, which is certainly something I could do with :) In hindsight, it makes perfect sense to exploit a simple solution with low implementation costs even if it has an unknown lifespan or potential risks - if it breaks, you're no worse off than you were before (well maybe not if your clients have come to depend on it and you don't have a backup), if it doesn't, great!
- jacquesm 12y ago
  I can't tell you how many nights of sleep I lost whenever a new browser release by one of the larger browser manufacturers was announced. Every time I was sure our house of cards would come tumbling down but it never did!
Sharlin 12y ago
Yes, I think this is a good example of the fact that "obvious" and "obvious in retrospect" are two very different things. In other words, hindsight is 20:20.
MichaelGG 12y ago
How would it even validate the request was sent first? It'd have to keep looking at the receive buffers and make sure they're always zero until it knows it actually sent the last packet. Right? It sounds like it'd be hard to implement and for what benefit?
- jacquesm 12y ago
  That was the million dollar question and I gambled that I understood enough of the implementation details that it would be impossible to close the hole. That didn't stop me from living in fear of just that :)
stevejones 12y ago
Well coming up with the strategy (and actually implementing it, edge cases abound) is the hard part. However the mechanism by which it works isn't a bug.

pbhjpbhj 12y ago

So once the initial request is made you can push anything you like to the browser using this pipelining method? What is stopping the responses from coming in too quickly and "overfilling" the queue? To make it work doesn't seem too hard but aren't there possibilities for exploits if you're loading in to memory unrequested data?

lallysingh 12y ago
TCP has flow control.
- jacquesm 12y ago
  Yes, but if the implementation is indeed as naive as sketched above the buffer will be emptied regardless rendering any flow-control measures moot.
  - colanderman 12y ago
    I'm not sure which implementation you're referring to; in stevejones's example, incoming data remains buffered by the OS until the client specifically requests one response's worth in read_response(). If the OS's buffer ever gets full, it will signal the server's OS to stop sending data; if it continues to get packets it will simply drop them (thus minimizing resource usage). It's of course totally possible for one to make a client that reads responses regardless of whether it send a request, but that's rather silly as giving up flow control like that immediately opens your application up to a DoS attack. (Of course just because it's silly doesn't mean no-one does it!)
    - jacquesm 12y ago
      Ah, I see, you're right, this assumes 'blocking' code. I wonder if that's really how browers work or if they employ an array of open connections that are periodically polled for responses to outstanding requests. Obviously, responding to something that wasn't requested is a bad idea. Flow control only works for large responses (which is good, because that is at least one resource that you can protect), makes you wonder what you could do with multiple answers small enough to fit in the same window and if that would allow you to identify HTTP implementations that have taken 'asynchronous' one step too far.