3 ms·
I was intrigued about the part about not being able to ready the response body more than once. Can anyone more experienced explain why this decision was made in
by pm90 11y ago
I was intrigued about the part about not being able to ready the response body more than once. Can anyone more experienced explain why this decision was made in the core libraries? And if there is a good reason to not read the body more than once, is the author making a mistake by trying to get around it?
- jerf 11y agoGo's core library specifies several interfaces for reading, writing, and closing streams. Here's Reader, for instance: https://golang.org/pkg/io/#Reader https://golang.org/pkg/io/#Reader Those interfaces do not promise seeking, so they apply to as many things as possible. (See https://golang.org/pkg/io/#Seeker https://golang.org/pkg/io/#Seeker .) The HTTP body is provided as an "io.Reader" which actually backs to the network stream, and since TCP streams can't be seeked, it's an error to think you can. This means that native Go HTTP handlers can handle a multi-gigabyte upload or download without consuming all that RAM at once (assuming there is some way to stream), but you lose seeking. You can easily recover read-only access by using io.ReadFull to fully consume a reader into a byte buffer, but then you pay the price of consuming all the RAM, of course. (https://golang.org/pkg/io/#ReadFull https://golang.org/pkg/io/#ReadFull ) Whether or not it's a mistake depends on your situation. Fortunately, in the HTTP context, you have a Content-Length that you can examine to make decisions if you need to; the generalized io.Reader does not have that, either. This all comes from the nature of HTTP itself. I tend to consider it a basic requirement for any web development environment that there be a way to correctly deal with an HTTP upload as a stream, and that the environment not "helpfully" unconditionally load the entire stream into memory for me.
- Someone 11y agoIf you want efficiency, I think it is the only way to do it. Responses can be huge. Why would the library commit to storing them at all, and determine the data type to store them in? Yes, you could make an API that allows the library user to control that, but that would be needlessly complex compared to having a single call "give me the response stream", and letting the caller of that decide how much of the data to read, and what to do with it.