6 ms·
This article contains some misunderstandings about the S3 API. > The interface to upload data into Amazon S3 is actually a bit simpler than Backblaze B2’s API.
by willglynn 8y ago
This article contains some misunderstandings about the S3 API.
> The interface to upload data into Amazon S3 is actually a bit simpler than Backblaze B2’s API. But it comes at a literal cost. It requires Amazon to have a massive and expensive choke point in their network: load balancers. When a customer tries to upload to S3, she is given a single upload URL to use. For instance, http://s3.amazonaws.com/<bucketname> http://s3.amazonaws.com/<bucketname>. This is great for the customer as she can just start pushing data to the URL. But that requires Amazon to be able to take that data and then, in a second step behind the scenes, find available storage space and then push that data to that available location. The second step creates a choke point as it requires having high bandwidth load balancers. That, in turn, carries a significant customer implication; load balancers cost significant money.
In fact, S3's REST API requires callers to follow HTTP redirects, and the PUT documentation expressly mentions the HTTP "Expect: 100-continue" mechanism precisely so that the S3 endpoint you reach in your initial PUT request does not have to handle the HTTP request body.
https://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.html https://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.ht...
https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPU...
> The Dispatching Server (the API server answering the b2_get_upload_url call) tells the Client “there is space over on “Vault-8329.” This next step is our magic. Armed with the knowledge of the open vault, the Client ends its connection with the Dispatching Server and creates a brand new request DIRECTLY to Vault-8329 (calling b2_upload_file or b2_upload_part). No load balancers involved!
Again, this could be done directly with HTTP. PUT to the first server, receive a redirect, PUT to vault-8329, receive "100 Continue", transmit file. There's no need to have a separate API call to get the "real" upload URL.
> 3) Expensive, time consuming data copy needs (and “eventual consistency”). Amazon S3 requires the copying of massive amounts of data from one part of their network (the upload server) to wherever the data’s ultimate resting place will be. This is at the root of one of the biggest frustrations when dealing with S3: Amazon’s “eventual consistency.”
Wait, I thought they were load balancers? Why does the load balancer need to copy any data once it's done uploading?
As for eventual consistency, there is truth to this complaint -- but much less truth than in the distant past. Every S3 region except us-standard has always had read-after-write consistency for new objects since launch, and as of August 2015, us-standard does too:
https://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3-introduces-new-usability-enhancements/ https://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3...
If your PUT returns 200 OK, a subsequent GET will return the object, assuming you're using unique keys. This prevents the 2015-and-earlier problem where you'd create a new S3 object and enqueue a job to process it, then the job gets 404 Not Found while retrieving the new object.
There are other cases where S3's eventual consistency can be an issue, but none of them have been dealbreakers for my applications. Having said that: S3's consistency model is a weaker than the model B2 provides, so this is not an argument against providing an S3-compatible interface.
- cmurphycode 8y agoFor the HTTP 100 vs 307 - You're right, but in my experience, S3 does not send 307 on each request (which it would have to do, if it were load balancing, right?)
- brianwski 8y agoDisclaimer: I'm the original author of the blog post. :-) > In fact, S3's REST API requires callers to follow HTTP redirects... Yes. Let's take the example of uploading 3 small files. With S3, every upload call must start by hitting the same URL that is then redirected. To upload the same 3 small files in B2, it is a little different. For B2, one call is made to the "dispatch server" to ask for where there is spare space, then there is no redirect. The client must disconnect, and contact the final destination three times. So for this example, S3 would need to do 3 redirects (3 total network calls) and Backblaze B2 would do zero redirects but make 4 total network calls to upload three files. I hope that makes sense. I'm not saying one is better than the other, and I freely admit S3 can be a little more intuitive. But it saves Backblaze money to not have such high loads and high uptime demand on the original URL. > Again, this could be done directly with HTTP. Yes I agree it could have been. > There's no need to have a separate API call to get the "real" upload URL. The "need" was to save money. See example above. Amazon S3 would require 3 redirects from the original URL, while Backblaze B2 chose a different tradeoff that is zero redirects and 4 total requests where 3 out of 4 requests were NOT redirected. The 3 out of 4 requests go to the final location DIRECTLY. If you are only uploading one file, the Amazon S3 API is about as efficient as Backblaze and I agree with you. But if you upload 1 million files, the Backblaze B2 architecture is cheaper/simpler. But either way, it is the tradeoff we chose. > as of August 2015, us-standard... has read-after-write consistency for new objects TIL. We launched the B2 API before August of 2015, sorry if I propagated old info!
- teraflop 8y ago> For B2, one call is made to the "dispatch server" to ask for where there is spare space, then there is no redirect. An API call that retrieves the address of another machine to contact is functionally the same thing as an HTTP redirect, just with different syntax, right? > The client must disconnect, and contact the final destination three times. This seems like the key idea that wasn't quite explained in the blog post: the client is expected to cache and reuse the same "vault" server for future requests, not just for multiple pieces of a single upload. If the client doesn't conform to that expectation, your approach would end up with exactly the same performance characteristics as S3. What I find interesting is that unless I'm misunderstanding the API documentation [1], file downloads apparently do go through a load balancer. That is, you make an HTTP request to a single endpoint that's the same for all files in the account, and it fetches the data from whichever "vault" server is actually storing it. So does that mean that Backblaze's rate of incoming uploads is much larger than the rate of downloads? Otherwise it seems like you could just reuse the same load-balancing infrastructure for both. [1]: https://www.backblaze.com/b2/docs/b2_download_file_by_name.html https://www.backblaze.com/b2/docs/b2_download_file_by_name.h...