> Why is it so hard to understand... No, they should not have used S3 to begin with. S3 is for Web, it's not for storage. These people used a wrong tool because

> Why is it so hard to understand... No, they should not have used S3 to begin with. S3 is for Web, it's not for storage. These people used a wrong tool because they don't even know what tools they have... S3 is absolutely for storage. So much so that it is quite literally in the name. I'm morbidly curious what purpose you think S3 fills, if not storage? > [capabilities of read/write] Yes, while read/write can operate on a socket, that doesn't mean it's usable by a library that's going to be expecting image data. Let's say our library which is going to expect to read image data does accept just any old FD, (and not a FILE pointer); even then, I can't simply hand it a socket: the socket is going to be speaking HTTP/TLS, and the library expects image data. In the best case, you'd get a parse error. I mention FUSE drivers in the comment, and so too does the article. The article itself notes they're on macOS, where FUSE is a second-class citizen. There's pragmatic reasons to avoid it. (They also note that it read the entire file, which, while that shouldn't be true, if they errantly believed it to be so, then it would reasonably drive their judgement in the direction they went.) > In case you ever had the "engineer" title, things like this should be the reason to lose it. I don't know why you feel the need to include personal attacks like this. Guess I'm not an engineer anymore! > It's not a given. It's never a given. It's data stored in the proprietary data-center "somehow". But it may be out of this engineers scope, or out of their control; they may have been given something resembling "hey, we have this data, already stored in Blobstore, can you do this task with it?", and within that scope, it's a given. Yes, in the grand design of the universe, it need not be, but for the engineer right there in OP, that might be boiling an ocean. But also … the data likely has to be stored somewhere, and there are good reasons to choose such a service. Given the file sizes mentioned in the article, they need a fair bit of storage (something S3 et al. is good at) and they need access to individual files without a lot of special needs that might be better filled by a true FS like ext4. Within reason, it seems like one can assume that Blobstore isn't an unreasonable choice here. > Not to mention that moving things inside that proprietary data-center is going to be much faster, so, even if they are required to acquire data as blob storage, they can almost instantly convert it into something more usable. And format change is going to be something you can do with S3, too. The only thing you really get by moving into a proprietary data center is that the data might be slightly more local, and thus have better latency. Format conversions are sometimes hard: that format may be required for other use cases. But again, this all seems like questioning things in the OP that … don't really need to be questioned from our armchairs.