4 ms·
As it turns out, it's really hard to monetize an S3 bucket at scale.
by Spivak 9y ago
As it turns out, it's really hard to monetize an S3 bucket at scale.
- TekMol 9y agoWould it be easier to monetize a small S3 bucket?
- michaelt 9y agoNo, but my $3 AWS bill is much easier to ignore than their $300,000 one.
- pier25 9y agoWhat if they moved to their own infrastructure? I imagine being an image hosting service it would be cheaper in the long run, although I have no idea by how much and if it would make sense all things considered.
- takeda 9y agoThings like that makes me wish NDN[1] would succeed. Essentially in this network instead using addressing to address hosts, they address the data. This brings interesting properties, since routers are aware what data is being transferred they can start caching it. In the end no CDNs are needed (network takes care of it) so people can host data themselves without worrying of it getting too popular. [1] https://named-data.net/ https://named-data.net/
- MichaelGG 9y agoExcept the end-user ISPs are quite content with requesting traffic from the providers; they like the imbalance as it gives them something to negotiate with (that's why inbound is free on cloud - they're happy to fight the imbalance). Why would they start caching your images for free? Also doesn't that mean everyone in between know you're requesting degeneration.jpg? Vs now where they'd only know the domain and maybe could do transfer size analysis? Sort of the opposite direction of TLS everywhere (CloudFlare exempted)? Also how do you bootstrap it? If I upload some image and it's instantly popular on reddit and generating many Mbps of traffic, don't I have to stick around long enough to get it up and cached? Or would there be free seed services that will get the first several GB out?
- takeda 9y ago> Except the end-user ISPs are quite content with requesting traffic from the providers; they like the imbalance as it gives them something to negotiate with (that's why inbound is free on cloud - they're happy to fight the imbalance). Why would they start caching your images for free? Normally that would benefit ISPs (they do have a network of routers so it would naturally help them, since they would not need to send the same thing over and over again, they do pay for traffic to their uplinks). But, on the other hand ISPs artificially reduce their bandwidth and expect companies like Netflix to pay and do peering with them. Well, this won't help against it, but while it can replace TCP/IP it can also work as an overlay network and you can skip the ISP. > Also doesn't that mean everyone in between know you're requesting degeneration.jpg? Vs now where they'd only know the domain and maybe could do transfer size analysis? Sort of the opposite direction of TLS everywhere (CloudFlare exempted)? The protocol itself doesn't leak that information. Remember, there are no addresses to individual machines to people, only addresses for the data. When you request something the routing protocol will route the request to the source of data. Each router remembers the previous hop the request came from. On each hop a router checks if it already caches the data and responds, otherwise forwards request further. So even the source website doesn't know who requested it. I suppose the issue might be that router immediately would know what you requested. You can mitigate it in two ways: - encrypt the data and only give key to decrypt it to subset of people who supposed to be able to access it - encrypt data individually per user, in that case you no longer can rely on caching properties of the network and the source server needs to be aware who requested what (since it is encrypting the data). i.e. you falling back to the way how TCP/IP now operates. > Also how do you bootstrap it? If I upload some image and it's instantly popular on reddit and generating many Mbps of traffic, don't I have to stick around long enough to get it up and cached? Or would there be free seed services that will get the first several GB out? Well you're expected to host it all the time. The network won't be caching it forever individual packets have TTL and also routers can purge data if they need to make space for other data that is now popular. I suggested NDN as a response to that S3 bucket doesn't handle highly popular content well. The NDN properties allow highly popular content being hosted even from places with low bandwidth, because the network will handle the load.