6 ms·

And there's no gotcha there, Google's always been open that retrieval fees are the tradeoff for Coldline being otherwise so cheap. Nearline is the better option

And there's no gotcha there, Google's always been open that retrieval fees are the tradeoff for Coldline being otherwise so cheap. Nearline is the better option for data you'll access more once a quarter.

gunapologist99 5y ago
"so cheap" might be a bit overly relative in this usage..
- immibis 5y ago
  I guess the only time you would use Coldline is if you rarely access the data, but when you do access it, a retrieval delay is unacceptable. If you access it frequently, use a cheaper retrieval tier; if you can tolerate a delay, use Glacier or GCP's Archive tier.
  - re 5y ago
    > if you can tolerate a delay, use Glacier S3 recently added a Coldline-like "Glacier Instant Retrieval" class, FYI. Their "Deep Archive" class (the cheapest) still does require restore operations that take hours to complete, though. https://aws.amazon.com/s3/storage-classes/glacier/instant-retrieval/ https://aws.amazon.com/s3/storage-classes/glacier/instant-re... > or GCP's Archive tier AFAIK, the GCS Archive tier has the same availability characteristics as Coldline and the same latency as all GCS class (10s of milliseconds). It seems like the primary factor for how you'd choose a GCS storage class would be your cost projections based on how long you store objects for and how frequently you access them. https://cloud.google.com/storage/docs/storage-classes#archive https://cloud.google.com/storage/docs/storage-classes#archiv...
    - Dylan16807 5y ago
      > AFAIK, the GCS Archive tier has the same availability characteristics as Coldline and the same latency as all GCS class (10s of milliseconds). That's true as far as I know. And the combination of "the data is accessible instantly" with "there is no low priority retrieval, there is only the ultra expensive kind" implies that their architecture is extremely weird and/or some very manipulative pricing is happening.
      - the-rc 5y ago
        Neither. It only means that hot and cold storage live right next to each other and pricing is used as a carrot/stick, as well as a signal, so that the two classes tend to stay the same (mainly to prevent cold from getting hot, of course) and a desirable balance is preserved. You could call it manipulative, if you want, but it's more like incentives.
        Dylan16807 5y ago
        Amazon can apparently get data cheaply out of cold storage with a day or so of warning. Can google not do that? And if they did, they could still maintain the same kind of tiering. If they can't do it, that's a very strange system. If they refuse to do it, that's manipulative. For archival storage Google's charging three and a half years of storage costs to retrieve data. Even in the most flattering scenario where roughly all of the cost is in doing the I/O, and disk space is "free", that implies that retrieval is at least 75% profit. If I/O is half the cost of storage and disk space is half the cost of storage, then retrieval approaches 90% profit.
        immibis 5y ago
        Sir, we live in a capitalist economy. Prices don't mirror costs.
        Dylan16807 5y ago
        My complaint is that the competition is awful here. Capitalism is supposed to pit companies against each other and drive profit margins below 50%. And it's failing to do that here. And moreso, it's a very cruel pricing system because it lures you in with low numbers, then overcharges to get your data back when you need it. Being antagonistic to your customers does have negative effects in the long run. And I think it's worth pointing out situations like that when people are shopping around.
        the-rc 5y ago
        If you're doing disaster recovery, do you really want to wait a day for your data? I don't understand why you want Google to slow things down. Why can't Amazon get cold data faster? It's fairly simple: all Google storage is online. There is public material on how it all works. Maybe you can ask them to add an artificial delay. There is offline storage, yes, but that's tape, which has bigger issues... Your pricing model is flawed, because you're not taking into account other factors such as rebalancing. Years ago, I would have loved for retrieval to be that cheap to perform behind the scenes.

gunapologist99 5y ago

"so cheap" might be a bit overly relative in this usage..

immibis 5y ago
I guess the only time you would use Coldline is if you rarely access the data, but when you do access it, a retrieval delay is unacceptable. If you access it frequently, use a cheaper retrieval tier; if you can tolerate a delay, use Glacier or GCP's Archive tier.
- re 5y ago
  > if you can tolerate a delay, use Glacier S3 recently added a Coldline-like "Glacier Instant Retrieval" class, FYI. Their "Deep Archive" class (the cheapest) still does require restore operations that take hours to complete, though. https://aws.amazon.com/s3/storage-classes/glacier/instant-retrieval/ https://aws.amazon.com/s3/storage-classes/glacier/instant-re... > or GCP's Archive tier AFAIK, the GCS Archive tier has the same availability characteristics as Coldline and the same latency as all GCS class (10s of milliseconds). It seems like the primary factor for how you'd choose a GCS storage class would be your cost projections based on how long you store objects for and how frequently you access them. https://cloud.google.com/storage/docs/storage-classes#archive https://cloud.google.com/storage/docs/storage-classes#archiv...
  - Dylan16807 5y ago
    > AFAIK, the GCS Archive tier has the same availability characteristics as Coldline and the same latency as all GCS class (10s of milliseconds). That's true as far as I know. And the combination of "the data is accessible instantly" with "there is no low priority retrieval, there is only the ultra expensive kind" implies that their architecture is extremely weird and/or some very manipulative pricing is happening.
    - the-rc 5y ago
      Neither. It only means that hot and cold storage live right next to each other and pricing is used as a carrot/stick, as well as a signal, so that the two classes tend to stay the same (mainly to prevent cold from getting hot, of course) and a desirable balance is preserved. You could call it manipulative, if you want, but it's more like incentives.
      - Dylan16807 5y ago
        Amazon can apparently get data cheaply out of cold storage with a day or so of warning. Can google not do that? And if they did, they could still maintain the same kind of tiering. If they can't do it, that's a very strange system. If they refuse to do it, that's manipulative. For archival storage Google's charging three and a half years of storage costs to retrieve data. Even in the most flattering scenario where roughly all of the cost is in doing the I/O, and disk space is "free", that implies that retrieval is at least 75% profit. If I/O is half the cost of storage and disk space is half the cost of storage, then retrieval approaches 90% profit.
        immibis 5y ago
        Sir, we live in a capitalist economy. Prices don't mirror costs.
        Dylan16807 5y ago
        My complaint is that the competition is awful here. Capitalism is supposed to pit companies against each other and drive profit margins below 50%. And it's failing to do that here. And moreso, it's a very cruel pricing system because it lures you in with low numbers, then overcharges to get your data back when you need it. Being antagonistic to your customers does have negative effects in the long run. And I think it's worth pointing out situations like that when people are shopping around.
        the-rc 5y ago
        If you're doing disaster recovery, do you really want to wait a day for your data? I don't understand why you want Google to slow things down. Why can't Amazon get cold data faster? It's fairly simple: all Google storage is online. There is public material on how it all works. Maybe you can ask them to add an artificial delay. There is offline storage, yes, but that's tape, which has bigger issues... Your pricing model is flawed, because you're not taking into account other factors such as rebalancing. Years ago, I would have loved for retrieval to be that cheap to perform behind the scenes.