4 ms·

I've found this is a common thing to say with AWS employees. One of them insisted that Amazon's ridiculous ephemeral storage policy (immediate, permanent, and i

I've found this is a common thing to say with AWS employees. One of them insisted that Amazon's ridiculous ephemeral storage policy (immediate, permanent, and irrevocable deletion on any halt or stop event, making accidental data loss a real possibility) had to be that way because it would just take too much hardware to allow a cooldown period before the drives were wiped. There's no way I believe that. I think Amazon is just used to intimidating customers with exactly that line of reasoning: "No offense, but you have no idea how hard the cloud is", and people buy it because "the cloud" is the new hotness.

jldugger 12y ago
I've had RAID 6 fail. It should be extremely rare, but isn't. And at AWS's scale, It's not hard to imagine servers going offline regularly. Ephemeral storage as a policy makes sense to me in the sense that you can separate out that what's important from that which is ephemeral, and provide cheaper storage than a more HA solution like ganeti.
saywhst 12y ago
if you need persistant storage, use EBS. Ephemeral is self-defined.
deleted 12y ago
[deleted]
MichaelCrawford 12y ago
I've been looking for work for a while, but I won't even respond to solicitations or job board posts that so much as mention the cloud, agile or scrum.
- MichaelCrawford 12y ago
  Rather than modding me down, perhaps you could explain why you disagree.
  - tracker1 12y ago
    What you said adds pretty much no value to the conversation, I didn't mod you down, and I didn't look to see precisely what field you work in, just the same, cloud is pretty much an obfuscation for on-demand co-location and shared-hosting services. While you may not like the trend, and some providers have better options than others, and you may require some operations over others... but very few businesses can afford to manage multi-site infrastructure that can dynamically scale. Most are probably served just fine with rented servers, traditional colocation, or don't even need more than one VPS... That doesn't make the technology bad, and only makes you seem ignorant in your prior statement.
    - MichaelCrawford 12y ago
      I admit I could have explained why. I'll go into it later, but for the most part I consider the cloud a really bad idea. I also regard most cloud companies as "buzzword-enabled" so as to attract investors. My gripe with scrum and agile is not so much with the methodologies, but with companies that think they have a methodology when in reality they have a bureaucracy.
DrStalker 12y ago
Why isn't the persistent data put onto an EBS volume? If ephemeral storage is a drive local to the virtual machine's host (which I think is the case) then having a cool down period would mean holding the hardware that used to be used by you in reserve until the grace period expired. It's possible but it's a lot of work to solve a problem that can be better solved by not relying on ephemeral storage persisting.
- cookiecaper 12y ago
  I mean, it's a nice idea in theory, but in practice stuff finds its way onto the ephemeral disk even if you have EBS volumes mounted, and "Sorry, we just deleted all your crap, I guess you should've had that on EBS" (which is an extra fee by the way) is not an acceptable solution to the problem. Yes, it would mean holding the hardware in reserve for the cooldown period. I'm not talking months here, just enough time to recover from an accidental "sudo shutdown -h now" instead of "sudo shutdown -r now" (or similar). It'd be nice if Amazon sent an email warning about the condition and giving you an hour or so to go in and save your data/restart your instance. They could even make it a policy that you're charged for the time your instance is running + 1 hour to facilitate cooldown feature if they're really that worried about it; it's better than wiping data as soon as someone stops (from AWS console) or shuts down (from real console) an instance and providing absolutely no avenue for recovery, no matter how quickly you notice the mistake.
  - NathanKP 12y ago
    I have never had stuff accidentally find its way onto ephemeral storage. The ephemeral storage is mounted at a specific location of /mnt. Everything else on the system (OS, binaries, application code and resources) is stored on an EBS volume. You have to specifically put something into the /mnt folder if you want it to be stored on the ephemeral storage. Any other location is safe and will persist through halts and stops. In practice the only thing you should ever use the /mnt folder is maybe a Nginx disk cache, or as an alternative /tmp or something like that. Basically if stuff you don't want to lose is finding its way onto the ephemeral storage then you are doing something wrong.

jldugger 12y ago

I've had RAID 6 fail. It should be extremely rare, but isn't. And at AWS's scale, It's not hard to imagine servers going offline regularly. Ephemeral storage as a policy makes sense to me in the sense that you can separate out that what's important from that which is ephemeral, and provide cheaper storage than a more HA solution like ganeti.

saywhst 12y ago

if you need persistant storage, use EBS. Ephemeral is self-defined.

deleted 12y ago

[deleted]

MichaelCrawford 12y ago

I've been looking for work for a while, but I won't even respond to solicitations or job board posts that so much as mention the cloud, agile or scrum.

MichaelCrawford 12y ago
Rather than modding me down, perhaps you could explain why you disagree.
- tracker1 12y ago
  What you said adds pretty much no value to the conversation, I didn't mod you down, and I didn't look to see precisely what field you work in, just the same, cloud is pretty much an obfuscation for on-demand co-location and shared-hosting services. While you may not like the trend, and some providers have better options than others, and you may require some operations over others... but very few businesses can afford to manage multi-site infrastructure that can dynamically scale. Most are probably served just fine with rented servers, traditional colocation, or don't even need more than one VPS... That doesn't make the technology bad, and only makes you seem ignorant in your prior statement.
  - MichaelCrawford 12y ago
    I admit I could have explained why. I'll go into it later, but for the most part I consider the cloud a really bad idea. I also regard most cloud companies as "buzzword-enabled" so as to attract investors. My gripe with scrum and agile is not so much with the methodologies, but with companies that think they have a methodology when in reality they have a bureaucracy.

DrStalker 12y ago

Why isn't the persistent data put onto an EBS volume? If ephemeral storage is a drive local to the virtual machine's host (which I think is the case) then having a cool down period would mean holding the hardware that used to be used by you in reserve until the grace period expired. It's possible but it's a lot of work to solve a problem that can be better solved by not relying on ephemeral storage persisting.

cookiecaper 12y ago
I mean, it's a nice idea in theory, but in practice stuff finds its way onto the ephemeral disk even if you have EBS volumes mounted, and "Sorry, we just deleted all your crap, I guess you should've had that on EBS" (which is an extra fee by the way) is not an acceptable solution to the problem. Yes, it would mean holding the hardware in reserve for the cooldown period. I'm not talking months here, just enough time to recover from an accidental "sudo shutdown -h now" instead of "sudo shutdown -r now" (or similar). It'd be nice if Amazon sent an email warning about the condition and giving you an hour or so to go in and save your data/restart your instance. They could even make it a policy that you're charged for the time your instance is running + 1 hour to facilitate cooldown feature if they're really that worried about it; it's better than wiping data as soon as someone stops (from AWS console) or shuts down (from real console) an instance and providing absolutely no avenue for recovery, no matter how quickly you notice the mistake.
- NathanKP 12y ago
  I have never had stuff accidentally find its way onto ephemeral storage. The ephemeral storage is mounted at a specific location of /mnt. Everything else on the system (OS, binaries, application code and resources) is stored on an EBS volume. You have to specifically put something into the /mnt folder if you want it to be stored on the ephemeral storage. Any other location is safe and will persist through halts and stops. In practice the only thing you should ever use the /mnt folder is maybe a Nginx disk cache, or as an alternative /tmp or something like that. Basically if stuff you don't want to lose is finding its way onto the ephemeral storage then you are doing something wrong.