4 ms·

While Jason Scott raises interesting points six years ago, the principles of data management remain the same as the days of dedicated servers with on premises s

by rietta 11y ago

While Jason Scott raises interesting points six years ago, the principles of data management remain the same as the days of dedicated servers with on premises systems. If you have only one copy of data and that machine goes down, then the data goes with it. In my experience in a business context 'the cloud' discussion is about the business' desire to expand capacity without the need for the capital expenditure needed to buy equipment and server admin salaries. By the numbers it is just better to rent instead of buy at the infrastructure level. The cloud is a murky, ambiguity-laden concept though. Both Netflix and my 92 year old grandmother on Facebook 'use the cloud', but the former is much more sophisticated in their network and data management practices. My grandmother just wants to see fun pictures of her family and great grandkids.

deleted 11y ago
[deleted]
FussyZeus 11y ago
That's what the author is getting at, "The Cloud" is sold as this awesome thing that will never die or break, and worse yet it's sold to people who often don't know any better. No one realizes that the Cloud is running on the same crap we've always had and is vulnerable to the same issues as everything else. MAYBE the company is better at data management, MAYBE the employees take pride in their job and do it properly, but that's all MAYBE MAYBE MAYBE, and could just as well be "no" and you're entrusting your data to people who really have nothing to lose if it goes into the garbage tomorrow.
- BookmarkSaver 11y ago
  Eh... I think that the maybes you are using are a little bit misleading. When a company sells space in their "cloud", like maybe Microsoft or Amazon, there is a business guarantee that goes along with it. If Amazon were to randomly lose a big chuck of Netflix data, AWS's business would tank immediately. AWS is a giant system that uses its scale and number of customers to efficiently provide a more stable system at a lower cost than all of those individual customers could achieve by building and maintaining their own IT. I think it is sort of like a delivery system. If USPS or FedEx or USP started losing a massive number of packages (I know that they do lose some) then they would get abandoned, just like "Cloud" companies have an incentive to maintain a baseline level of quality. The alternative is that every business would have to create their own shipping services. I think it makes sense to assume that in most cases, unless the business is already massive enough to warrant it, that it is cheaper and more reliable to use the aggregate, dedicated ones for hire. The "Cloud" will be cheaper than individual implementations, and it won't be nearly as suspect to individual implementation errors because the identical system will have been proven by many other customers (otherwise it would be abandoned).
  - fleitz 11y ago
    > If Amazon were to randomly lose a big chuck of Netflix data, AWS's business would tank immediately. http://www.theregister.co.uk/2015/09/20/aws_database_outage/ http://www.theregister.co.uk/2015/09/20/aws_database_outage/ AWS is the new Microsoft / IBM, nobody ever got fired for picking AWS.
    - BookmarkSaver 11y ago
      There is a difference between a service outages, which do happen and are unavoidable no matter who builds your system, and actually losing permanent data. Which is Amazon does guarantee only 99.95% uptime. There is a difference in a discussion about trusting the Cloud with your data and services between it going down briefly on occasion (somewhat acceptable, within very narrow limits) and actually losing data or longterm traffic because of a service failure. Seriously breaching the SLA causes compensation as well as a big loss of reputation and business, going down for a couple hours once a year is hardly the type of instability that would terrify most online businesses, nor is it something that individual companies are able to avoid themselves.
  - iofj 11y ago
    That must be why Amazon's SLA is defined as follows[1] : If amazon loses more than 3 datacenters (only total loss of external connectivity for all of your instances in an entire availability zone, or total loss of hard disk access, again only counts if all your instances completely lose hard disk/EBS access) for more than 45 minutes in a month you get 10% of what you pay as a voucher for future ec2 usage. If they lose it for more than 7 hours you get 30%. So no, Amazon, or at least their legal department, does not trust their own competency. Or at least, they're not willing to risk any revenue on that, but they're willing to give you a small future discount to encourage you to restart using the service. Oh and you only get that if you explicitly ask for it. If they lose your data on EBS/S3/Dynamo/..., you get nothing. So having any data exclusively on any Amazon service should be cause for getting fired, and this of course also means that using Dynamo for storing anything non-trivial is a big no-no from a disaster recovery standpoint. So I have to say, I would suggest you do not trust Amazon with either your data, nor with keeping your site online. Yes, historically their performance has been better than this, but ... This reads worse than the SLAs on internet connectivity from places like level3 and cogent (pay 10% less if they fuck up completely for more than 2 days). [1] https://aws.amazon.com/ec2/sla/ https://aws.amazon.com/ec2/sla/
asuffield 11y ago
(Tedious disclaimer: not speaking for anybody else, my opinion only, etc. I'm an SRE at Google.) The key piece that's missing here is the idea that risk is something you have to compare, and can combine in interesting ways, then trade off against costs. There are a bunch of ways in which you can do compute, storage, and networking. You get to pick zero or more of these ways. One of them is "buy a bunch of iron and make a pile of it in your bedroom". Major risk factors here are your house burning down, you getting evicted, or there being a power cut. Another is "rent those services from an infrastructure provider". Risk factors here are much harder for you to visualise, but include things like "governments ban that company from operating in your country". You can look at the risk of any of these options, and quantify it with an SLO, like "we intend for this compute resource to be available 99.99% of the time in a given quarter". You can then have an SLA that defines what will happen if that objective is not met, and measure how often this is complied with over time. There are lots of ways to analyse this information, but let's suppose that you can reduce it to a single number measuring how safe the resource is for your use case. If you only look at a single option, and say "this has a safety of X", then the only thing you can get out of this effort is anxiety. This only becomes interesting when you start looking at differences between alternatives, like "the safety of servers in my bedroom is X, but the safety of buying resources on GCE is Y, so I can get this much of an improvement by spending that amount of money", or "by doing both of these things I improve my safety to Z, and I am willing to pay the additional cost of doing so". Or perhaps your position would be "this option is less safe but much cheaper and I'm willing to accept the extra risk". The problem I have with the "fuck the cloud" article is that it doesn't do any of this. All it says is "the safety of this option is only X, you should experience anxiety". Is X higher or lower than that pile of iron in your bedroom? You still don't know. (Realistically, unless you have the ability to build a system in your bedroom that has continental diversity for storage, N+2 of everything for hardware failure, etc, your bedroom is likely to be far less safe than the major cloud services - unless you live in a country which regularly bans American companies from doing business with you, which a sixth of the world's population does.)
- fche 11y ago
  There should exist an option that is between those spectrum endpoints. One that runs a lot more free (inspectable, trustworthy) software than present cloud services. One that leaves some control with the owner of the data.

by rietta 11y ago

deleted 11y ago

[deleted]

FussyZeus 11y ago

That's what the author is getting at, "The Cloud" is sold as this awesome thing that will never die or break, and worse yet it's sold to people who often don't know any better. No one realizes that the Cloud is running on the same crap we've always had and is vulnerable to the same issues as everything else. MAYBE the company is better at data management, MAYBE the employees take pride in their job and do it properly, but that's all MAYBE MAYBE MAYBE, and could just as well be "no" and you're entrusting your data to people who really have nothing to lose if it goes into the garbage tomorrow.

BookmarkSaver 11y ago
Eh... I think that the maybes you are using are a little bit misleading. When a company sells space in their "cloud", like maybe Microsoft or Amazon, there is a business guarantee that goes along with it. If Amazon were to randomly lose a big chuck of Netflix data, AWS's business would tank immediately. AWS is a giant system that uses its scale and number of customers to efficiently provide a more stable system at a lower cost than all of those individual customers could achieve by building and maintaining their own IT. I think it is sort of like a delivery system. If USPS or FedEx or USP started losing a massive number of packages (I know that they do lose some) then they would get abandoned, just like "Cloud" companies have an incentive to maintain a baseline level of quality. The alternative is that every business would have to create their own shipping services. I think it makes sense to assume that in most cases, unless the business is already massive enough to warrant it, that it is cheaper and more reliable to use the aggregate, dedicated ones for hire. The "Cloud" will be cheaper than individual implementations, and it won't be nearly as suspect to individual implementation errors because the identical system will have been proven by many other customers (otherwise it would be abandoned).
- fleitz 11y ago
  > If Amazon were to randomly lose a big chuck of Netflix data, AWS's business would tank immediately. http://www.theregister.co.uk/2015/09/20/aws_database_outage/ http://www.theregister.co.uk/2015/09/20/aws_database_outage/ AWS is the new Microsoft / IBM, nobody ever got fired for picking AWS.
  - BookmarkSaver 11y ago
    There is a difference between a service outages, which do happen and are unavoidable no matter who builds your system, and actually losing permanent data. Which is Amazon does guarantee only 99.95% uptime. There is a difference in a discussion about trusting the Cloud with your data and services between it going down briefly on occasion (somewhat acceptable, within very narrow limits) and actually losing data or longterm traffic because of a service failure. Seriously breaching the SLA causes compensation as well as a big loss of reputation and business, going down for a couple hours once a year is hardly the type of instability that would terrify most online businesses, nor is it something that individual companies are able to avoid themselves.
- iofj 11y ago
  That must be why Amazon's SLA is defined as follows[1] : If amazon loses more than 3 datacenters (only total loss of external connectivity for all of your instances in an entire availability zone, or total loss of hard disk access, again only counts if all your instances completely lose hard disk/EBS access) for more than 45 minutes in a month you get 10% of what you pay as a voucher for future ec2 usage. If they lose it for more than 7 hours you get 30%. So no, Amazon, or at least their legal department, does not trust their own competency. Or at least, they're not willing to risk any revenue on that, but they're willing to give you a small future discount to encourage you to restart using the service. Oh and you only get that if you explicitly ask for it. If they lose your data on EBS/S3/Dynamo/..., you get nothing. So having any data exclusively on any Amazon service should be cause for getting fired, and this of course also means that using Dynamo for storing anything non-trivial is a big no-no from a disaster recovery standpoint. So I have to say, I would suggest you do not trust Amazon with either your data, nor with keeping your site online. Yes, historically their performance has been better than this, but ... This reads worse than the SLAs on internet connectivity from places like level3 and cogent (pay 10% less if they fuck up completely for more than 2 days). [1] https://aws.amazon.com/ec2/sla/ https://aws.amazon.com/ec2/sla/

asuffield 11y ago

(Tedious disclaimer: not speaking for anybody else, my opinion only, etc. I'm an SRE at Google.) The key piece that's missing here is the idea that risk is something you have to compare, and can combine in interesting ways, then trade off against costs. There are a bunch of ways in which you can do compute, storage, and networking. You get to pick zero or more of these ways. One of them is "buy a bunch of iron and make a pile of it in your bedroom". Major risk factors here are your house burning down, you getting evicted, or there being a power cut. Another is "rent those services from an infrastructure provider". Risk factors here are much harder for you to visualise, but include things like "governments ban that company from operating in your country". You can look at the risk of any of these options, and quantify it with an SLO, like "we intend for this compute resource to be available 99.99% of the time in a given quarter". You can then have an SLA that defines what will happen if that objective is not met, and measure how often this is complied with over time. There are lots of ways to analyse this information, but let's suppose that you can reduce it to a single number measuring how safe the resource is for your use case. If you only look at a single option, and say "this has a safety of X", then the only thing you can get out of this effort is anxiety. This only becomes interesting when you start looking at differences between alternatives, like "the safety of servers in my bedroom is X, but the safety of buying resources on GCE is Y, so I can get this much of an improvement by spending that amount of money", or "by doing both of these things I improve my safety to Z, and I am willing to pay the additional cost of doing so". Or perhaps your position would be "this option is less safe but much cheaper and I'm willing to accept the extra risk". The problem I have with the "fuck the cloud" article is that it doesn't do any of this. All it says is "the safety of this option is only X, you should experience anxiety". Is X higher or lower than that pile of iron in your bedroom? You still don't know. (Realistically, unless you have the ability to build a system in your bedroom that has continental diversity for storage, N+2 of everything for hardware failure, etc, your bedroom is likely to be far less safe than the major cloud services - unless you live in a country which regularly bans American companies from doing business with you, which a sixth of the world's population does.)

fche 11y ago
There should exist an option that is between those spectrum endpoints. One that runs a lot more free (inspectable, trustworthy) software than present cloud services. One that leaves some control with the owner of the data.