6 ms·

At this scale, why not invest in your own datacenter, since you already have a lot of servers to maintain? I understand you have to use Office 365, but what is

At this scale, why not invest in your own datacenter, since you already have a lot of servers to maintain? I understand you have to use Office 365, but what is the value prop of Azure to you beyond that ?

ptd 7y ago
One point(though I will admit this is the cynical part of my brain speaking) immediately jumps to mind. If the server goes down, you can blame Microsoft. Even the least technical person can’t blame you for that. If you create your own data center however, if it goes down, you may potentially be on the chopping block if service is interrupted. Self preservation is a strong motivator, perhaps the strongest in a business environment.
- adev_ 7y ago
  > Self preservation is a strong motivator, perhaps the strongest in a business environment. And it's the most bullshit one. Outsourcing the risk does not mean avoiding it, it means however putting it out of control. This is a typical manager bullshit attitude "nobody got fired to buy IBM" that generally lead to adopt unadapted, bloated, overpriced solutions to trivial problems. Just because they do not have the balls to do things properly. Running out of its responsibilities should be a criteria to get fired when things go badly wrong. The outsource of the 737 MAX MCAS code to India today is a perfect example of that
  - scarface74 7y ago
    Yes, surprisingly enough, people look out for their own self interests. What makes outsourcing hardware different from the dozens of other software as a service vendors that most companies depend on? Why spend the time developing competency in managing servers if that doesn’t give you a competitive advantage. But, going with the biggest most stable vendor is usually good. If you bought IBM hardware in the 70s you can still buy new hardware that supports your software. If you went with their competitors - not so much.
    - adev_ 7y ago
      > But, going with the biggest most stable vendor is usually good. If you bought IBM hardware in the 70s you can still buy new hardware that supports your software. If you went with their competitors - not so much. For 20 times the cost of commodities x86 that you should have bought by doing the right thing. And this 20 times the cost will very likely also please your own competitors because you will yourself be less competitive. That's how you finish with entire airline industry or bank still running on COBOL with no possibility of migration. IBM itself is very happy about it however.
      - scarface74 7y ago
        How much will it cost to rewrite everything? What were they suppose to choose in the 70s? Do you think that all of the people still using IBM and running legacy code are dumb or just maybe they did a cost benefit analysis and decided they didn’t need to rewrite everything in Node and React?
        adev_ 7y ago
        Maybe you should ask why some sector that "fear" failures more than anything else (banks, aviations) are still trapped into these systems while everyone else is not. This is exactly related to what I was saying before.
  - lotsofpulp 7y ago
    Bullshit means something is not true. People choosing to go with a vendor to avoid the risk of being blamed for failure, thereby reducing their chances of being fired, is a true phenomenon. And one that works. So how is it bullshit?
  - vsareto 7y ago
    >that generally lead to adopt unadapted, bloated, overpriced solutions to trivial problems. There's almost no meaningful business downside for choosing poorly. Most businesses just don't care that someone else thinks they didn't use the right tool for the job. Should it be that way? Definitely not. But it won't change unless there's consequences (and when that happens, those products will die off quickly)
  - jtwaleson 7y ago
    Agreed that outsourcing risk does not free you from the responsibilities. However, for many many services you have to rely on others to provide the service better than you could do yourself, as you are limited in time, money and other resources. For running production grade databases at startups, it's much more cost-efficient to run on AWS RDS than to hire a systems engineering team with 24/7 standby. For more trivial systems you might be right and managers might be cowards. However, you have to realize it's a spectrum. Whether you're running it yourself or you outsource it, things will go wrong at some time. When this happens, and you run it yourself, you also have to explain why it went wrong in a post-mortem or RCA. The problem is that your customers have no point of reference for your explanation (how likely is it that this occurs again?). In my experience, the following message goes down a lot smoother: "This is an Azure/AWS/GCP outage which affected not just us, but 1000s of other companies. We rely on {provider} to continuously learn from their mistakes and improve their service and they've shown this in the past. Here's their post-mortem report about this outage.". Note that not all cloud providers fit this bill.
    - adev_ 7y ago
      > However, for many many services you have to rely on others to provide the service better than you could do yourself, as you are limited in time, money and other resources. I do not criticized the fact of outsourcing when an outsourced service does a better job. This is normal and should be like that. What I am criticizing however, is outsourcing even when local/home made/OSS solutions are a better and cheaper fit even considering SLA.... just to avoid responsibilities. This is in my experience common, especially if the management has no technical background, no trust in his team and fear its upper layer.
  - coredog64 7y ago
    Boeing didn’t outsource MCAS to India. The outsourcing contract was for display software.
  - la6470 7y ago
    You just lost trust
jeswin 7y ago
Active Directory, Domain Services, Exchange Server and SharePoint Servers (which can use integrated Windows auth). Also many internal Enterprise apps are integrated with AD for auth and permissions.
luckydata 7y ago
It would be a tremendous hassle for little to no payoff as they probably don’t have the skill set to actually run a data center, and probably their size is actually not that large, they just run a lot of software I suppose (that’s usually the case in the public administration). It would be rampant malpractice to open your own data center for something like that.
- deleted 7y ago
  [deleted]
fulafel 7y ago
An potential problem is that now half of your IT recruits go to reinventing this stuff instead of working on domain problems. There aren't that many it staff per 10k municipal employees. (Of course this is assuming Azure specific hassles take much less staff time than running your own infra, not a given...)
nprateem 7y ago
Haha. Here's another one: "At this scale, why not write your own OS, since you already have a lot of servers to maintain?"
- corporateslave5 7y ago
  Haha!
sgt101 7y ago
At this scale why not contact all your neighbouring or peer municipalities and go for a joint cloud migration?
- JohannesH 7y ago
  As someone who worked in the very same municipality... one word; bureaucracy. More specific reasons were. Legal boundaries between ownership and taxpayer money separation. Of cause GDPR is also a big obstacle in joining forces on this scale. I worked on a project which tried to make a joined venture between several municipalities. I'm not sure if that partnership has ended yet, but overall you could only progress if there were people on both sides collaborating with a "better ask forgiveness, than permission" mindset. As soon as the municipality lawyers or their security officers got involved, all progress stopped immediately.
  - scarface74 7y ago
    Or not dealing with infrastructure gatekeepers.
moksly 7y ago
Our setup is mixed, all our own servers are virtual on rented space at a local server rental shop. I’m not a hardware tech, so I’m afraid that’s as technical I can get. So our SQL cluster, and most of our web applications run on our own iron. Anything external as well as a lot of managed services run in Azure where the added security and operations dev-tools are invaluable compared to what we had when we self-hosted. We do make a conscious choice about everything that moves to the cloud. We’ve spent a lot of time figuring out if buying new rental space for our SQL cluster would be cheaper than moving it to Azure. With the current political climate, we’re also a little more hesitant about moving things to clouds operated by American companies, because there is risk attached, in that we may have to move it back rather suddenly. Not something we expect will ever happen, but we don’t like risk in the public sector. So my post wasn’t so much a “we run everything in Azure” as it was “well Azure is the obvious choice for the things we do operate in the cloud”. The relationship Microsoft has with enterprise, and the expertise staff has with their products means they become the best choice for a lot of enterprise. Unless Microsoft solutions are radically more expensive than their competitors, they simply have an advantage by already being a big, and typically well liked, player in most large organisation.
chairmanwow1 7y ago
I say this as someone who used to firmly agree with this position, but in the last year my mind has been changed without reservation. The time where this was a good idea has genuinely passed. There is no earthly way that any reasonable organization will be able to provide even the security that is present by relying on a large-scale cloud provider. There is no way they will come out on top money-wise either. The big cloud players have a _absurd_ number of servers that _teams_ of some of the best CA talent the globe has to offer stressing to improve KPIs for. 10k employees in a municipality is small potatoes to what azure / AWS / GCP have dedicated to their products.
- dx034 7y ago
  Maybe not own dc, but colocation could still make sense? You don't need to be more efficient than Azure & Co. They have pretty solid profit margins, even being 30% less efficient should still be cheaper for you. And beyond a few thousand servers, I'm not even sure if scale matters that much (for server virtualization only).
- vidarh 7y ago
  I have literally never seen a setup where cloud came out less than ~twice the cost of dedicated hosting. I have seen instances where dedicated hosting comes out cheaper than putting your own equipment in a colo, but even that depends on being in a location where energy and property prices (and so colo rental costs) are high. And this does factor in devops - when doing consulting I earned consistently more (because of more hours) from clients that went for cloud setups; often they'd end up spending more times solving problems that generally didn't exist in the first place in a dedicated setup. I do see lots of people that keep assuming that the cloud players must be cheap because they're so big, but I just have never seen that bear out in practice - in part because of that attitude, the margins they can charge are far higher. Cloud providers are great for ease and for the number of services they provide, but they are generally an expensive step up.
  - dcolebatch 7y ago
    I suspect you’re doing it wrong, or there’s some niche type of computing you specialize in. In general purpose IT, looking across an enterprise portfolio of applications, we consistently see customers of Tidal Migrations replatform their applications to cloud and save 95+% in OpEx vs dedicated on-premise hosting. IMO, The first step to realizing those cost benefits is recognizing that the cloud is not your datacenter and you need to architect differently. Yes, cloud spend can grow as you open up access to more developers, but that’s why we have a plethora of tools and governance people to help make that manageable. I believe the business benefit of the agility gains that come from instant and decentralized resource provisioning will always trump any cloud bill... especially if you’re in a competitive industry & don’t want to get left behind.
christophilus 7y ago
My experience with home-rolled solutions suggests there are plenty of hidden costs. Got a new project, and want to spin up some experimental servers? Gotta wade through a bunch of IT guys. Documentation and training around Azure, etc is also going to be better than homegrown alternatives, most likely. Things like that. It’s hard to put a monetary figure on the frictions costs, but they are there.
- sergiosgc 7y ago
  You're comparing one extreme (public cloud) to another extreme (sysadmin-managed infrastructure). The alternative, today, is either a private or a hybrid cloud, which have all the advantages of short-circuiting sysadmins that public clouds have.
username90 7y ago
Public institutions usually have a lot of systems but not a lot of traffic (if we compare to tech companies), so it makes sense for them to put things in the cloud instead of building everything themselves. Also public institutions rarely get top talent and instead do everything via contractors, do you really think it is cheaper to get government contractors to build and run infrastructure instead of letting Microsoft do it?

ptd 7y ago

One point(though I will admit this is the cynical part of my brain speaking) immediately jumps to mind. If the server goes down, you can blame Microsoft. Even the least technical person can’t blame you for that. If you create your own data center however, if it goes down, you may potentially be on the chopping block if service is interrupted. Self preservation is a strong motivator, perhaps the strongest in a business environment.

adev_ 7y ago
> Self preservation is a strong motivator, perhaps the strongest in a business environment. And it's the most bullshit one. Outsourcing the risk does not mean avoiding it, it means however putting it out of control. This is a typical manager bullshit attitude "nobody got fired to buy IBM" that generally lead to adopt unadapted, bloated, overpriced solutions to trivial problems. Just because they do not have the balls to do things properly. Running out of its responsibilities should be a criteria to get fired when things go badly wrong. The outsource of the 737 MAX MCAS code to India today is a perfect example of that
- scarface74 7y ago
  Yes, surprisingly enough, people look out for their own self interests. What makes outsourcing hardware different from the dozens of other software as a service vendors that most companies depend on? Why spend the time developing competency in managing servers if that doesn’t give you a competitive advantage. But, going with the biggest most stable vendor is usually good. If you bought IBM hardware in the 70s you can still buy new hardware that supports your software. If you went with their competitors - not so much.
  - adev_ 7y ago
    > But, going with the biggest most stable vendor is usually good. If you bought IBM hardware in the 70s you can still buy new hardware that supports your software. If you went with their competitors - not so much. For 20 times the cost of commodities x86 that you should have bought by doing the right thing. And this 20 times the cost will very likely also please your own competitors because you will yourself be less competitive. That's how you finish with entire airline industry or bank still running on COBOL with no possibility of migration. IBM itself is very happy about it however.
    - scarface74 7y ago
      How much will it cost to rewrite everything? What were they suppose to choose in the 70s? Do you think that all of the people still using IBM and running legacy code are dumb or just maybe they did a cost benefit analysis and decided they didn’t need to rewrite everything in Node and React?
      - adev_ 7y ago
        Maybe you should ask why some sector that "fear" failures more than anything else (banks, aviations) are still trapped into these systems while everyone else is not. This is exactly related to what I was saying before.
- lotsofpulp 7y ago
  Bullshit means something is not true. People choosing to go with a vendor to avoid the risk of being blamed for failure, thereby reducing their chances of being fired, is a true phenomenon. And one that works. So how is it bullshit?
- vsareto 7y ago
  >that generally lead to adopt unadapted, bloated, overpriced solutions to trivial problems. There's almost no meaningful business downside for choosing poorly. Most businesses just don't care that someone else thinks they didn't use the right tool for the job. Should it be that way? Definitely not. But it won't change unless there's consequences (and when that happens, those products will die off quickly)
- jtwaleson 7y ago
  Agreed that outsourcing risk does not free you from the responsibilities. However, for many many services you have to rely on others to provide the service better than you could do yourself, as you are limited in time, money and other resources. For running production grade databases at startups, it's much more cost-efficient to run on AWS RDS than to hire a systems engineering team with 24/7 standby. For more trivial systems you might be right and managers might be cowards. However, you have to realize it's a spectrum. Whether you're running it yourself or you outsource it, things will go wrong at some time. When this happens, and you run it yourself, you also have to explain why it went wrong in a post-mortem or RCA. The problem is that your customers have no point of reference for your explanation (how likely is it that this occurs again?). In my experience, the following message goes down a lot smoother: "This is an Azure/AWS/GCP outage which affected not just us, but 1000s of other companies. We rely on {provider} to continuously learn from their mistakes and improve their service and they've shown this in the past. Here's their post-mortem report about this outage.". Note that not all cloud providers fit this bill.
  - adev_ 7y ago
    > However, for many many services you have to rely on others to provide the service better than you could do yourself, as you are limited in time, money and other resources. I do not criticized the fact of outsourcing when an outsourced service does a better job. This is normal and should be like that. What I am criticizing however, is outsourcing even when local/home made/OSS solutions are a better and cheaper fit even considering SLA.... just to avoid responsibilities. This is in my experience common, especially if the management has no technical background, no trust in his team and fear its upper layer.
- coredog64 7y ago
  Boeing didn’t outsource MCAS to India. The outsourcing contract was for display software.
- la6470 7y ago
  You just lost trust

jeswin 7y ago

Active Directory, Domain Services, Exchange Server and SharePoint Servers (which can use integrated Windows auth). Also many internal Enterprise apps are integrated with AD for auth and permissions.

luckydata 7y ago

It would be a tremendous hassle for little to no payoff as they probably don’t have the skill set to actually run a data center, and probably their size is actually not that large, they just run a lot of software I suppose (that’s usually the case in the public administration). It would be rampant malpractice to open your own data center for something like that.

deleted 7y ago
[deleted]

fulafel 7y ago

An potential problem is that now half of your IT recruits go to reinventing this stuff instead of working on domain problems. There aren't that many it staff per 10k municipal employees. (Of course this is assuming Azure specific hassles take much less staff time than running your own infra, not a given...)

nprateem 7y ago

Haha. Here's another one: "At this scale, why not write your own OS, since you already have a lot of servers to maintain?"

corporateslave5 7y ago
Haha!

sgt101 7y ago

At this scale why not contact all your neighbouring or peer municipalities and go for a joint cloud migration?

JohannesH 7y ago
As someone who worked in the very same municipality... one word; bureaucracy. More specific reasons were. Legal boundaries between ownership and taxpayer money separation. Of cause GDPR is also a big obstacle in joining forces on this scale. I worked on a project which tried to make a joined venture between several municipalities. I'm not sure if that partnership has ended yet, but overall you could only progress if there were people on both sides collaborating with a "better ask forgiveness, than permission" mindset. As soon as the municipality lawyers or their security officers got involved, all progress stopped immediately.
- scarface74 7y ago
  Or not dealing with infrastructure gatekeepers.

moksly 7y ago

Our setup is mixed, all our own servers are virtual on rented space at a local server rental shop. I’m not a hardware tech, so I’m afraid that’s as technical I can get. So our SQL cluster, and most of our web applications run on our own iron. Anything external as well as a lot of managed services run in Azure where the added security and operations dev-tools are invaluable compared to what we had when we self-hosted. We do make a conscious choice about everything that moves to the cloud. We’ve spent a lot of time figuring out if buying new rental space for our SQL cluster would be cheaper than moving it to Azure. With the current political climate, we’re also a little more hesitant about moving things to clouds operated by American companies, because there is risk attached, in that we may have to move it back rather suddenly. Not something we expect will ever happen, but we don’t like risk in the public sector. So my post wasn’t so much a “we run everything in Azure” as it was “well Azure is the obvious choice for the things we do operate in the cloud”. The relationship Microsoft has with enterprise, and the expertise staff has with their products means they become the best choice for a lot of enterprise. Unless Microsoft solutions are radically more expensive than their competitors, they simply have an advantage by already being a big, and typically well liked, player in most large organisation.

chairmanwow1 7y ago

I say this as someone who used to firmly agree with this position, but in the last year my mind has been changed without reservation. The time where this was a good idea has genuinely passed. There is no earthly way that any reasonable organization will be able to provide even the security that is present by relying on a large-scale cloud provider. There is no way they will come out on top money-wise either. The big cloud players have a _absurd_ number of servers that _teams_ of some of the best CA talent the globe has to offer stressing to improve KPIs for. 10k employees in a municipality is small potatoes to what azure / AWS / GCP have dedicated to their products.

dx034 7y ago
Maybe not own dc, but colocation could still make sense? You don't need to be more efficient than Azure & Co. They have pretty solid profit margins, even being 30% less efficient should still be cheaper for you. And beyond a few thousand servers, I'm not even sure if scale matters that much (for server virtualization only).
vidarh 7y ago
I have literally never seen a setup where cloud came out less than ~twice the cost of dedicated hosting. I have seen instances where dedicated hosting comes out cheaper than putting your own equipment in a colo, but even that depends on being in a location where energy and property prices (and so colo rental costs) are high. And this does factor in devops - when doing consulting I earned consistently more (because of more hours) from clients that went for cloud setups; often they'd end up spending more times solving problems that generally didn't exist in the first place in a dedicated setup. I do see lots of people that keep assuming that the cloud players must be cheap because they're so big, but I just have never seen that bear out in practice - in part because of that attitude, the margins they can charge are far higher. Cloud providers are great for ease and for the number of services they provide, but they are generally an expensive step up.
- dcolebatch 7y ago
  I suspect you’re doing it wrong, or there’s some niche type of computing you specialize in. In general purpose IT, looking across an enterprise portfolio of applications, we consistently see customers of Tidal Migrations replatform their applications to cloud and save 95+% in OpEx vs dedicated on-premise hosting. IMO, The first step to realizing those cost benefits is recognizing that the cloud is not your datacenter and you need to architect differently. Yes, cloud spend can grow as you open up access to more developers, but that’s why we have a plethora of tools and governance people to help make that manageable. I believe the business benefit of the agility gains that come from instant and decentralized resource provisioning will always trump any cloud bill... especially if you’re in a competitive industry & don’t want to get left behind.

christophilus 7y ago

My experience with home-rolled solutions suggests there are plenty of hidden costs. Got a new project, and want to spin up some experimental servers? Gotta wade through a bunch of IT guys. Documentation and training around Azure, etc is also going to be better than homegrown alternatives, most likely. Things like that. It’s hard to put a monetary figure on the frictions costs, but they are there.

sergiosgc 7y ago
You're comparing one extreme (public cloud) to another extreme (sysadmin-managed infrastructure). The alternative, today, is either a private or a hybrid cloud, which have all the advantages of short-circuiting sysadmins that public clouds have.

username90 7y ago

Public institutions usually have a lot of systems but not a lot of traffic (if we compare to tech companies), so it makes sense for them to put things in the cloud instead of building everything themselves. Also public institutions rarely get top talent and instead do everything via contractors, do you really think it is cheaper to get government contractors to build and run infrastructure instead of letting Microsoft do it?