5 ms·

> It’s frustrating to hear the statements “you must use our infrastructure” and “you can’t do that with our infrastructure.” You waste time either working aroun

> It’s frustrating to hear the statements “you must use our infrastructure” and “you can’t do that with our infrastructure.” You waste time either working around the infrastructure or sitting in meetings where you attempt to convince the owners of the infrastructure to meet your needs. Listen, I'm an infra guy, so maybe this hit a nerve, but 'othering' infra because it's inconvenient has got to stop. Infra serves many masters and you are probably the most flexible of them -- that's why there's so much push back.

anotherhue 3y ago
And I will nuke your off-books deployments from orbit.
- comprev 3y ago
  I'm involved in implementing some big changes where I work and one of them is CI runner-only deployments. Both infrastructure and applications will follow the same workflow - all changes via pull/merge requests, Jira integration, etc. Developers won't have the ability to create resources outside of the CI platform.
  - candiddevmike 3y ago
    I really hope you sold the devs on this instead of commanding them to use it, otherwise their director will use his credit card to sign up for their own AWS account.
    - MattPalmer1086 3y ago
      And that director would be fired as soon as this was discovered. At least where I work, but we are heavily regulated.
      - comprev 3y ago
        This would likely be the same outcome. We're implementing changes because an outside entity states we require them.
    - comprev 3y ago
      It's a regulated environment which needed locking down. The developers still have access to the same resources albeit in a more controlled process. It's not like we flicked a switch overnight - that would be insane - it's a large scale cloud migration project over many years.
  - anotherhue 3y ago
    I've seen many cases where someone signs up for a free trial at PaaS-du-jour and then starts pointing prod HTTP traffic at mycorp.magicpaas.com
    - comprev 3y ago
      I've also seen it plenty of times in the startup world. Unfortunately I've also been involved in the emergency bandage fix once it was discovered...
- Silhouette 3y ago
  Through the forcefield of a VP two levels above your boss who authorised it? Even if you know about it - which is a big "if" - you probably won't have either the authority or the practical means to shut it down. It's become a classic problem. Organisations want to centralise infrastructure and services because consolidation saves on costs and increases visibility and control. Particularly if you're under specific regulations that can make things much easier. But that only happens if the centralised provision actually works. It has to be adequately resourced and reasonably well managed and supportive of the people using the infrastructure or services it provides. If the centralised provision is not good then it had one job and failed to do it. Whose fault it is doesn't really matter. It's not supporting the other parts of the organisation to do their jobs and inevitably a point will come where that infra gets "othered". We've been seeing this phenomenon ever since we had central system administrators who could lock down the OS on individual employees' workstations. The admins argue that they need to make sure security updates are properly deployed and they need to prevent non-expert users from breaking things and they need to ensure consistency because they're also running the help desk and ultimately they're responsible for these things so they need the authority to control them. All very fair and reasonable. And all completely irrelevant to someone with actual revenue-generating work to do who is being blocked for days or weeks because some sysadmin whose opinions are not necessarily justified by their effectiveness hasn't enabled or authorised some essential facility.
- jayd16 3y ago
  Seems pretty silly. Why do they have permissions if it's not allowed?
candiddevmike 3y ago
IMO... infra teams need to realize that folks "othering" their infrastructure means the infra team isn't meeting their needs. Infra teams are now competing against DIY and PaaS solutions, especially all of the fly by night YC startups. Infra teams need to prove their worth or get out of the way and let the devs go buy a solution that replaces the infra team. The business cares about time to market, not the perfect little homegrown PaaS the infra team built. For the record, I'm an infrastructure person.
- anotherhue 3y ago
  100% this. If your company can run on a managed platform then it should. The reasons the whole world isn't already include: GDPR, SOC2 (audit, role level responsibilities), customer firewalls, encryption requirements (hello FIPS).
  - echelon 3y ago
    Managed platforms are obscenely expensive. I know, I know - "time to market", "scaling on demand", "open source isn't free", "not our core competency", etc. Infra staffing, hardware, scaling, and upgrades are expensive. But it doesn't help the fact that cloud costs an arm and a leg too. I can't believe we haven't seen more of a race to the bottom in terms of costs and offerings. Amazon is gorging itself on your margin. I wish we lived in a world with 20 AWSes, all at the same scale and market penetration.
    - MobileVet 3y ago
      I don't agree. People are far more expensive than managed infrastructure and that is what the comparison should be... not against 'bare metal.' When you compare the managed infra costs to bare metal directly, you are comparing apples to oranges. We run an 8 figure business on mid 5 figure managed infra costs. An infrastructure team would be several multiples more expensive than that... even if it was a single individual. Just because you can do it yourself or more cheaply, doesn't mean the effective business costs are less. Edited: for grammar

anotherhue 3y ago

And I will nuke your off-books deployments from orbit.

comprev 3y ago
I'm involved in implementing some big changes where I work and one of them is CI runner-only deployments. Both infrastructure and applications will follow the same workflow - all changes via pull/merge requests, Jira integration, etc. Developers won't have the ability to create resources outside of the CI platform.
- candiddevmike 3y ago
  I really hope you sold the devs on this instead of commanding them to use it, otherwise their director will use his credit card to sign up for their own AWS account.
  - MattPalmer1086 3y ago
    And that director would be fired as soon as this was discovered. At least where I work, but we are heavily regulated.
    - comprev 3y ago
      This would likely be the same outcome. We're implementing changes because an outside entity states we require them.
  - comprev 3y ago
    It's a regulated environment which needed locking down. The developers still have access to the same resources albeit in a more controlled process. It's not like we flicked a switch overnight - that would be insane - it's a large scale cloud migration project over many years.
- anotherhue 3y ago
  I've seen many cases where someone signs up for a free trial at PaaS-du-jour and then starts pointing prod HTTP traffic at mycorp.magicpaas.com
  - comprev 3y ago
    I've also seen it plenty of times in the startup world. Unfortunately I've also been involved in the emergency bandage fix once it was discovered...
Silhouette 3y ago
Through the forcefield of a VP two levels above your boss who authorised it? Even if you know about it - which is a big "if" - you probably won't have either the authority or the practical means to shut it down. It's become a classic problem. Organisations want to centralise infrastructure and services because consolidation saves on costs and increases visibility and control. Particularly if you're under specific regulations that can make things much easier. But that only happens if the centralised provision actually works. It has to be adequately resourced and reasonably well managed and supportive of the people using the infrastructure or services it provides. If the centralised provision is not good then it had one job and failed to do it. Whose fault it is doesn't really matter. It's not supporting the other parts of the organisation to do their jobs and inevitably a point will come where that infra gets "othered". We've been seeing this phenomenon ever since we had central system administrators who could lock down the OS on individual employees' workstations. The admins argue that they need to make sure security updates are properly deployed and they need to prevent non-expert users from breaking things and they need to ensure consistency because they're also running the help desk and ultimately they're responsible for these things so they need the authority to control them. All very fair and reasonable. And all completely irrelevant to someone with actual revenue-generating work to do who is being blocked for days or weeks because some sysadmin whose opinions are not necessarily justified by their effectiveness hasn't enabled or authorised some essential facility.
jayd16 3y ago
Seems pretty silly. Why do they have permissions if it's not allowed?

candiddevmike 3y ago

IMO... infra teams need to realize that folks "othering" their infrastructure means the infra team isn't meeting their needs. Infra teams are now competing against DIY and PaaS solutions, especially all of the fly by night YC startups. Infra teams need to prove their worth or get out of the way and let the devs go buy a solution that replaces the infra team. The business cares about time to market, not the perfect little homegrown PaaS the infra team built. For the record, I'm an infrastructure person.

anotherhue 3y ago
100% this. If your company can run on a managed platform then it should. The reasons the whole world isn't already include: GDPR, SOC2 (audit, role level responsibilities), customer firewalls, encryption requirements (hello FIPS).
- echelon 3y ago
  Managed platforms are obscenely expensive. I know, I know - "time to market", "scaling on demand", "open source isn't free", "not our core competency", etc. Infra staffing, hardware, scaling, and upgrades are expensive. But it doesn't help the fact that cloud costs an arm and a leg too. I can't believe we haven't seen more of a race to the bottom in terms of costs and offerings. Amazon is gorging itself on your margin. I wish we lived in a world with 20 AWSes, all at the same scale and market penetration.
  - MobileVet 3y ago
    I don't agree. People are far more expensive than managed infrastructure and that is what the comparison should be... not against 'bare metal.' When you compare the managed infra costs to bare metal directly, you are comparing apples to oranges. We run an 8 figure business on mid 5 figure managed infra costs. An infrastructure team would be several multiples more expensive than that... even if it was a single individual. Just because you can do it yourself or more cheaply, doesn't mean the effective business costs are less. Edited: for grammar