5 ms·

I like DevOps, I don't like being on call 24x7 so I tell my clients I am a software developer not an ops. DevOps was sold as a way to avoid dev and ops teams bl

I like DevOps, I don't like being on call 24x7 so I tell my clients I am a software developer not an ops. DevOps was sold as a way to avoid dev and ops teams blaming each other for not shipping software. It was at the time true that devs did not understand hardware and networking and ops did not particularly understand software. We got nice things like Docker and k8s, but the core issues have not gone away, devs don't understand hardware and networking and ops are an almost extinct species. We are re-learning the hard way that certain performance problems are not solved by throwing more hardware at them and that devs will always write Helm charts for convenience and not security. We are also learning that the complexity of networking and backed designs has not gone away we just got more modern tools to describe it. One other unintended negative consequence of misunderstanding DevOps is the proliferation of "full-stack engineers", which is basically an ops, a DBA, a secops, a backend dev, a frontend dev, and a website designer rolled into one. What we have today is a situation pretty much the same as it was before the arrival of DevOps... devs thinking ops are useless, ops thinking dev are no good and ... both thinking the others have no sense of style.

dspillett 2y ago
> ops are an almost extinct species. This is infuriating at times. We ingest a lot of data from 3rd party systems of our clients for things like KPI analysis, the devs¹ provided various APIs for submitting data and we can read from various others, but good old reliable dumping-CSV-files-to-us-via-SFTP is still what most clients want to do. I have no problem with this², BUT the amount of times I end up talking to clients (sometimes large banking organisations) that don't have anyone available who actually understands SFTP/SSH amazes me. We often resort to password based auth because they simply don't get private/public keys (or claim their other infrastructure just doesn't work with them). ---- [1] used to be a full-stacker myself, but I've let large parts of my skillset rot so except for when working on legacy issues I'm a more specialised database/data-processing/infrastructure fellow these days [2] other than, of course, dealing with bad CSV output implementations⁴ or just all the combinations of how to encode problems like quotes, eols in long stings, and so forth. JSON³ is starting to get some traction as an alternative which is handy because being more modern it is more rigidly designed compared to the mess that CSV is due to decades of implementations being slapped together in an it'll-do-until-it-doesn't manner. [3] Don't expect structured data though: the clients & 3rd parties I deal with are sending the same simple table format data, with multiple files for us to join together later when there is more complex structure, that they would send as CSV, just using the more bloated format⁵ for the gain in reduced encoding confusions. [4] not quoting values containing the separator character or otherwise escaping it, not dealing with existing quotes when values do get quoted, etc. - if you'd be surprised to find CSV is far from a solved problem today, then prepare to be very surprised! [5] Don't get me wrong when I mention the format bloat: this _is_ a good trade for easing the other issues. Bandwidth & storage (or compute resource for compression) are more than cheap & fast enough to make it so.
- fifilura 2y ago
  Would it be possible for you to take a step back and describe the problem you are infuriated by again?
  - dspillett 2y ago
    Sitting on the phone with people who's job it is to know these things for their organisation, having to explain to them that no, they shouldn't send me the private key, not even (especially not even) if they send the passphrase too, and many other problems. There seems to be a dearth of people for the ops jobs that need doing, hence OP's “ops are an almost extinct species”, so the tasks are thrown around and have to be done by the first person who doesn't duck fast enough. Those of us on the other side of the company interaction boundary end up having to help them with what should be the basics. I'm not their infrastructure team, I don't have access to be able to answer some of the questions they ask about their own infrastructure. I shouldn't even be client facing never mind teaching them how to do their jobs. The infuriating part (perhaps “frightening” is a better word here) is that there are people managing the secure transmission of data (sometimes sensitive personal data) who do not understand what is going on.
    - Corrado 2y ago
      Tell me about it. I'm in the healthcare industry and hospital systems are way too complex and/or they don't have the proper people to manage them. Getting them to understand SFTP is hard enough, let alone public key encryption. Additionally, their systems are often so old and out of date that they don't support anything newer than SFTP. On top of that the regulations they have to follow are just overwhelming and completely out of date. Honestly, I kinda feel sorry for the poor DevOps people in hospitals. It's got to be torture.
    - fifilura 2y ago
      Thank you for taking the time to reply. FWIW my comment was added a few minutes before your edits so it only contained the [1] footnote and possibly other edits. I guess I was missing the "why" where your post focused on the "what". Of course when you are working with sensitive data like that it makes a lot of sense to be really careful about security. I am well aware about the problems with the csv format. But a lot of times you can not really expect clients to be able to download and decode a parquet file, when all they want is to submit a password, download the data and look at it in excel.
threatofrain 2y ago
DevOps is dying? As in the job market for DevOps is actually shrinking?
- DarkNova6 2y ago
  I'd rather think that the need for Ops is growing much faster than there is supply. It is a primarily knowledge-intense profession where you can't just hire some untrained professionals and let them hack together an app. Or cannibaliize data-analystis and make them build some backend.
  - threatofrain 2y ago
    That implies the reverse! That DevOps is hotter than ever.
    - surfingdino 2y ago
      No. Ops is back. DevOps has just not worked in practice over the last decade as clients had let go of ops and dumped all work onto devs. We will be seeing an increase in demand for ops + cloud + k8s skills, which is the right direction to go into. We will also be seeing a significant move away from the cloud and you need old-school ops skills for that. Not necessarily for deploying, but definitely for configuring server and networks and debugging performance issues.
      - DarkNova6 2y ago
        Wait. Not every company needs to work like Google Or Netflix? (sorry, couldn't resist). Ops-knowledge is so much in demand that anybody with an actual "DevOps" role has done Ops nearly exclusively. In all seriousness, this is what I am seeing already. Often I see a platform team providing an Openshift for developers to rule as they want. There is cooperation on some levels and a technical architect who is setting standards. And most companies don't want to pay the price tag that comes with an actually good DevOps guy anyway.

dspillett 2y ago

> ops are an almost extinct species. This is infuriating at times. We ingest a lot of data from 3rd party systems of our clients for things like KPI analysis, the devs¹ provided various APIs for submitting data and we can read from various others, but good old reliable dumping-CSV-files-to-us-via-SFTP is still what most clients want to do. I have no problem with this², BUT the amount of times I end up talking to clients (sometimes large banking organisations) that don't have anyone available who actually understands SFTP/SSH amazes me. We often resort to password based auth because they simply don't get private/public keys (or claim their other infrastructure just doesn't work with them). ---- [1] used to be a full-stacker myself, but I've let large parts of my skillset rot so except for when working on legacy issues I'm a more specialised database/data-processing/infrastructure fellow these days [2] other than, of course, dealing with bad CSV output implementations⁴ or just all the combinations of how to encode problems like quotes, eols in long stings, and so forth. JSON³ is starting to get some traction as an alternative which is handy because being more modern it is more rigidly designed compared to the mess that CSV is due to decades of implementations being slapped together in an it'll-do-until-it-doesn't manner. [3] Don't expect structured data though: the clients & 3rd parties I deal with are sending the same simple table format data, with multiple files for us to join together later when there is more complex structure, that they would send as CSV, just using the more bloated format⁵ for the gain in reduced encoding confusions. [4] not quoting values containing the separator character or otherwise escaping it, not dealing with existing quotes when values do get quoted, etc. - if you'd be surprised to find CSV is far from a solved problem today, then prepare to be very surprised! [5] Don't get me wrong when I mention the format bloat: this _is_ a good trade for easing the other issues. Bandwidth & storage (or compute resource for compression) are more than cheap & fast enough to make it so.

fifilura 2y ago
Would it be possible for you to take a step back and describe the problem you are infuriated by again?
- dspillett 2y ago
  Sitting on the phone with people who's job it is to know these things for their organisation, having to explain to them that no, they shouldn't send me the private key, not even (especially not even) if they send the passphrase too, and many other problems. There seems to be a dearth of people for the ops jobs that need doing, hence OP's “ops are an almost extinct species”, so the tasks are thrown around and have to be done by the first person who doesn't duck fast enough. Those of us on the other side of the company interaction boundary end up having to help them with what should be the basics. I'm not their infrastructure team, I don't have access to be able to answer some of the questions they ask about their own infrastructure. I shouldn't even be client facing never mind teaching them how to do their jobs. The infuriating part (perhaps “frightening” is a better word here) is that there are people managing the secure transmission of data (sometimes sensitive personal data) who do not understand what is going on.
  - Corrado 2y ago
    Tell me about it. I'm in the healthcare industry and hospital systems are way too complex and/or they don't have the proper people to manage them. Getting them to understand SFTP is hard enough, let alone public key encryption. Additionally, their systems are often so old and out of date that they don't support anything newer than SFTP. On top of that the regulations they have to follow are just overwhelming and completely out of date. Honestly, I kinda feel sorry for the poor DevOps people in hospitals. It's got to be torture.
  - fifilura 2y ago
    Thank you for taking the time to reply. FWIW my comment was added a few minutes before your edits so it only contained the [1] footnote and possibly other edits. I guess I was missing the "why" where your post focused on the "what". Of course when you are working with sensitive data like that it makes a lot of sense to be really careful about security. I am well aware about the problems with the csv format. But a lot of times you can not really expect clients to be able to download and decode a parquet file, when all they want is to submit a password, download the data and look at it in excel.

threatofrain 2y ago

DevOps is dying? As in the job market for DevOps is actually shrinking?

DarkNova6 2y ago
I'd rather think that the need for Ops is growing much faster than there is supply. It is a primarily knowledge-intense profession where you can't just hire some untrained professionals and let them hack together an app. Or cannibaliize data-analystis and make them build some backend.
- threatofrain 2y ago
  That implies the reverse! That DevOps is hotter than ever.
  - surfingdino 2y ago
    No. Ops is back. DevOps has just not worked in practice over the last decade as clients had let go of ops and dumped all work onto devs. We will be seeing an increase in demand for ops + cloud + k8s skills, which is the right direction to go into. We will also be seeing a significant move away from the cloud and you need old-school ops skills for that. Not necessarily for deploying, but definitely for configuring server and networks and debugging performance issues.
    - DarkNova6 2y ago
      Wait. Not every company needs to work like Google Or Netflix? (sorry, couldn't resist). Ops-knowledge is so much in demand that anybody with an actual "DevOps" role has done Ops nearly exclusively. In all seriousness, this is what I am seeing already. Often I see a platform team providing an Openshift for developers to rule as they want. There is cooperation on some levels and a technical architect who is setting standards. And most companies don't want to pay the price tag that comes with an actually good DevOps guy anyway.