4 ms·

Today I asked a devops engineer to tell me how much time a long (3 seconds avg) api call was spending on database queries, application logic, and network etc. H

Today I asked a devops engineer to tell me how much time a long (3 seconds avg) api call was spending on database queries, application logic, and network etc. He couldn’t understand the request and instead opened up the azure console and recommended we increase the number cpu cores / memory if performance is an issue. I look at posts like this and cry.

tomrod 3y ago
Your DevOps engineer needs to learn more about observability. Jaeger or similar could be helpful. Also, with DevOps pushing out traditional administrators, companies are often spending way more on infra than needed.
jiggawatts 3y ago
I keep telling people that a hundred buses will let you take a hundred times more people, but nobody will get to their destination a hundred times faster. Inevitably this comment is followed by quiet blinking as they digest this and then this question: “Are you saying we need to scale up one hundred times bigger?” Sigh…
- QuercusMax 3y ago
  Really makes one wonder what kind of thought process these people go through, and what kind of education they actually had.
  - teddyh 3y ago
    On two occasions I have been asked, — “Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?” In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. — Charles Babbage, Passages from the Life of a Philosopher (1864), chapter 5, Difference Engine No. 1
turtlebits 3y ago
Implementing traces should be done by devs, not the infrastructure team. Devops should implement/support the platform that supports traces.
- jiggawatts 3y ago
  In Azure, use Application Insights for this. It's easy to set up and shows distributed performance traces across multiple services in a GUI.
- latency-guy2 3y ago
  Disagree, DevOps teams should be looking for resources that are being hit hard unnecessarily and request moves to better solutions when possible. DevOps teams should be looking at CPU spikes, and should be performing RCAs, and they should be maintaining resources in a healthy state, and they should reject/revert changes and notify problem areas in code by product focused devs. Product devs, for the most part, are only implementing human lex traces to debug business logic when it arises. Product devs are not equipped with the knowledge to identify system errors that are not "bugs in the code", i.e. they will not be good at telling you why SPROC_LXC1 fails as a result of making a ExcelParserFactoryFactory
RetroTechie 3y ago
> Today I asked a devops engineer (..) He couldn’t understand the request "Engineer"?!? Given above description, that makes me cry.

tomrod 3y ago

Your DevOps engineer needs to learn more about observability. Jaeger or similar could be helpful. Also, with DevOps pushing out traditional administrators, companies are often spending way more on infra than needed.

jiggawatts 3y ago

I keep telling people that a hundred buses will let you take a hundred times more people, but nobody will get to their destination a hundred times faster. Inevitably this comment is followed by quiet blinking as they digest this and then this question: “Are you saying we need to scale up one hundred times bigger?” Sigh…

QuercusMax 3y ago
Really makes one wonder what kind of thought process these people go through, and what kind of education they actually had.
- teddyh 3y ago
  On two occasions I have been asked, — “Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?” In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. — Charles Babbage, Passages from the Life of a Philosopher (1864), chapter 5, Difference Engine No. 1

turtlebits 3y ago

Implementing traces should be done by devs, not the infrastructure team. Devops should implement/support the platform that supports traces.

jiggawatts 3y ago
In Azure, use Application Insights for this. It's easy to set up and shows distributed performance traces across multiple services in a GUI.
latency-guy2 3y ago
Disagree, DevOps teams should be looking for resources that are being hit hard unnecessarily and request moves to better solutions when possible. DevOps teams should be looking at CPU spikes, and should be performing RCAs, and they should be maintaining resources in a healthy state, and they should reject/revert changes and notify problem areas in code by product focused devs. Product devs, for the most part, are only implementing human lex traces to debug business logic when it arises. Product devs are not equipped with the knowledge to identify system errors that are not "bugs in the code", i.e. they will not be good at telling you why SPROC_LXC1 fails as a result of making a ExcelParserFactoryFactory

RetroTechie 3y ago

> Today I asked a devops engineer (..) He couldn’t understand the request "Engineer"?!? Given above description, that makes me cry.