5 ms·

These types of situations always wind up being the most fun I have at work and leave me feeling most fulfilled. I wish I could do that all day instead of going

by paulz_ 6y ago

These types of situations always wind up being the most fun I have at work and leave me feeling most fulfilled. I wish I could do that all day instead of going to standups.

4e530344963049 6y ago
A "firefighting" consultancy might be fun, no? And you would give people a lot of time off!
- paulz_ 6y ago
  I've thought about that before. You get the call. "It's 30 minutes away. I'll be there in 10" If anyone happens to know of a career path or company that does that sort of work I would be interested to hear about it. Bonus points if the pay is half decent.
  - munchbunny 6y ago
    Blue teaming in big company cyber security teams will get that for you. Not everything is a true positive but it’s always urgent. Pay is decent too.
    - meowface 6y ago
      I've worked such a job at a large enterprise. It really does feel like firefighting (minus all the smoke inhalation and physical strain and death risk). However, not only is not everything a true positive, probably only about 0.001% of things are true positives, among a sea of alerts and reports and dashboards across myriad systems. Some coming from your SIEM, some generated by security appliances and products, some from internal employee reports. An ideal place will have people who continuously work on trying to reduce alert fatigue and false positive noise - but, in practice, at most big companies it's probably like working at a fire station and getting hundreds of dispatch calls per hour, every hour, every day, each about a potential fire at a different residence. And then you drive up and see they just used the stove for a few minutes or a character said the word "fire" in a TV show they were watching. But you have to urgently show up every time no matter what because, occasionally, the house actually is engulfed in flames and might be on the verge of igniting the whole town.
    - htrp 6y ago
      I spoke with a CISO of a large F500, his biggest gripe was that he has a team of 30 that can barely keep their heads above water, let alone respond to incidents
  - insomniacity 6y ago
    I know Mandiant do this (disaster response) for security incidents. Don't know of a generalised service, but I imagine the big consultancies offer it.
  - clipradiowallet 6y ago
    I don't know of a company that does this sort of work, but I know of some technologies that experts in receive these types of calls. The one that comes to mind is an ERP-focused database system. It's called "Progress" by "OpenEdge". IMHO, it's awful, but this has no hindered adoption in the slightest. I wrote Progress/4gl (their query language) often enough in a prior position to have it on my resume. Every 2 or 3 months, I'll get an email/call, asking if I could be available for short-term contracting upwards of $200/hr for Progress emergencies. I have declined all of these, because I found it soul crushing to work with in the past. However, if you could enjoy that sort of thing, that's one example of a very lucrative field to dabble in.
    - yellowapple 6y ago
      I've gotten a few pings along similar lines for my HighJump experience (warehouse management system, for those not acquainted). And much like OpenEdge, it's pretty... rough around the edges (at its core it's basically a runtime for a VM that's programmed with a "language" (if you can call it that) driven entirely by conditional GOTOs developed entirely in a half-baked SQL-backed IDE called "Architect"; this is paired with a DB schema from hell for all data storage, and it's filled to the brim with sprocs because even fucking T-SQL is more ergonomic than anything doable in Architect). And yet, it was actually kind of fun (in the twisted, Dwarf-Fortress-esque sense of the word) to hack on that system and abuse the hell out of it. And for some reason warehouse managers seem to swear by it, so it still gets a decent number of new customers - meaning those customers need implementers. And since it's a giant pile of hacks, the average deployment needs a whole lot of customizations - so more implementation man-hours, and a steady stream of maintenance man-hours. Thus, I get pinged every once in awhile for some long-term contract. Too bad they didn't ping me when I was actually looking for work last year, else I probably would've accepted one for the hell of it. Still tempted to; would be an interesting side job, albeit probably soul crushing.
- easton 6y ago
  This is my ideal job, I think. Charge a big rate to be the consultant you call when everything dies from ransomware or OPs problem or whatever, I calmly fix it to normal standards (as possible), take three weeks off after.
- notyourday 6y ago
  > A "firefighting" consultancy might be fun, no? It is... interesting. I have been doing this over last 4 years. The biggest problem, as I pointed out before, is the disconnect between what companies claim they would do to fix the problems vs. what they would actually do. So ~90% of selling is repeatedly explaining the same things to different people whose jobs are in the eminent danger of being restructured/deprioritized/eliminated if your services are successfully engaged.
  - beckingz 6y ago
    hopefully you can skip most of those people and talk to the people who are in danger of being eliminated if your services are NOT engaged?
    - notyourday 6y ago
      CEO: We are doing project X. We have engaged so and so to do it. You are to support this. <multiple layers down, Jack, head of IT in charge of backups>: I'm uncomfortable sharing this information with the outside party. I will need to get the approvals for <blah blah blah blah> ... CEO: Yes, tell them to do this. ... Jack, head of IT in charge of backups: Ok, you can have this access. Consultant: Great. <comes back ten minutes later> It says I'm not authorized to perform this operation. It is a blocker, could this be fixed so I could continue? Jack: This was not in the scope of authorization that I have received. I'm uncomfortable providing this level of access without additional authorization. <repeat>
      - digi59404 6y ago
        This happens, and honestly it happens rarely. When you’re in this role your goal and job is to build relationships and help people. That includes Jack. The job is getting done with or without Jacks help. So he can either jump onboard or he can be removed and someone else can be added to give access. In the end people like Jack benefit nothing from being gatekeepers. Because the third time I have to ask for permissions to be changed - I’m asking for Jack to be removed from the task and someone else to be put in place.
- pratik661 6y ago
  I always wondered this: If you are an external consultant (with no prior knowledge of the company's systems/processes/hierarchies) brought in to "fight" a particular fire, how do quickly get ramped up to a level of knowledge where you can analyze the root cause of/recommend a fix for a particular "fire"?
  - Clubber 6y ago
    I would say the analysis is 90-99% of the work, based on my experience. Especially the more jacked up systems where cheapest devs found work on the codebase and it's a mess. 90% only comes in when it's so jacked up it requires a lot of refactoring.
Aeolun 6y ago
Come work for our company! We have an infinite amount of fires to put out. To be fair, the number of fires is finite, new ones are just getting started faster than they are put out.
- sanedigital 6y ago
  I believe that implies that the fires are indeed infinite, just countably so.
m463 6y ago
It would be fun just to be able to talk to management with the same sort of candor the "cleaner" in pulp fiction did. (harvey keitel)
mattlondon 6y ago
Constant "emergencies" soon wear you down. Sure it is fun, but the stakes can be high and the stress gets piled on. It starts with congratulations all-round. You are a hero! Good job! Nice save! Your boss's boss's boss's boss comes over and thanks you personally. Take the rest of the week off! Later, its people asking when will you fix it? Why haven't you got this back working yet? Didn'y you fix the same thing last month? (no) We really need you to fix this before 5pm or the TPS reports wont go out, and the management will be pissed. Failures become normalised. They get reliant on people doing heroics. People forget that the systems are crap and need investment, and start to rely on you being there to fix it, and if things don't get fixed then it is your fault the TPS reports didn't go out, not tech-debt/lack-of-investment/bad-design/whatever.
- ido 6y ago
  I remember stuff like that, once the COO called the entire company to the common room to present me a bottle of champagne for my efforts (another time I was given a fancy bottle of whiskey that some marketing guys drank 3/4 of before I ever got around to it). I rather not work any overtime instead.
papreclip 6y ago
Being an SRE is a little more of this and a little less scheduled work. You still have standup and plenty of other meetings, though

by paulz_ 6y ago

These types of situations always wind up being the most fun I have at work and leave me feeling most fulfilled. I wish I could do that all day instead of going to standups.

4e530344963049 6y ago

A "firefighting" consultancy might be fun, no? And you would give people a lot of time off!

paulz_ 6y ago
I've thought about that before. You get the call. "It's 30 minutes away. I'll be there in 10" If anyone happens to know of a career path or company that does that sort of work I would be interested to hear about it. Bonus points if the pay is half decent.
- munchbunny 6y ago
  Blue teaming in big company cyber security teams will get that for you. Not everything is a true positive but it’s always urgent. Pay is decent too.
  - meowface 6y ago
    I've worked such a job at a large enterprise. It really does feel like firefighting (minus all the smoke inhalation and physical strain and death risk). However, not only is not everything a true positive, probably only about 0.001% of things are true positives, among a sea of alerts and reports and dashboards across myriad systems. Some coming from your SIEM, some generated by security appliances and products, some from internal employee reports. An ideal place will have people who continuously work on trying to reduce alert fatigue and false positive noise - but, in practice, at most big companies it's probably like working at a fire station and getting hundreds of dispatch calls per hour, every hour, every day, each about a potential fire at a different residence. And then you drive up and see they just used the stove for a few minutes or a character said the word "fire" in a TV show they were watching. But you have to urgently show up every time no matter what because, occasionally, the house actually is engulfed in flames and might be on the verge of igniting the whole town.
  - htrp 6y ago
    I spoke with a CISO of a large F500, his biggest gripe was that he has a team of 30 that can barely keep their heads above water, let alone respond to incidents
- insomniacity 6y ago
  I know Mandiant do this (disaster response) for security incidents. Don't know of a generalised service, but I imagine the big consultancies offer it.
- clipradiowallet 6y ago
  I don't know of a company that does this sort of work, but I know of some technologies that experts in receive these types of calls. The one that comes to mind is an ERP-focused database system. It's called "Progress" by "OpenEdge". IMHO, it's awful, but this has no hindered adoption in the slightest. I wrote Progress/4gl (their query language) often enough in a prior position to have it on my resume. Every 2 or 3 months, I'll get an email/call, asking if I could be available for short-term contracting upwards of $200/hr for Progress emergencies. I have declined all of these, because I found it soul crushing to work with in the past. However, if you could enjoy that sort of thing, that's one example of a very lucrative field to dabble in.
  - yellowapple 6y ago
    I've gotten a few pings along similar lines for my HighJump experience (warehouse management system, for those not acquainted). And much like OpenEdge, it's pretty... rough around the edges (at its core it's basically a runtime for a VM that's programmed with a "language" (if you can call it that) driven entirely by conditional GOTOs developed entirely in a half-baked SQL-backed IDE called "Architect"; this is paired with a DB schema from hell for all data storage, and it's filled to the brim with sprocs because even fucking T-SQL is more ergonomic than anything doable in Architect). And yet, it was actually kind of fun (in the twisted, Dwarf-Fortress-esque sense of the word) to hack on that system and abuse the hell out of it. And for some reason warehouse managers seem to swear by it, so it still gets a decent number of new customers - meaning those customers need implementers. And since it's a giant pile of hacks, the average deployment needs a whole lot of customizations - so more implementation man-hours, and a steady stream of maintenance man-hours. Thus, I get pinged every once in awhile for some long-term contract. Too bad they didn't ping me when I was actually looking for work last year, else I probably would've accepted one for the hell of it. Still tempted to; would be an interesting side job, albeit probably soul crushing.
easton 6y ago
This is my ideal job, I think. Charge a big rate to be the consultant you call when everything dies from ransomware or OPs problem or whatever, I calmly fix it to normal standards (as possible), take three weeks off after.
notyourday 6y ago
> A "firefighting" consultancy might be fun, no? It is... interesting. I have been doing this over last 4 years. The biggest problem, as I pointed out before, is the disconnect between what companies claim they would do to fix the problems vs. what they would actually do. So ~90% of selling is repeatedly explaining the same things to different people whose jobs are in the eminent danger of being restructured/deprioritized/eliminated if your services are successfully engaged.
- beckingz 6y ago
  hopefully you can skip most of those people and talk to the people who are in danger of being eliminated if your services are NOT engaged?
  - notyourday 6y ago
    CEO: We are doing project X. We have engaged so and so to do it. You are to support this. <multiple layers down, Jack, head of IT in charge of backups>: I'm uncomfortable sharing this information with the outside party. I will need to get the approvals for <blah blah blah blah> ... CEO: Yes, tell them to do this. ... Jack, head of IT in charge of backups: Ok, you can have this access. Consultant: Great. <comes back ten minutes later> It says I'm not authorized to perform this operation. It is a blocker, could this be fixed so I could continue? Jack: This was not in the scope of authorization that I have received. I'm uncomfortable providing this level of access without additional authorization. <repeat>
    - digi59404 6y ago
      This happens, and honestly it happens rarely. When you’re in this role your goal and job is to build relationships and help people. That includes Jack. The job is getting done with or without Jacks help. So he can either jump onboard or he can be removed and someone else can be added to give access. In the end people like Jack benefit nothing from being gatekeepers. Because the third time I have to ask for permissions to be changed - I’m asking for Jack to be removed from the task and someone else to be put in place.
pratik661 6y ago
I always wondered this: If you are an external consultant (with no prior knowledge of the company's systems/processes/hierarchies) brought in to "fight" a particular fire, how do quickly get ramped up to a level of knowledge where you can analyze the root cause of/recommend a fix for a particular "fire"?
- Clubber 6y ago
  I would say the analysis is 90-99% of the work, based on my experience. Especially the more jacked up systems where cheapest devs found work on the codebase and it's a mess. 90% only comes in when it's so jacked up it requires a lot of refactoring.

Aeolun 6y ago

Come work for our company! We have an infinite amount of fires to put out. To be fair, the number of fires is finite, new ones are just getting started faster than they are put out.

sanedigital 6y ago
I believe that implies that the fires are indeed infinite, just countably so.

m463 6y ago

It would be fun just to be able to talk to management with the same sort of candor the "cleaner" in pulp fiction did. (harvey keitel)

mattlondon 6y ago

Constant "emergencies" soon wear you down. Sure it is fun, but the stakes can be high and the stress gets piled on. It starts with congratulations all-round. You are a hero! Good job! Nice save! Your boss's boss's boss's boss comes over and thanks you personally. Take the rest of the week off! Later, its people asking when will you fix it? Why haven't you got this back working yet? Didn'y you fix the same thing last month? (no) We really need you to fix this before 5pm or the TPS reports wont go out, and the management will be pissed. Failures become normalised. They get reliant on people doing heroics. People forget that the systems are crap and need investment, and start to rely on you being there to fix it, and if things don't get fixed then it is your fault the TPS reports didn't go out, not tech-debt/lack-of-investment/bad-design/whatever.

ido 6y ago
I remember stuff like that, once the COO called the entire company to the common room to present me a bottle of champagne for my efforts (another time I was given a fancy bottle of whiskey that some marketing guys drank 3/4 of before I ever got around to it). I rather not work any overtime instead.

papreclip 6y ago

Being an SRE is a little more of this and a little less scheduled work. You still have standup and plenty of other meetings, though