5 ms·

I found PagerDuty to be grossly underwhelming when our team moved to it. I'd focus on stupid-easy integration & some reasonable calendar management to different

I found PagerDuty to be grossly underwhelming when our team moved to it. I'd focus on stupid-easy integration & some reasonable calendar management to differentiate. PagerDuty Calendar/Holiday/Workday management and cross-regional scheduling were very poorly implemented. The amount of manual schedule adjustment when someone actually wanted to take their 1 week vacation was insane. We ended up with overrides on top of overrides. I'd imagine some of the common tools like Workday should give you integrations for that. Taking a holiday became like a 12 step program - email boss in advance, put in HR system request, cross check your PagerDuty schedule, negotiate a trade with another teammate for PagerDuty rotation, decline meetings, update your outlook calendar out-of-office, set your slack status & notifications, and re-remind everyone the week before you go. It's almost like they wanted it to be easier to just not take time off? The other problem with these tools is you only get value out of them depending on the level of systems integration you spend your time on. If even a single system is not integrated & still sends emails/slacks/only updates a dashboard .. then PagerDuty is simply yet-another-tool to monitor, rather than the single pane of glass.

mads_quist 4y ago
I tried to make managing the incident escalation policies as simple as possible. It currently works with a simple drag&drop mechanism. The way PagerDuty is doing this also seemed very cumbersome to me. Of course All Quiet is currently missing holiday / off hours features. I'm still figuring out what's the best approach there. Regarding your last paragraph - can you elaborate on this? I tried to make All Quiet be integratable with basically every system by Email and Webhook integrations. Do you see this as a problem? Thanks for your feedback in general!
- steveBK123 4y ago
  I think the problem with integration is probably a man-hours thing. A lot of shops have a hodgepodge of 3rd party tools which are not necessarily the latest&greatest Cloud/FAANG darlings - Geneos, Jira, ServiceNow, ControlM, Autosys, Tidal. These tools can vary by industry & org size, so maybe finding some traction in a particular slice so that you can really serve that niche can give you more traction, I dunno. It just has to be dead simple to integrate, and it's worth considering some sort of professional services/customer success engineering layer. The problem I've seen is that in every (financial services) org I have worked, the teams responsible for integrations to these types of tools are really operations type teams that do a little development in their spare time. So it never gets done well, or completely. To me, the only benefit of PagerDuty type tool is when it becomes the single point of escalation. If its just one of many, the product will not be sticky within a given org.
- steveBK123 4y ago
  Re: cost & holidays/hours On pricing - I don't think it matters for small orgs the way it does for big orgs. Let's say today I am in a role where I am effectively the CTO of a 5ish person startup, with only 3 technical. So whether I spend $200/year or $1500/year, I don't really care. Both are more than $0, and require me to review a contract, do a POC, have a lawyer or someone review things, etc. My internal time to do all those things is more like $10-30k, easily. Now, do I decide it's worth my time knowing that my devs are going to basically turn off their phones / go to do-not-disturb-mode still because it's sense of working hours is non-existent (making it worse than PagerDuty & Slack notification tuning, both of which I already despise)?
PragmaticPulp 4y ago
I’m glad to hear I’m not the only one. I constantly felt like I was missing something. A tool this mature and popular should have been a little more polished by now.
- steveBK123 4y ago
  It felt like a tool that was sold to CTOs, not users. As a recipient of pages, I abhorred PagerDuty more than Slack, Teams, email or even phone calls. At least my phone notifications & do not disturb mode can be tuned around some of those other apps. And if a support guy or teammate calls me on holiday, I can berate them such that they remember to check the holiday calendar / out-of-office status next time. No such luck with PagerDuty. If your org is not mature enough to put in the time to have a single pane of glass / view of the world / alert state.. you shouldn't be buying any of these tools.
singron 4y ago
For instance at Google, taking vacation in workday puts an OOO event on your Google calendar. Then the oncall scheduler takes that into account along with your oncall history when it schedules new rotations. You only have to get someone to cover if you are taking time off in the near future, and any surplus or deficit will be fixed by the scheduler going forward. It's kind of wild that an expensive product with tons of funding and employees is worse than some scripts cooked up by some SREs.
- steveBK123 4y ago
  I think outside some FAANG firms with strong engineering management, the purchase of these tools is a top-down affair. So as cynical as my statement may seem- " It's almost like they wanted it to be easier to just not take time off?" .. user ergonomics just doesn't enter into the conversation at all, because the users are internal devs.. who cares. I also worked at an org that gave each of 50+ large customers a dedicated slack channel with.. wait for it.. the entire internal 200+ member engineering org in channel with them. You can imagine the mayhem as each customer figured out how to get attention (@here/@channel/@theguythatfixedthislasttime) and "hack the phone tree" as I used to call it. This also meant the every one of our 200+ engineers was a member of 50+ customer channels which were filled with all levels of noise that 95% of the time had nothing to do with their app/team/function, and could/should be ignored.
  - danpalmer 4y ago
    > the purchase of these tools is a top-down affair This is absolutely it. Pager Duty don't sell on-call tools to engineers, they sell the idea of having an on-call rota to CTOs, the tools are an implementation detail. Bottom up engineering tools take more time to start with, but almost always cost less in the long run, contribute to a good engineering culture, and build a sense of ownership.
    - thayne 4y ago
      That isn't universally true. At my medium sized company, the people on call are the ones responsible for choosing an on-call system. But we've rotated through most of the main options, because we haven't really been happy with any of them.
BlackjackCF 4y ago
I’m not sure why PD is so expensive, but I think why it’s just known over competitors is just the sheer number of integrations it supports.
throwmeaway1212 4y ago
'Taking a holiday became like a 12 step program - email boss in advance, put in HR system request, cross check your PagerDuty schedule, negotiate a trade with another teammate for PagerDuty rotation, decline meetings, update your outlook calendar out-of-office, set your slack status & notifications, and re-remind everyone the week before you go. It's almost like they wanted it to be easier to just not take time off?' Most of this has nothing to do with pager duty, but instead is around the companies process.
- steveBK123 4y ago
  The problem is top-down CTO purchasing of these magic bullet tools w/o budgeting/planning integration. Instead becoming "the place to look" it becomes "the Nth place to look" for alerts. Without thoughtful planning, its entirely possible to be in worse shape for having purchased it.
SergeAx 4y ago
> decline meetings, update your outlook calendar out-of-office When I am adding OOO to my Google Calendar, there's an option to auto-decline meetings. Isn't it like that in Outlook?

mads_quist 4y ago

I tried to make managing the incident escalation policies as simple as possible. It currently works with a simple drag&drop mechanism. The way PagerDuty is doing this also seemed very cumbersome to me. Of course All Quiet is currently missing holiday / off hours features. I'm still figuring out what's the best approach there. Regarding your last paragraph - can you elaborate on this? I tried to make All Quiet be integratable with basically every system by Email and Webhook integrations. Do you see this as a problem? Thanks for your feedback in general!

steveBK123 4y ago
I think the problem with integration is probably a man-hours thing. A lot of shops have a hodgepodge of 3rd party tools which are not necessarily the latest&greatest Cloud/FAANG darlings - Geneos, Jira, ServiceNow, ControlM, Autosys, Tidal. These tools can vary by industry & org size, so maybe finding some traction in a particular slice so that you can really serve that niche can give you more traction, I dunno. It just has to be dead simple to integrate, and it's worth considering some sort of professional services/customer success engineering layer. The problem I've seen is that in every (financial services) org I have worked, the teams responsible for integrations to these types of tools are really operations type teams that do a little development in their spare time. So it never gets done well, or completely. To me, the only benefit of PagerDuty type tool is when it becomes the single point of escalation. If its just one of many, the product will not be sticky within a given org.
steveBK123 4y ago
Re: cost & holidays/hours On pricing - I don't think it matters for small orgs the way it does for big orgs. Let's say today I am in a role where I am effectively the CTO of a 5ish person startup, with only 3 technical. So whether I spend $200/year or $1500/year, I don't really care. Both are more than $0, and require me to review a contract, do a POC, have a lawyer or someone review things, etc. My internal time to do all those things is more like $10-30k, easily. Now, do I decide it's worth my time knowing that my devs are going to basically turn off their phones / go to do-not-disturb-mode still because it's sense of working hours is non-existent (making it worse than PagerDuty & Slack notification tuning, both of which I already despise)?

PragmaticPulp 4y ago

I’m glad to hear I’m not the only one. I constantly felt like I was missing something. A tool this mature and popular should have been a little more polished by now.

steveBK123 4y ago
It felt like a tool that was sold to CTOs, not users. As a recipient of pages, I abhorred PagerDuty more than Slack, Teams, email or even phone calls. At least my phone notifications & do not disturb mode can be tuned around some of those other apps. And if a support guy or teammate calls me on holiday, I can berate them such that they remember to check the holiday calendar / out-of-office status next time. No such luck with PagerDuty. If your org is not mature enough to put in the time to have a single pane of glass / view of the world / alert state.. you shouldn't be buying any of these tools.

singron 4y ago

For instance at Google, taking vacation in workday puts an OOO event on your Google calendar. Then the oncall scheduler takes that into account along with your oncall history when it schedules new rotations. You only have to get someone to cover if you are taking time off in the near future, and any surplus or deficit will be fixed by the scheduler going forward. It's kind of wild that an expensive product with tons of funding and employees is worse than some scripts cooked up by some SREs.

steveBK123 4y ago
I think outside some FAANG firms with strong engineering management, the purchase of these tools is a top-down affair. So as cynical as my statement may seem- " It's almost like they wanted it to be easier to just not take time off?" .. user ergonomics just doesn't enter into the conversation at all, because the users are internal devs.. who cares. I also worked at an org that gave each of 50+ large customers a dedicated slack channel with.. wait for it.. the entire internal 200+ member engineering org in channel with them. You can imagine the mayhem as each customer figured out how to get attention (@here/@channel/@theguythatfixedthislasttime) and "hack the phone tree" as I used to call it. This also meant the every one of our 200+ engineers was a member of 50+ customer channels which were filled with all levels of noise that 95% of the time had nothing to do with their app/team/function, and could/should be ignored.
- danpalmer 4y ago
  > the purchase of these tools is a top-down affair This is absolutely it. Pager Duty don't sell on-call tools to engineers, they sell the idea of having an on-call rota to CTOs, the tools are an implementation detail. Bottom up engineering tools take more time to start with, but almost always cost less in the long run, contribute to a good engineering culture, and build a sense of ownership.
  - thayne 4y ago
    That isn't universally true. At my medium sized company, the people on call are the ones responsible for choosing an on-call system. But we've rotated through most of the main options, because we haven't really been happy with any of them.

BlackjackCF 4y ago

I’m not sure why PD is so expensive, but I think why it’s just known over competitors is just the sheer number of integrations it supports.

throwmeaway1212 4y ago

'Taking a holiday became like a 12 step program - email boss in advance, put in HR system request, cross check your PagerDuty schedule, negotiate a trade with another teammate for PagerDuty rotation, decline meetings, update your outlook calendar out-of-office, set your slack status & notifications, and re-remind everyone the week before you go. It's almost like they wanted it to be easier to just not take time off?' Most of this has nothing to do with pager duty, but instead is around the companies process.

steveBK123 4y ago
The problem is top-down CTO purchasing of these magic bullet tools w/o budgeting/planning integration. Instead becoming "the place to look" it becomes "the Nth place to look" for alerts. Without thoughtful planning, its entirely possible to be in worse shape for having purchased it.

SergeAx 4y ago

> decline meetings, update your outlook calendar out-of-office When I am adding OOO to my Google Calendar, there's an option to auto-decline meetings. Isn't it like that in Outlook?