7 ms·

Dead Man's Snitch – Monitoring for Scheduled Tasks

94 points by xmjw 13y ago

mmelin 13y ago
Looks pretty cool! You should really rework the Plans page. At first I didn't realize there was more content below the "Free" plan, because I'm on a 13" OS X machine so scroll bars aren't visible by default.
petercooper 13y ago
I don't know if this is what lead to this going up, but patio11 mentioned this service the other day in his epic post about Tarsnap: http://www.kalzumeus.com/2014/04/03/fantasy-tarsnap/ http://www.kalzumeus.com/2014/04/03/fantasy-tarsnap/ .. he uses Dead Man's Snitch to ensure his backups continue working (or more accurately, to notify him when they don't!)
mrmondo 13y ago
Looks very interesting, but unless I can host it myself it's a no-go for me.
- bowlofpetunias 13y ago
  Uhm, the whole point of services like these is that they are external and not tied into your own hosting. Also, it knows absolutely nothing about your services and doesn't require any level of access. All you do is ping it.
  - peterwwillis 13y ago
    It requires internet-level access. So internal hosts that shouldn't have internet access (like a backup machine, where all your most critical data is stored) might not be able to reach it.
- genericacct 13y ago
  I can set it up for you on your *ix machine in an afternoon for $200 :)
- rhoml 13y ago
  So, in case you didn't notice. This can be a way to let others take care of notifying you when cron's fail so you can focus on more productive tasks.
samcrawford 13y ago
Small gripe, based upon prior painful experience... you should always set a timeout on curl requests (or any others for that matter). There's not one by default! It's -m <seconds> in the curl command line client.
- jameskilton 13y ago
  Good point, we'll make a note to include this in our documentation!
bowlofpetunias 13y ago
Been using DMS for over quite a while now. It is an absolute must-have for anybody depending on scheduled tasks, and I haven't found anything like it. And it's so cheap that there is no reason not to do it. My only gripe is that I can only get alerted via email, which is the most often overlooked channel. I've hooked those emails up to HipChat, but some direct options for services like HipChat, Pagerduty etcetera would be nice.
- ralfn 13y ago
  Doesn't it do push notifications on iOS?
  - jameskilton 13y ago
    Yes, the iOS app has full push notification support! To suggest other integration points, please contact us at hi@deadmanssnitch.com.
ralfn 13y ago
Nice branding. Most monitoring tools are overly complicated, both in their features as well the use-cases they try to cover. I like the clear focus of this; the product itself doesn't require one to invest a lot of time, which is a good thing. Just some questions: - how do you prevent unauthorized snitches from screwing with your data? Wouldn't it be wise to limit a snitch to a certain server ip? - why only an iOS app? This excludes lots of potential customers (including me). - why not offer an API, so we can securely fetch the status of our snitches? I like the singular focus. Some people don't want very complicated feature rich monitor systems. I can imagine using a solution like this, esspecially in those cases, where more feature rich monitor systems are not being considered. Yet, you offer a pricing model that suggests something completely different. That you are a very complicated feature-rich monitor system. The value you provide at each price point is simply not warranted. I doubt this will have a large uptake, but if it does, i'm the first one tempted to compete. The required architecture is not complicated enough to warrant such high margins on such low costs. Cloud providers, even the budget ones like DigitalOcean, could easily offer something like this for free to their customers. Now, you could argue that the price includes high quality support. But that's the not the audience of such a singular focus tool. If i need support, i'm obviously spending enough time interacting with it, that i would be better off with a more feature-rich monitor system. (which are available at similar price points) Who's going to pay 600 dollars a year for 300 snitches? To build this at scale may take more effort, but from the perspective of your customer .. wouldn't most of them be able to build something like this for internal use in a single day? There are open source solutions with more features you can install with a single command. I'm just wondering: but wouldn't you have much more customers, and hence, more profit, if you would provide 500 snitches at 100 dollars a year? Or alternatively, offer the service for free, but just charge for the apps yearly? In the end, a single snitch will do at most 24 empty GET requests a day, where you store a timestamp. A single VPS should be able to handle 10k snitches easily. (less than 3 requests per second)
- struppi 13y ago
  You do realize that 600 dollars a year is next to nothing for a business? I don't even run a business, I am just a freelancer, but I'd happily pay 600 USD/year for this service if I needed it. The price doesn't make a real dent in any serious budget, so why bother? And 600 dollars is really cheap compared to anything that could happen when you miss an important event. Email campaign not sent? This could cost thousands of dollars or more. Backup not done? Could be millions of dollars for some kinds of businesses. I think the service is priced reasonably. If you think otherwise, start a competitor. If you can convince me that your service will reliably mail me on incidents no matter what catastrophe happens to your data center, I might use it over theirs (Should I ever need such a service. As I said before, I don't need it right now.).
lifeisstillgood 13y ago
So instead of MAILTO=sendmail I use MAILTO=http://deadmandssnitch/report http://deadmandssnitch/report or similar?
- jameskilton 13y ago
  Actually something like: 0 6 * * * /bin/sh my_cron_job.sh && curl https://nosnch.in/abc123 https://nosnch.in/abc123 Then, when my_cron_job.sh fails (non-0 exit status), no snitch is triggered. DMS notices this hole and notifies you that the job did not run.
- peterwwillis 13y ago
  That would actually be more stable, since it would rely on a network designed for fault-tolerant redundant store-and-forward messaging. Instead, each cron job runs an HTTP GET, and later you can view the service's website to see the tests that have requested successfully or haven't responded at all. Unfortunately for your system, if there's a network hiccup, you won't know why your jobs failed and the job will hang on the HTTP call. Also, you're paying for it instead of just sending e-mail alerts to yourself.
  - patio11 13y ago
    Unfortunately for your system, if there's a network hiccup, you won't know why your jobs failed and the job will hang on the HTTP call. Also, you're paying for it instead of just sending e-mail alerts to yourself. Critically importantly for this use case, if you are doing something critical for your business (say, backups or your daily billing run or sending reminders to thousands of people or what have you) and something goes wrong with your network and makes you unable to curl, you get told about that. If you instead do self-hosted email, and your network fails, you don't get told about that. It's fail-open vs. fail-close. That's literally so core to the product it is alluded to in the name. A Dead Man's Switch doesn't go off if anything goes wrong, it goes off if everything doesn't go right.
    - peterwwillis 13y ago
KaiserPro 13y ago
as much as I hate to admit it, jenkins makes a superb replacement for cron. It keeps a history, has excellent error handling capabilities, and can scale quite well. also the integration with git and the like makes updating jobs super easy.
- khill 13y ago
  Hudson/Jenkins works well for us. We've moved all our scheduling jobs out of cron and other uglier options (like maestro) into Hudson. In addition to sending emails when things break, it can also post to our Twitter feed and send SMS alerts.
mpclark 13y ago
As an aside, does anyone make a cron panel for humans? I'd find that really useful as I'm usually so busy being distracted by the next big thing that I forget to take care of repetitive tasks. I know I can achieve something like that with my Google Calendar but I don't like the way repeating events fill it up -- it would be better if they just showed for the next occurrence.
- icebraining 13y ago
  Why not make a second calendar in GC, put the repeating events there (with notifications enabled), then hide it from the main view?
- gtCameron 13y ago
  I use FollowUpThen - https://www.followupthen.com/ https://www.followupthen.com/ It does a lot of stuff, but I have it email me every Thursday to remind me to take the trash to the curb. Its just an email, not a calendar alert, but it keeps the clutter off the calendar and I usually see an email within a couple hours anyway.
apinstein 13y ago
Nice job! -- we actually needed this so badly a few years back that I wrote a simple app in a day to do this myself, though it's not very robust. I am actually surprised it's taken so long for someone to offer a SaaS version. Your pricing is so reasonable that I'd switch to your app if you could just flush out one more feature. I actually don't want another app for reporting. I already use a monitoring service (pingdom) and would prefer to keep all of our "alerts" organized through a single service. Could you offer a http-based public status page? Our system allowed each snitch to be tagged with a tag like "net.tourbuzz.db.replication.upToDate" and then a url like http://foo.com/deadoralive?filter=net.tourbuzz http://foo.com/deadoralive?filter=net.tourbuzz would allow us to use a single Pingdom monitor to alert us if any of snitches in that namespace failed. This is a simple tagging structure that makes it easy to organize large numbers of snitches into manageable groups and easily integrate it into existing monitoring/alerting infrastructure. Feel free to email me directly if you want more info.
zufallsheld 13y ago
If you are like me and already have nagios and nrpe in use, you can use a simple python script[0] to monitor the output of the cronjob and get an alert by nagios. [0]https://github.com/rndmh3ro/check_exit-code https://github.com/rndmh3ro/check_exit-code
kareemm 13y ago
Been using DMS for years now (it was recently bought by Collective Idea). Love it so much that I'm on the testimonials page. The reason we started using it is because it's hard to notice the absence of an email alerting you that job xyz is complete. E.g. when our nightly backup would complete, we'd get an email. We'd also get N other emails nightly emails about other tasks that would run. On the rare occasion that e.g. backup wouldn't complete it would be really tough to notice that you didn't get the email. What you really want is to hear nothing UNLESS something goes wrong, which DMS makes happen. Love love love it.
cjjuice 13y ago
I like it. Pricing seems a bit high though. Development wise this is a 1-2 day project. I can't see myself spending much on something I could roll my own version of in a day or two.
spncr2 13y ago
the referral features is nice though. Free snitches for every person you refer.

94 points by xmjw 13y ago

Looks pretty cool! You should really rework the Plans page. At first I didn't realize there was more content below the "Free" plan, because I'm on a 13" OS X machine so scroll bars aren't visible by default.

petercooper 13y ago

I don't know if this is what lead to this going up, but patio11 mentioned this service the other day in his epic post about Tarsnap: http://www.kalzumeus.com/2014/04/03/fantasy-tarsnap/ http://www.kalzumeus.com/2014/04/03/fantasy-tarsnap/ .. he uses Dead Man's Snitch to ensure his backups continue working (or more accurately, to notify him when they don't!)

mrmondo 13y ago

Looks very interesting, but unless I can host it myself it's a no-go for me.

bowlofpetunias 13y ago
Uhm, the whole point of services like these is that they are external and not tied into your own hosting. Also, it knows absolutely nothing about your services and doesn't require any level of access. All you do is ping it.
- peterwwillis 13y ago
  It requires internet-level access. So internal hosts that shouldn't have internet access (like a backup machine, where all your most critical data is stored) might not be able to reach it.
genericacct 13y ago
I can set it up for you on your *ix machine in an afternoon for $200 :)
rhoml 13y ago
So, in case you didn't notice. This can be a way to let others take care of notifying you when cron's fail so you can focus on more productive tasks.

samcrawford 13y ago

Small gripe, based upon prior painful experience... you should always set a timeout on curl requests (or any others for that matter). There's not one by default! It's -m <seconds> in the curl command line client.

jameskilton 13y ago
Good point, we'll make a note to include this in our documentation!

bowlofpetunias 13y ago

Been using DMS for over quite a while now. It is an absolute must-have for anybody depending on scheduled tasks, and I haven't found anything like it. And it's so cheap that there is no reason not to do it. My only gripe is that I can only get alerted via email, which is the most often overlooked channel. I've hooked those emails up to HipChat, but some direct options for services like HipChat, Pagerduty etcetera would be nice.

ralfn 13y ago
Doesn't it do push notifications on iOS?
- jameskilton 13y ago
  Yes, the iOS app has full push notification support! To suggest other integration points, please contact us at hi@deadmanssnitch.com.

ralfn 13y ago

Nice branding. Most monitoring tools are overly complicated, both in their features as well the use-cases they try to cover. I like the clear focus of this; the product itself doesn't require one to invest a lot of time, which is a good thing. Just some questions: - how do you prevent unauthorized snitches from screwing with your data? Wouldn't it be wise to limit a snitch to a certain server ip? - why only an iOS app? This excludes lots of potential customers (including me). - why not offer an API, so we can securely fetch the status of our snitches? I like the singular focus. Some people don't want very complicated feature rich monitor systems. I can imagine using a solution like this, esspecially in those cases, where more feature rich monitor systems are not being considered. Yet, you offer a pricing model that suggests something completely different. That you are a very complicated feature-rich monitor system. The value you provide at each price point is simply not warranted. I doubt this will have a large uptake, but if it does, i'm the first one tempted to compete. The required architecture is not complicated enough to warrant such high margins on such low costs. Cloud providers, even the budget ones like DigitalOcean, could easily offer something like this for free to their customers. Now, you could argue that the price includes high quality support. But that's the not the audience of such a singular focus tool. If i need support, i'm obviously spending enough time interacting with it, that i would be better off with a more feature-rich monitor system. (which are available at similar price points) Who's going to pay 600 dollars a year for 300 snitches? To build this at scale may take more effort, but from the perspective of your customer .. wouldn't most of them be able to build something like this for internal use in a single day? There are open source solutions with more features you can install with a single command. I'm just wondering: but wouldn't you have much more customers, and hence, more profit, if you would provide 500 snitches at 100 dollars a year? Or alternatively, offer the service for free, but just charge for the apps yearly? In the end, a single snitch will do at most 24 empty GET requests a day, where you store a timestamp. A single VPS should be able to handle 10k snitches easily. (less than 3 requests per second)

struppi 13y ago
You do realize that 600 dollars a year is next to nothing for a business? I don't even run a business, I am just a freelancer, but I'd happily pay 600 USD/year for this service if I needed it. The price doesn't make a real dent in any serious budget, so why bother? And 600 dollars is really cheap compared to anything that could happen when you miss an important event. Email campaign not sent? This could cost thousands of dollars or more. Backup not done? Could be millions of dollars for some kinds of businesses. I think the service is priced reasonably. If you think otherwise, start a competitor. If you can convince me that your service will reliably mail me on incidents no matter what catastrophe happens to your data center, I might use it over theirs (Should I ever need such a service. As I said before, I don't need it right now.).

lifeisstillgood 13y ago

So instead of MAILTO=sendmail I use MAILTO=http://deadmandssnitch/report http://deadmandssnitch/report or similar?

jameskilton 13y ago
Actually something like: 0 6 * * * /bin/sh my_cron_job.sh && curl https://nosnch.in/abc123 https://nosnch.in/abc123 Then, when my_cron_job.sh fails (non-0 exit status), no snitch is triggered. DMS notices this hole and notifies you that the job did not run.
peterwwillis 13y ago
That would actually be more stable, since it would rely on a network designed for fault-tolerant redundant store-and-forward messaging. Instead, each cron job runs an HTTP GET, and later you can view the service's website to see the tests that have requested successfully or haven't responded at all. Unfortunately for your system, if there's a network hiccup, you won't know why your jobs failed and the job will hang on the HTTP call. Also, you're paying for it instead of just sending e-mail alerts to yourself.
- patio11 13y ago
  Unfortunately for your system, if there's a network hiccup, you won't know why your jobs failed and the job will hang on the HTTP call. Also, you're paying for it instead of just sending e-mail alerts to yourself. Critically importantly for this use case, if you are doing something critical for your business (say, backups or your daily billing run or sending reminders to thousands of people or what have you) and something goes wrong with your network and makes you unable to curl, you get told about that. If you instead do self-hosted email, and your network fails, you don't get told about that. It's fail-open vs. fail-close. That's literally so core to the product it is alluded to in the name. A Dead Man's Switch doesn't go off if anything goes wrong, it goes off if everything doesn't go right.
  - peterwwillis 13y ago

KaiserPro 13y ago

as much as I hate to admit it, jenkins makes a superb replacement for cron. It keeps a history, has excellent error handling capabilities, and can scale quite well. also the integration with git and the like makes updating jobs super easy.

khill 13y ago
Hudson/Jenkins works well for us. We've moved all our scheduling jobs out of cron and other uglier options (like maestro) into Hudson. In addition to sending emails when things break, it can also post to our Twitter feed and send SMS alerts.

mpclark 13y ago

As an aside, does anyone make a cron panel for humans? I'd find that really useful as I'm usually so busy being distracted by the next big thing that I forget to take care of repetitive tasks. I know I can achieve something like that with my Google Calendar but I don't like the way repeating events fill it up -- it would be better if they just showed for the next occurrence.

icebraining 13y ago
Why not make a second calendar in GC, put the repeating events there (with notifications enabled), then hide it from the main view?
gtCameron 13y ago
I use FollowUpThen - https://www.followupthen.com/ https://www.followupthen.com/ It does a lot of stuff, but I have it email me every Thursday to remind me to take the trash to the curb. Its just an email, not a calendar alert, but it keeps the clutter off the calendar and I usually see an email within a couple hours anyway.

apinstein 13y ago

Nice job! -- we actually needed this so badly a few years back that I wrote a simple app in a day to do this myself, though it's not very robust. I am actually surprised it's taken so long for someone to offer a SaaS version. Your pricing is so reasonable that I'd switch to your app if you could just flush out one more feature. I actually don't want another app for reporting. I already use a monitoring service (pingdom) and would prefer to keep all of our "alerts" organized through a single service. Could you offer a http-based public status page? Our system allowed each snitch to be tagged with a tag like "net.tourbuzz.db.replication.upToDate" and then a url like http://foo.com/deadoralive?filter=net.tourbuzz http://foo.com/deadoralive?filter=net.tourbuzz would allow us to use a single Pingdom monitor to alert us if any of snitches in that namespace failed. This is a simple tagging structure that makes it easy to organize large numbers of snitches into manageable groups and easily integrate it into existing monitoring/alerting infrastructure. Feel free to email me directly if you want more info.

zufallsheld 13y ago

If you are like me and already have nagios and nrpe in use, you can use a simple python script[0] to monitor the output of the cronjob and get an alert by nagios. [0]https://github.com/rndmh3ro/check_exit-code https://github.com/rndmh3ro/check_exit-code

kareemm 13y ago

Been using DMS for years now (it was recently bought by Collective Idea). Love it so much that I'm on the testimonials page. The reason we started using it is because it's hard to notice the absence of an email alerting you that job xyz is complete. E.g. when our nightly backup would complete, we'd get an email. We'd also get N other emails nightly emails about other tasks that would run. On the rare occasion that e.g. backup wouldn't complete it would be really tough to notice that you didn't get the email. What you really want is to hear nothing UNLESS something goes wrong, which DMS makes happen. Love love love it.

cjjuice 13y ago

I like it. Pricing seems a bit high though. Development wise this is a 1-2 day project. I can't see myself spending much on something I could roll my own version of in a day or two.

spncr2 13y ago

the referral features is nice though. Free snitches for every person you refer.