5 ms·

Show HN: Errorship, use datadog as an error tracker

47 points by RabbitmqGuy 6y ago

RabbitmqGuy 6y ago
Hey HN! errorShip is a python library that sends exceptions/errors generated by your application to your datadog account. Are you tired of looking at metrics in datadog and then switching over to another website to track your applications exceptions? errorship exists to solve that context-switching problem, among others. It's a bit like sentry, bugsnag, rollbar etc; except implemented in your own datadog account. I'm happy to receive any feedback or just chat about it.
- tyingq 6y ago
  Can you add a little more color commentary? I haven't used datadog, but they seem clear that there's both a logging and metrics (apm) side to their product. Is the logging side neglected somehow? I see, for example, docs on how to log from Python: https://www.datadoghq.com/blog/python-logging-best-practices/ https://www.datadoghq.com/blog/python-logging-best-practices...
  - RabbitmqGuy 6y ago
    Hi. > but they seem clear that there's both a logging and metrics (apm) side to their product. yes they have those two sides. what they do not have is an error tracking side; something like sentry[1]. errorship exists to bring the error tracking side into datadog. The logging side of datadog is like splunk or elasticsearch, and their APM side is like newrelic. errorship is bringing the error tracking side. some of the analogies may not apply 100%, but I hope I'm making sense. 1. https://sentry.io/ https://sentry.io/
    - tyingq 6y ago
      That's helpful. Basically zero-config central logging of exceptions/stderr/whatever?
      - RabbitmqGuy 6y ago
        Yeah. But also, errorship will still capture(and send to datadog) any exceptions that you may have forgot to capture or log.
  - EdwardDiego 6y ago
    I'm guessing that using the DD event stream rather than purchasing logging support could be a cost saving, although not sure how DD would feel about that.
    - RabbitmqGuy 6y ago
      errorship is not using the datadog eventstream to send all logs to it. Rather, errorship just sends exceptions, their stack traces(and a few other metadata like hostname) to the eventstream. > although not sure how DD would feel about that The datadog documentation[1] says; "An event represents any record of activity noteworthy for engineers"[1]. We certainly believe that exceptions that occur in engineer's applications are noteworthy activities and as such belong to the eventstream. 1. https://docs.datadoghq.com/events/ https://docs.datadoghq.com/events/
      - EdwardDiego 6y ago
        > errorship is not using the datadog eventstream to send all logs to it. Yep, noted that, and I like how you preserve syntax highlighting in the stack traces :) Out of curiosity, I've only ever dabbled in Datadog, is there any limit on event stream size or retention? I couldn't find anything mentioned in the docs.
        RabbitmqGuy 6y ago
        I had also noticed the lack of mention in the docs, so I reached out to datadog sometime last year. Their response was that it doesn't have any limits or quota
        tyingq 6y ago
        Interesting. Is there some disincentive that discourages it? Is it hard (or costly) to query, for example...compared to paying specifically for logging? Seems like a big loophole.
kcmastrpc 6y ago
Why not use distributed tracing? You get the context of entire request in addition to the exception.
- inetknght 6y ago
  Do you have examples of how that might work with other languages or tools?
ohnoesjmr 6y ago
Benefits over sentry?
- RabbitmqGuy 6y ago
  The main benefit is that you do not have to context switch from datadog metrics/logs to go look at your exceptions in sentry. With errorship, all these are made available in one place; in your datadog account. You also do not need to maintain two services. If you are already using datadog(maybe their APM and their logging and metrics service) then you might as well use them for error tracking instead of maintaining an additional account with sentry. However, it is not a must that you give up sentry to use errorship. errorship will work just as fine, if you choose to continue with sentry.
  - exhaze 6y ago
    > The main benefit is that you do not have to context switch from datadog metrics/logs to go look at your exceptions in sentry. Sentry lets you send exceptions to Datadog though?
AtroxDev 6y ago
The library calls back home to validate the key (URL: https://errorship.com/api/?errorshipLicensekey={errorship_license_key} https://errorship.com/api/?errorshipLicensekey={errorship_li...). If it fails it raises an Exception. I don't think that this is the correct way to do this. If your server is down, my application would crash too. Just cut the license validation out of the library. If I wanted to use the library without an license I could do so anyway. edit: as noted by the author below, this is not the case :). If the server is not available, it won't raise an exception. I did miss that part somehow.
- RabbitmqGuy 6y ago
  Hi. > If your server is down, my application would crash too. The errorship library is written in such a way that it fails open. If our servers are down(or any other failure), it does not affect your application and your application continues to work okay.
  - AtroxDev 6y ago
    Hey, I edited my comment. I did miss the part in http.py somehow :). Thank you for clarifying.
- citruspi 6y ago
  > If it fails it raises an Exception... If your server is down, my application would crash too So first off, let me just say that I completely agree. If this were the case, that'd be fucking atrocious and would definitely be a blocker for using it. But I'm curious how you came to that conclusion. It took me < 2 minutes of looking at the source code[0] to determine that your claim was incorrect. Not only does it appear to gracefully handle the server being unavailable, the developer literally commented that code explaining that they wanted to ensure users could continue uninterrupted if the errorship server is unavailable. > We give people the benefit of doubt. > We only consider people to be not authorized if the backend comes back with an authoritative answer to that effect. > Else, any errors or any other outcome; we assume authorization is there and also assume they belong to the highest pricing plan: Enterprise > # failure of errorship should not lead to people been unable to ship exceptions And it took even less time than that to run a new Python Docker container, install the library, run the sample code, and validate my assumptions[1] (the first attempt fails because the key is invalid, I disabled Internet access for the second attempt and it succeeded). So I'm legitimately curious - did I miss something? Is there another failure case I didn't catch or test for? Or did you just make an assumption and not bother to verify it? And if it's the latter, why? What was the point? Like, to be frank, if this was a news piece I could understand the (possibly inaccurate) commentary. But why take the time and energy to write your comment and tear down someone's personal project with seemingly inaccurate claims? (To be clear, no affiliation with errorship, I'm not even a DataDog user. Just a random dev browsing HN). [0] https://gitlab.com/errorship/errorship_python/-/blob/master/errorship/http.py#L70 https://gitlab.com/errorship/errorship_python/-/blob/master/... [1] https://gist.github.com/citruspi/16d359ac2dafef6fc876e2dd101089bf https://gist.github.com/citruspi/16d359ac2dafef6fc876e2dd101...
oefrha 6y ago
Can you resolve exceptions like in Sentry? Having to look at the list of all exceptions and figure out which ones are of interest wouldn’t be ideal. (In the demo I did notice the level/priorities filters, but nothing resembling an unresolved filter.) Also, nitpick: it’s 2020, maybe update the front page screenshot to Python 3? My immediate reaction seeing that py27 screenshot: is this even maintained?
- RabbitmqGuy 6y ago
  > Having to look at the list of all exceptions and figure out which ones are of interest wouldn’t be ideal. You can filter for exceptions by tags. You can also use full text search to filter for exceptions. ie all the functionality provided[1] by the datadog eventstream[1] is available for your errors. > Also, nitpick: it’s 2020, maybe update the front page screenshot to Python 3: is this even maintained? errorship is compatible with both python2 and python3. Yes it is maintained. We have a testsuite that is ran in CI under both python2 and python3. Some of our customers during early trials still had some python2 applications that they had yet to port over. 1. https://docs.datadoghq.com/events/#event-stream https://docs.datadoghq.com/events/#event-stream
karmakaze 6y ago
For Java I'd written a Logback exception sender[0] that creates metrics tagged by level, class, exception class, cause classes. I'll say that it is a quick and convenient way to see changes in system error characteristics. I had used a similar setup for Graphite but didn't work as well without tags. Since DataDog charges by number of metrics and each tag combination counts as one, actual exception messages should only be included in tags if they don't have instance-varying text. [0] https://github.com/karmakaze/logback-metrics-datadog https://github.com/karmakaze/logback-metrics-datadog
jere 6y ago
Incredibly dumb question, but doesn't datadog monitor errors out of the box?
- sputnikus 6y ago
  IMO only if you are using the Logs functionality. You can log exceptions/errors, and it should be pretty easy to set-up monitoring around these events.
  - stingraycharles 6y ago
    As a Datadog customer, I still use Sentry next to it, mostly for the top notch exception grouping, reporting etc. It’s just their core product, they’re better at it when you need to zoom in and actually debug. For the bird’s eye view, Datadog is fine.
MastrChefRocks 6y ago
How does this compare to Datadog's APM[0]? APM looks like it has far more features and is already integrated with Datadog. [0] https://www.datadoghq.com/apm/ https://www.datadoghq.com/apm/
- sa46 6y ago
  This looks like an admittedly clever hack to avoid the fantastically expensive APM and still get some useful exception logging. I really wanted to use Datadog’s APM but $31 per host was too much. The downsides to logging to the event stream like Errorship is that you can’t associate exceptions with requests and aggregation is limited to tags. I think event tags have a low-ish cardinality limit similar to metrics.
sa46 6y ago
I’m going to make the classic engineer mistake and assume that it’s almost as easy to implement this myself. My understanding is that errorship does something like: - register a global exception handler, probably tweaked to hook into framework specific exception mechanisms. - make the exceptions “pretty” - send the exceptions to the datadog event log. The benefits of rolling my own is avoiding a soft dependency on errorship and that I can tweak exception aggregation and reporting. Is the primary defense that errorship only costs $10 per month or is there additional complexity I’ve missed?
- RabbitmqGuy 6y ago
  Hi. That's a good summary of what errorship does. The pricing [1] page has an faq that touches on the build vs buy question 1. https://errorship.com/pricing.html https://errorship.com/pricing.html
reinkaos 6y ago
Doesn't sending the exception as tags increase metrics cardinality slowing down other queries?

47 points by RabbitmqGuy 6y ago

Hey HN! errorShip is a python library that sends exceptions/errors generated by your application to your datadog account. Are you tired of looking at metrics in datadog and then switching over to another website to track your applications exceptions? errorship exists to solve that context-switching problem, among others. It's a bit like sentry, bugsnag, rollbar etc; except implemented in your own datadog account. I'm happy to receive any feedback or just chat about it.

tyingq 6y ago
Can you add a little more color commentary? I haven't used datadog, but they seem clear that there's both a logging and metrics (apm) side to their product. Is the logging side neglected somehow? I see, for example, docs on how to log from Python: https://www.datadoghq.com/blog/python-logging-best-practices/ https://www.datadoghq.com/blog/python-logging-best-practices...
- RabbitmqGuy 6y ago
  Hi. > but they seem clear that there's both a logging and metrics (apm) side to their product. yes they have those two sides. what they do not have is an error tracking side; something like sentry[1]. errorship exists to bring the error tracking side into datadog. The logging side of datadog is like splunk or elasticsearch, and their APM side is like newrelic. errorship is bringing the error tracking side. some of the analogies may not apply 100%, but I hope I'm making sense. 1. https://sentry.io/ https://sentry.io/
  - tyingq 6y ago
    That's helpful. Basically zero-config central logging of exceptions/stderr/whatever?
    - RabbitmqGuy 6y ago
      Yeah. But also, errorship will still capture(and send to datadog) any exceptions that you may have forgot to capture or log.
- EdwardDiego 6y ago
  I'm guessing that using the DD event stream rather than purchasing logging support could be a cost saving, although not sure how DD would feel about that.
  - RabbitmqGuy 6y ago
    errorship is not using the datadog eventstream to send all logs to it. Rather, errorship just sends exceptions, their stack traces(and a few other metadata like hostname) to the eventstream. > although not sure how DD would feel about that The datadog documentation[1] says; "An event represents any record of activity noteworthy for engineers"[1]. We certainly believe that exceptions that occur in engineer's applications are noteworthy activities and as such belong to the eventstream. 1. https://docs.datadoghq.com/events/ https://docs.datadoghq.com/events/
    - EdwardDiego 6y ago
      > errorship is not using the datadog eventstream to send all logs to it. Yep, noted that, and I like how you preserve syntax highlighting in the stack traces :) Out of curiosity, I've only ever dabbled in Datadog, is there any limit on event stream size or retention? I couldn't find anything mentioned in the docs.
      - RabbitmqGuy 6y ago
        I had also noticed the lack of mention in the docs, so I reached out to datadog sometime last year. Their response was that it doesn't have any limits or quota
        tyingq 6y ago
        Interesting. Is there some disincentive that discourages it? Is it hard (or costly) to query, for example...compared to paying specifically for logging? Seems like a big loophole.

kcmastrpc 6y ago

Why not use distributed tracing? You get the context of entire request in addition to the exception.

inetknght 6y ago
Do you have examples of how that might work with other languages or tools?

ohnoesjmr 6y ago

Benefits over sentry?

RabbitmqGuy 6y ago
The main benefit is that you do not have to context switch from datadog metrics/logs to go look at your exceptions in sentry. With errorship, all these are made available in one place; in your datadog account. You also do not need to maintain two services. If you are already using datadog(maybe their APM and their logging and metrics service) then you might as well use them for error tracking instead of maintaining an additional account with sentry. However, it is not a must that you give up sentry to use errorship. errorship will work just as fine, if you choose to continue with sentry.
- exhaze 6y ago
  > The main benefit is that you do not have to context switch from datadog metrics/logs to go look at your exceptions in sentry. Sentry lets you send exceptions to Datadog though?

AtroxDev 6y ago

The library calls back home to validate the key (URL: https://errorship.com/api/?errorshipLicensekey={errorship_license_key} https://errorship.com/api/?errorshipLicensekey={errorship_li...). If it fails it raises an Exception. I don't think that this is the correct way to do this. If your server is down, my application would crash too. Just cut the license validation out of the library. If I wanted to use the library without an license I could do so anyway. edit: as noted by the author below, this is not the case :). If the server is not available, it won't raise an exception. I did miss that part somehow.

RabbitmqGuy 6y ago
Hi. > If your server is down, my application would crash too. The errorship library is written in such a way that it fails open. If our servers are down(or any other failure), it does not affect your application and your application continues to work okay.
- AtroxDev 6y ago
  Hey, I edited my comment. I did miss the part in http.py somehow :). Thank you for clarifying.
citruspi 6y ago
> If it fails it raises an Exception... If your server is down, my application would crash too So first off, let me just say that I completely agree. If this were the case, that'd be fucking atrocious and would definitely be a blocker for using it. But I'm curious how you came to that conclusion. It took me < 2 minutes of looking at the source code[0] to determine that your claim was incorrect. Not only does it appear to gracefully handle the server being unavailable, the developer literally commented that code explaining that they wanted to ensure users could continue uninterrupted if the errorship server is unavailable. > We give people the benefit of doubt. > We only consider people to be not authorized if the backend comes back with an authoritative answer to that effect. > Else, any errors or any other outcome; we assume authorization is there and also assume they belong to the highest pricing plan: Enterprise > # failure of errorship should not lead to people been unable to ship exceptions And it took even less time than that to run a new Python Docker container, install the library, run the sample code, and validate my assumptions[1] (the first attempt fails because the key is invalid, I disabled Internet access for the second attempt and it succeeded). So I'm legitimately curious - did I miss something? Is there another failure case I didn't catch or test for? Or did you just make an assumption and not bother to verify it? And if it's the latter, why? What was the point? Like, to be frank, if this was a news piece I could understand the (possibly inaccurate) commentary. But why take the time and energy to write your comment and tear down someone's personal project with seemingly inaccurate claims? (To be clear, no affiliation with errorship, I'm not even a DataDog user. Just a random dev browsing HN). [0] https://gitlab.com/errorship/errorship_python/-/blob/master/errorship/http.py#L70 https://gitlab.com/errorship/errorship_python/-/blob/master/... [1] https://gist.github.com/citruspi/16d359ac2dafef6fc876e2dd101089bf https://gist.github.com/citruspi/16d359ac2dafef6fc876e2dd101...

oefrha 6y ago

Can you resolve exceptions like in Sentry? Having to look at the list of all exceptions and figure out which ones are of interest wouldn’t be ideal. (In the demo I did notice the level/priorities filters, but nothing resembling an unresolved filter.) Also, nitpick: it’s 2020, maybe update the front page screenshot to Python 3? My immediate reaction seeing that py27 screenshot: is this even maintained?

RabbitmqGuy 6y ago
> Having to look at the list of all exceptions and figure out which ones are of interest wouldn’t be ideal. You can filter for exceptions by tags. You can also use full text search to filter for exceptions. ie all the functionality provided[1] by the datadog eventstream[1] is available for your errors. > Also, nitpick: it’s 2020, maybe update the front page screenshot to Python 3: is this even maintained? errorship is compatible with both python2 and python3. Yes it is maintained. We have a testsuite that is ran in CI under both python2 and python3. Some of our customers during early trials still had some python2 applications that they had yet to port over. 1. https://docs.datadoghq.com/events/#event-stream https://docs.datadoghq.com/events/#event-stream

karmakaze 6y ago

For Java I'd written a Logback exception sender[0] that creates metrics tagged by level, class, exception class, cause classes. I'll say that it is a quick and convenient way to see changes in system error characteristics. I had used a similar setup for Graphite but didn't work as well without tags. Since DataDog charges by number of metrics and each tag combination counts as one, actual exception messages should only be included in tags if they don't have instance-varying text. [0] https://github.com/karmakaze/logback-metrics-datadog https://github.com/karmakaze/logback-metrics-datadog

jere 6y ago

Incredibly dumb question, but doesn't datadog monitor errors out of the box?

sputnikus 6y ago
IMO only if you are using the Logs functionality. You can log exceptions/errors, and it should be pretty easy to set-up monitoring around these events.
- stingraycharles 6y ago
  As a Datadog customer, I still use Sentry next to it, mostly for the top notch exception grouping, reporting etc. It’s just their core product, they’re better at it when you need to zoom in and actually debug. For the bird’s eye view, Datadog is fine.

MastrChefRocks 6y ago

How does this compare to Datadog's APM[0]? APM looks like it has far more features and is already integrated with Datadog. [0] https://www.datadoghq.com/apm/ https://www.datadoghq.com/apm/

sa46 6y ago
This looks like an admittedly clever hack to avoid the fantastically expensive APM and still get some useful exception logging. I really wanted to use Datadog’s APM but $31 per host was too much. The downsides to logging to the event stream like Errorship is that you can’t associate exceptions with requests and aggregation is limited to tags. I think event tags have a low-ish cardinality limit similar to metrics.

sa46 6y ago

I’m going to make the classic engineer mistake and assume that it’s almost as easy to implement this myself. My understanding is that errorship does something like: - register a global exception handler, probably tweaked to hook into framework specific exception mechanisms. - make the exceptions “pretty” - send the exceptions to the datadog event log. The benefits of rolling my own is avoiding a soft dependency on errorship and that I can tweak exception aggregation and reporting. Is the primary defense that errorship only costs $10 per month or is there additional complexity I’ve missed?

RabbitmqGuy 6y ago
Hi. That's a good summary of what errorship does. The pricing [1] page has an faq that touches on the build vs buy question 1. https://errorship.com/pricing.html https://errorship.com/pricing.html

reinkaos 6y ago

Doesn't sending the exception as tags increase metrics cardinality slowing down other queries?