7 ms·

David from Sentry here. We're hoping to do a sort-of AMA in the future about rebuilding Sentry (and the open source approach), but if anyone has any questions

by zeeg 11y ago

David from Sentry here. We're hoping to do a sort-of AMA in the future about rebuilding Sentry (and the open source approach), but if anyone has any questions about the new version I'm happy to answer them.

ddorian43 11y ago
Did you use a library for the search-bar (frontend) ?
- zeeg 11y ago
  We wanted to but nothing really handled tokenization well. In the end we wrote our own (which is why there are various caveats). For other things we're using select2. At one point we had tried to switch to selectize but we found limitations in its API and frankly didn't want to spend the time to find ways to work around them.
  - ddorian43 11y ago
    Did you try: https://documentcloud.github.io/visualsearch/ https://documentcloud.github.io/visualsearch/ ?
    - bentlegen 11y ago
      Nope. I don't think we were even aware of this project. It's pretty slick; I'd probably experiment putting it into Sentry.
      - conesus 11y ago
        Hey, I wrote that widget! I'm also a long-time Sentry user and would love to see VisualSearch.js integrated.
willtheperson 11y ago
How do you see yourself stacking up against Rollbar https://rollbar.com https://rollbar.com which has a free tier and unlimited users for paid plans? What makes your product more valuable?
- zeeg 11y ago
  We've been around for the better part of a decade and are completely open source (and free). Everything from our clients (which a lot of the community started) to every piece of the server. The only code you don't get access to is our billing infrastructure. We're also fortunate to call many of the biggest names in Silicon Valley (as well as outside) our customers. What you'll quickly find is anything that looks like Sentry, acts a lot like Sentry, and usually was inspired by Sentry. What you'll also discover is that even with this in mind, they are far from the same level of maturity. For example, we've spent a lot of time tuning our grouping algorithms, expanding support for power users, and generally solving problems that aren't obvious until you've invested a lot of time into using the tool. This is a long winded way of me saying that Sentry is the best in the business.
  - brianr 11y ago
    Rollbar actually wasn't inspired by Sentry -- it was inspired by our founding team's efforts to solve exception tracking internally at Lolapps. Several of the new features in Sentry (like release management / deploy tracking) actually look a lot like things that have been in Rollbar for a long while, though I expect that is the result of two smart teams attacking the same problem, rather than one inspiring the other.
  - ergo14 11y ago
    I believe there were services like errormator, airbrake or arecibo that predate sentry, and I'm sure there were others I don't remember right now :-)
buremba 11y ago
Do you have any numbers about what percentage of your customers running Sentry on their own nodes instead of just using cloud version that you provide?
- zeeg 11y ago
  We do have a beacon that reports anonymous statistics to us as well as giving us an estimate on unique installations. That said, it's an optional feature, and its only in newer versions. One thing that is generally true is many companies don't update frequently so we end up not getting a large chunk of data due to them running on older versions. If take only our data (ignoring anyone who isnt actively reporting to us), It's a fairly close 50% split of on-premise companies to hosted customers. That said, our estimate is its more around 30% of users run off of getsentry.com, and the other chunk runs on-premise.
wheaties 11y ago
Do you plan on supporting JVM based languages?
- zeeg 11y ago
  Absolutely! We do have some support for Java, but its unfortunately heavily tied into Logging right now. Our short term goal is to at the very least ensure we have proper documentation (and support) for getting things up and running in Clojure and Scala.
conesus 11y ago
David, I've been a Sentry customer for years and I rely on it. But one feature I miss dearly is having a 1 day graph that shows me the health of my servers based on a single count. A few months ago the top-right dashboard lost its total count and I had to rely on hovering over the graph bars to see what the numbers looked like. I generate a hundred exceptions in a day (on a good day), and I like to use that count to gauge if anything is wrong. A bad day can generate hundreds of exceptions and I like to have an easy way to spot those days. The dashboard view kind of gives that, but there is no single number of exceptions that I can use. And I greatly miss being able to have that count and graph on the same list of exceptions. Can you bring back that 1 day graph, complete with a total exception count?
- zeeg 11y ago
  We're definitely going to be improving things like the dashboard (and organization dash). I do agree regarding better visibility into what the graphs represent, and we've opened an issue to track this: https://github.com/getsentry/sentry/issues/2131 https://github.com/getsentry/sentry/issues/2131 (There's a few other related issues to this, but I think this addresses that core problem)
lobster_johnson 11y ago
Sentry looks cool. I have previously tested Rollbar and Airbrake, but was deeply unsatisfied with both. The big problem I have with all these services is that you need set up each application individually: Each one is a silo, with its own notification settings, API key, and so on. We have dozens of apps (several products + lots of microservices), so this is just unacceptable. I looked into whether I could use a single API key with Rollbar or Airbrake, and simply tag each event with the name of the sender, but it turns out this is not feasible. They don't even have APIs that let you programmatically configure apps remotely. How does Sentry compare? (Many other SaaS services suffer from the same issue; for example, Semaphore, which we use for CI, requires a build script for each project. We worked around this by letting the build script be a single curl command that pipes a remote script to bash, shared across all projects. But hardly ideal.)
- zeeg 11y ago
  We're pretty similar in a lot of ways. It's definitely a pain point, but from a UX perspective its tough to solve. In theory you could manage a lot of things with the API (most of our UI runs off the API these days). Even with the limitations, we have many people with hundreds of projects, and the general problems we see are more around notification settings than anything else. The usual suspect is that people are getting too many emails when they either want none, or they want digests. We are giving this a lot of thought as part of our upcoming infrastructure pass. From my point of view it'd make a lot of sense if you could just have a massive stream of events and sort through them however you want. That causes a lot of challenges so it may not be feasible. Either way its a problem we're aware of and looking to address. The simplest solution from our point of view is to allow you to configure some kind of organization-wide defaults. That won't resolve the API key constraints, but we don't feel that specifically is a pain point.
  - atombender 11y ago
    I see. What I don't understand is why settings need to be so granular. For example, our company uses Slack. All notifications go to Slack. There's no need for per-project settings because it's all just one room. At some point we will probably want some projects to go to other rooms, but that's a coarse-grained exception, not something that warrants a per-project setting. Also, if you need per-person notification settings, then surely that's part of the user's account/profile, not the project. That would be just a single dashboard page with (say) on/off sliders for each project. In my mind, the data model used by exception trackers (including, it seems, Sentry) is wrong. There needs to be just one stream of events, tagged by things like application name that you can filter on. How those events map to things like notifications and aggregations and rules and source files and teams and so on is completely orthogonal to the stream. Those things then apply to a subset that you can modify dynamically without having to reconfigure the sender.
ryanpetrich 11y ago
The link at the bottom of the blog post leads to a 404
- zeeg 11y ago
  Doh! Fixed ;)
lobster_johnson 11y ago
Addendum to my last comment: There really should be a requirement that SaaS services supported some kind of load/dump configuration API, where you can deal with raw config. The admin UI should be completely optional. Most SaaS vendors don't have APIs that cover all parts of the service. Some do have decent APIs -- for example, we store our DNS records in files and use Gandi's APIs to update our zones, using a custom script (with some nice diffing capabilities), and we do some automatic configuring of Mailgun -- but APIs are always stateful and never declarative, which means that a client will always have to issue half a dozen calls to diff and converge its state. It's painful.
- zeeg 11y ago
  That's an interesting idea. We're Datadog users and one behavior I like is that, while they have a fancy way to build the charts, you can simply click an advanced mode and grab the full JSON configuration (thus being able to copy paste it). We'll give some thought into this, as it's not too far from being a possibility.
voltagex_ 11y ago
Can I come work for you?
- zeeg 11y ago
  We're always looking to expand the team, though our team is small so we may not always have open roles. That said feel free to throw your hat into the ring: https://www.getsentry.com/jobs/ https://www.getsentry.com/jobs/
robbiet480 11y ago
I already implemented releases a few weeks ago. They're great, thanks for that! I'm wondering what the reasoning is behind suggesting I upload source artifacts to you as part of a release. The blog post even specifically calls out Go, which is what the majority of our application is written in. So from my understanding, uploading artifacts would allow the Sentry UI to show me exactly where we did a Sentry Dump? On the same topic, our current deploy flow makes it a bit hard to upload specific code changes per release. It would be great if Sentry could get the code from Github directly (assuming that I login and give you access to the repo) Thanks for Sentry, we use it daily and would have a much harder time tracing errors and monitoring overall application health without it.
- mattrobenolt 11y ago
  > I'm wondering what the reasoning is behind suggesting I upload source artifacts to you as part of a release. The blog post even specifically calls out Go, which is what the majority of our application is written in. So from my understanding, uploading artifacts would allow the Sentry UI to show me exactly where we did a Sentry Dump? For Go, you ultimately compile your application into a binary. This binary doesn't have the source code, but it does retain references to the filenames and line number in the binary's metadata. What this means is, if you send along a stack trace to us, we can tell you that information, but we can't show you what the actual line of code is, or what the surrounding code is. By uploading the sources to us explicitly, then baking in the release version into your compiled binary, we can map those up and show you a stack trace that includes source code. This is relatively the same idea with any modern JavaScript. Typically a project is compiled down into a minified version, or run through a transpiler, etc, and the original sources aren't accessible to us. So it's then necessary to upload to us so we can map things up. > On the same topic, our current deploy flow makes it a bit hard to upload specific code changes per release. It would be great if Sentry could get the code from Github directly (assuming that I login and give you access to the repo) We have plans for VCS integrations in the future (GitHub included) that would likely work with releases so we can just use the working directory at a current sha.
  - robbiet480 11y ago
crudbug 11y ago
Great work, I would suggest decoupling the front-end react components to create a separate widget library.
- zeeg 11y ago
  Our goal is definitely to decouple the frontend from the backend (entirely). We're still a ways out from that, but we've made a lot of progress.
  - crudbug 11y ago
    +1

by zeeg 11y ago

ddorian43 11y ago

Did you use a library for the search-bar (frontend) ?

zeeg 11y ago
We wanted to but nothing really handled tokenization well. In the end we wrote our own (which is why there are various caveats). For other things we're using select2. At one point we had tried to switch to selectize but we found limitations in its API and frankly didn't want to spend the time to find ways to work around them.
- ddorian43 11y ago
  Did you try: https://documentcloud.github.io/visualsearch/ https://documentcloud.github.io/visualsearch/ ?
  - bentlegen 11y ago
    Nope. I don't think we were even aware of this project. It's pretty slick; I'd probably experiment putting it into Sentry.
    - conesus 11y ago
      Hey, I wrote that widget! I'm also a long-time Sentry user and would love to see VisualSearch.js integrated.

willtheperson 11y ago

How do you see yourself stacking up against Rollbar https://rollbar.com https://rollbar.com which has a free tier and unlimited users for paid plans? What makes your product more valuable?

zeeg 11y ago
We've been around for the better part of a decade and are completely open source (and free). Everything from our clients (which a lot of the community started) to every piece of the server. The only code you don't get access to is our billing infrastructure. We're also fortunate to call many of the biggest names in Silicon Valley (as well as outside) our customers. What you'll quickly find is anything that looks like Sentry, acts a lot like Sentry, and usually was inspired by Sentry. What you'll also discover is that even with this in mind, they are far from the same level of maturity. For example, we've spent a lot of time tuning our grouping algorithms, expanding support for power users, and generally solving problems that aren't obvious until you've invested a lot of time into using the tool. This is a long winded way of me saying that Sentry is the best in the business.
- brianr 11y ago
  Rollbar actually wasn't inspired by Sentry -- it was inspired by our founding team's efforts to solve exception tracking internally at Lolapps. Several of the new features in Sentry (like release management / deploy tracking) actually look a lot like things that have been in Rollbar for a long while, though I expect that is the result of two smart teams attacking the same problem, rather than one inspiring the other.
- ergo14 11y ago
  I believe there were services like errormator, airbrake or arecibo that predate sentry, and I'm sure there were others I don't remember right now :-)

buremba 11y ago

Do you have any numbers about what percentage of your customers running Sentry on their own nodes instead of just using cloud version that you provide?

zeeg 11y ago
We do have a beacon that reports anonymous statistics to us as well as giving us an estimate on unique installations. That said, it's an optional feature, and its only in newer versions. One thing that is generally true is many companies don't update frequently so we end up not getting a large chunk of data due to them running on older versions. If take only our data (ignoring anyone who isnt actively reporting to us), It's a fairly close 50% split of on-premise companies to hosted customers. That said, our estimate is its more around 30% of users run off of getsentry.com, and the other chunk runs on-premise.

wheaties 11y ago

Do you plan on supporting JVM based languages?

zeeg 11y ago
Absolutely! We do have some support for Java, but its unfortunately heavily tied into Logging right now. Our short term goal is to at the very least ensure we have proper documentation (and support) for getting things up and running in Clojure and Scala.

conesus 11y ago

David, I've been a Sentry customer for years and I rely on it. But one feature I miss dearly is having a 1 day graph that shows me the health of my servers based on a single count. A few months ago the top-right dashboard lost its total count and I had to rely on hovering over the graph bars to see what the numbers looked like. I generate a hundred exceptions in a day (on a good day), and I like to use that count to gauge if anything is wrong. A bad day can generate hundreds of exceptions and I like to have an easy way to spot those days. The dashboard view kind of gives that, but there is no single number of exceptions that I can use. And I greatly miss being able to have that count and graph on the same list of exceptions. Can you bring back that 1 day graph, complete with a total exception count?

zeeg 11y ago
We're definitely going to be improving things like the dashboard (and organization dash). I do agree regarding better visibility into what the graphs represent, and we've opened an issue to track this: https://github.com/getsentry/sentry/issues/2131 https://github.com/getsentry/sentry/issues/2131 (There's a few other related issues to this, but I think this addresses that core problem)

lobster_johnson 11y ago

Sentry looks cool. I have previously tested Rollbar and Airbrake, but was deeply unsatisfied with both. The big problem I have with all these services is that you need set up each application individually: Each one is a silo, with its own notification settings, API key, and so on. We have dozens of apps (several products + lots of microservices), so this is just unacceptable. I looked into whether I could use a single API key with Rollbar or Airbrake, and simply tag each event with the name of the sender, but it turns out this is not feasible. They don't even have APIs that let you programmatically configure apps remotely. How does Sentry compare? (Many other SaaS services suffer from the same issue; for example, Semaphore, which we use for CI, requires a build script for each project. We worked around this by letting the build script be a single curl command that pipes a remote script to bash, shared across all projects. But hardly ideal.)

zeeg 11y ago
We're pretty similar in a lot of ways. It's definitely a pain point, but from a UX perspective its tough to solve. In theory you could manage a lot of things with the API (most of our UI runs off the API these days). Even with the limitations, we have many people with hundreds of projects, and the general problems we see are more around notification settings than anything else. The usual suspect is that people are getting too many emails when they either want none, or they want digests. We are giving this a lot of thought as part of our upcoming infrastructure pass. From my point of view it'd make a lot of sense if you could just have a massive stream of events and sort through them however you want. That causes a lot of challenges so it may not be feasible. Either way its a problem we're aware of and looking to address. The simplest solution from our point of view is to allow you to configure some kind of organization-wide defaults. That won't resolve the API key constraints, but we don't feel that specifically is a pain point.
- atombender 11y ago
  I see. What I don't understand is why settings need to be so granular. For example, our company uses Slack. All notifications go to Slack. There's no need for per-project settings because it's all just one room. At some point we will probably want some projects to go to other rooms, but that's a coarse-grained exception, not something that warrants a per-project setting. Also, if you need per-person notification settings, then surely that's part of the user's account/profile, not the project. That would be just a single dashboard page with (say) on/off sliders for each project. In my mind, the data model used by exception trackers (including, it seems, Sentry) is wrong. There needs to be just one stream of events, tagged by things like application name that you can filter on. How those events map to things like notifications and aggregations and rules and source files and teams and so on is completely orthogonal to the stream. Those things then apply to a subset that you can modify dynamically without having to reconfigure the sender.

ryanpetrich 11y ago

The link at the bottom of the blog post leads to a 404

zeeg 11y ago
Doh! Fixed ;)

lobster_johnson 11y ago

Addendum to my last comment: There really should be a requirement that SaaS services supported some kind of load/dump configuration API, where you can deal with raw config. The admin UI should be completely optional. Most SaaS vendors don't have APIs that cover all parts of the service. Some do have decent APIs -- for example, we store our DNS records in files and use Gandi's APIs to update our zones, using a custom script (with some nice diffing capabilities), and we do some automatic configuring of Mailgun -- but APIs are always stateful and never declarative, which means that a client will always have to issue half a dozen calls to diff and converge its state. It's painful.

zeeg 11y ago
That's an interesting idea. We're Datadog users and one behavior I like is that, while they have a fancy way to build the charts, you can simply click an advanced mode and grab the full JSON configuration (thus being able to copy paste it). We'll give some thought into this, as it's not too far from being a possibility.

voltagex_ 11y ago

Can I come work for you?

zeeg 11y ago
We're always looking to expand the team, though our team is small so we may not always have open roles. That said feel free to throw your hat into the ring: https://www.getsentry.com/jobs/ https://www.getsentry.com/jobs/

robbiet480 11y ago

I already implemented releases a few weeks ago. They're great, thanks for that! I'm wondering what the reasoning is behind suggesting I upload source artifacts to you as part of a release. The blog post even specifically calls out Go, which is what the majority of our application is written in. So from my understanding, uploading artifacts would allow the Sentry UI to show me exactly where we did a Sentry Dump? On the same topic, our current deploy flow makes it a bit hard to upload specific code changes per release. It would be great if Sentry could get the code from Github directly (assuming that I login and give you access to the repo) Thanks for Sentry, we use it daily and would have a much harder time tracing errors and monitoring overall application health without it.

mattrobenolt 11y ago
> I'm wondering what the reasoning is behind suggesting I upload source artifacts to you as part of a release. The blog post even specifically calls out Go, which is what the majority of our application is written in. So from my understanding, uploading artifacts would allow the Sentry UI to show me exactly where we did a Sentry Dump? For Go, you ultimately compile your application into a binary. This binary doesn't have the source code, but it does retain references to the filenames and line number in the binary's metadata. What this means is, if you send along a stack trace to us, we can tell you that information, but we can't show you what the actual line of code is, or what the surrounding code is. By uploading the sources to us explicitly, then baking in the release version into your compiled binary, we can map those up and show you a stack trace that includes source code. This is relatively the same idea with any modern JavaScript. Typically a project is compiled down into a minified version, or run through a transpiler, etc, and the original sources aren't accessible to us. So it's then necessary to upload to us so we can map things up. > On the same topic, our current deploy flow makes it a bit hard to upload specific code changes per release. It would be great if Sentry could get the code from Github directly (assuming that I login and give you access to the repo) We have plans for VCS integrations in the future (GitHub included) that would likely work with releases so we can just use the working directory at a current sha.
- robbiet480 11y ago

crudbug 11y ago

Great work, I would suggest decoupling the front-end react components to create a separate widget library.

zeeg 11y ago
Our goal is definitely to decouple the frontend from the backend (entirely). We're still a ways out from that, but we've made a lot of progress.
- crudbug 11y ago
  +1