8 ms·

I gave up on self-hosted Sentry (2024)

186 points by roywashere 1y ago

XCSme 1y ago
This is what usually happens with self-hosted software that offers a cloud paid alternative. If it's too easy to install and maintain, the cloud version would make no money. For my self-hosted analytics platform [0], I chose to only use the LAMP stack (PHP/MySQL), as it's one of the easiest to set up and maintain. Yes, it won't be able to track 100 million events per day, but most people don't have that amount of traffic anyway. It runs on the cheapest VPS, and can be installed in many different ways (adding more and more, if the platform is self-hosted only, my goal is to make installation as simple as possible, especially with the free trial, so customers can try it anywhere in a few minutes). Sorry if it sounds like a rant, but it's hard competing with huge corporations promising free "self-hosted options", only to use it as a lead-magnet for the cloud service. I saw this trend growing, even with smaller companies, where their self-hosted one-click app installs fail and they provide no support for it or have no intention to fix the installer. [0]: https://www.uxwizz.com https://www.uxwizz.com
vanschelven 1y ago
Hey, that's me! When I posted this myself on Reddit, I said the following: I've long held off on actually posting this article to a platform like this one (don't bash your competition and all that), but "isn't Sentry self-hosted?" _is_ one of the most asked questions I get, and multiple people have told me this blog-post actually explains the rationale for Bugsink better than the rest of the site, so there you have it.
- yarekt 1y ago
  Well done! I came to the same conclusion (with the exact same bewilderment steps) as I do love Sentry myself. I will definitely try Bugsink, it’s something i’ve been looking for ages. Feedback on competition bashing: sometimes they deserve it, they should really just come out and say it: “open sourcing our stuff isn’t working for us, we want to keep making money on the hosting”, and that would be ok
  - zeeg 1y ago
    fwiw I was always pretty transparent about our priorities: https://blog.sentry.io/building-an-open-source-service/ https://blog.sentry.io/building-an-open-source-service/ We enable self-hosting because not everyone can use a cloud service (e.g. government regulation), otherwise we probably wouldn't even spend energy on it. We dont commercialize it at all, and likely never will. I strongly believe people should not run many systems themselves, and something that monitors your reliability is one such system. The lesson you learn building a venture backed company, and one that most folks miss: focus on growth, not cost-cutting. Self-hosting for many is a form of cost-cutting. We do invest in making it easier, and its 100% a valid complaint that the entire thing is awful today to self-host, and most people dont need a lot of the functionality we ship. Its not intentional by any means, its just really hard to enable a tiny-scale use-case while also enabling someone like Disney Plus.
    - yarekt 1y ago
      Despite my earlier comment, the way Sentry approaches open source is at least a little better than the majority, so its good that you at least provide an escape hatch that means heavy integration with Sentry isn't a complete vendor lock. On your second point, you're right that self-hosting is cost cutting, but I found that when a business is growing, the features that sentry offers are more of a nice to have along the way, not quite critical to its success (No point capturing errors if no-one's using your bloody thing). Its easy to rack up that monthly bill with nothing but nice to have hosted services. What I'd love to have is dumb versions of tools to self host initially when the requirements (and traffic) is very low. Kibana for example is a pig to self host, at one point it was taking up 25% of our production capacity. I found Loki is much better for simple cases in the beginning. Good example that strikes a balance is Grafana / Prometheus. Its practically impossible to run a software shop without them, and everywhere I worked went through the same phases: Chuck a set of random containers in prod -> Deploy helm chart -> Migrate to Thanos -> Move to hosted Grafana when user/teams management gets out of hand. Benefit is that absolutely everyone is familiar (and even likes) your tooling, I hope that's enough of a reason to give away a simpler offering.
- miyuru 1y ago
  Your project is awesome, you should do a show HN later.
  - vanschelven 1y ago
    Thanks for the kind words! In fact I did one last week, but it got only a fraction of today's article's traction... I'll try again in whatever the prescribed interval is :-)
    - apitman 1y ago
      You can submit 2-3 times over a couple days.
vanschelven 1y ago
also in recent news: https://it-notes.dragas.net/2024/12/28/i-almost-died-for-a-full-sentry-database/ https://it-notes.dragas.net/2024/12/28/i-almost-died-for-a-f...
- rmnclmnt 1y ago
  > Code needs to be written properly; you can’t just waste money and resources endlessly to cover up inefficiencies. Quite rare to hear this wise line these days. An I guess with AI coding assistant, this is only the beginning of this kind of horror story
  - nurettin 1y ago
    For me, the horror story started when people ditched optimal desktop apps for Electron because they knew js and css.
    - beng-nl 1y ago
      Perhaps we can make an AI translate electron apps into native apps…
    - poisonborz 1y ago
      So tired of hearing this trope. Electron is alright. Memory is cheap. Tell me a single better way to write cross-platform UI other than a worse version of Electron.
      - nurettin 1y ago
        Sure, but what do you think would count as better?
      - mdaniel 1y ago
        > Memory is cheap If you're running on commodity hardware, sure; if you happen to be a $1T company that solders 3x marked-up RAM then that's definitely not true https://www.apple.com/shop/buy-mac/macbook-pro/16-inch-space-black-standard-display-apple-m4-pro-with-14-core-cpu-and-20-core-gpu-24gb-memory-512gb https://www.apple.com/shop/buy-mac/macbook-pro/16-inch-space... is the entry model, and clicking the 48GB option dials the price up to $3k
stebian_dable 1y ago
FOSS Sentry fork GlitchTip keeps things more simple and self-hosting friendly. https://glitchtip.com/ https://glitchtip.com/
- crimsonnoodle58 1y ago
  +1 for Glitchtip. We also found the same problem as OP with self hosting sentry. Each release would unleash more containers and consume more memory until we couldn't run anything on the 32gb server except Sentry. We looked at both GlitchTip and BugSink and only settled on GlitchTip as it was maintained by a bigger team. Feature wise they were quite similar and both good alternatives. So far so good with GlitchTip. And thanks Op for making BugSink, the more alternatives the better.
- vanschelven 1y ago
  Indeed it does! Although with Bugsink (which is what came out of this origin story of annoyance) I'm aiming for _even more_ simple (1 Docker container at minimum, rather than 4), 30 seconds up and running etc.
  - bufke 1y ago
    Hello, ping me (GlitchTip lead) if you want to collaborate. Your stack is also Django. I'd be open to simplifying - we could probably make Redis optional. I have an experimental script that runs celery and Django in one process. But I think Postgres is a must have. So that gets down to two. My hope is that GlitchTip works for super small use cases and scales with minimal thought required.
- anonzzzies 1y ago
  We run them both as we are evaluating glitchtip, but, at least for us, it has so many bugs vs sentry. But it's so much lighter so we try to stick with it.
  - npodbielski 1y ago
    Can you say what kind of bugs? This application looks interesting; I want to try it instead of https://healthchecks.io/ https://healthchecks.io/
whatnow37373 1y ago
> I’m not going to run a piece of software that requires 16GB of RAM, has a complex installation script, and is known to be a pain to maintain. This has to be self-hosted eventually either by you or Sentry themselves so the full cost of this is coming down somewhere. The planet is round and all that and there's no getting away from these inefficiencies, but you can kick the proverbial can down the road. Also, they are incentivized to make the open product as hard to use as possible so color me surprised the self-hosted edition is hard to operate.
- vanschelven 1y ago
  > color me surprised last time a similar thing was pointed out on HN Armin Ronacher (former Sentry) came out and pointed at the following issue: https://github.com/getsentry/team-ospo/issues/232 https://github.com/getsentry/team-ospo/issues/232 but that is now closed with > We are no longer working on this effort for now
  - the_mitsuhiko 1y ago
    I no longer work at Sentry but the complexity of operating Sentry is something that we still worked on when I was there. There is no secret incentive to make Sentry hard to operate. You have to consider that engineers run a version of Sentry on their machines too and any complexity ends up there too. One of the biggest pain points of running Sentry is the number of containers and a lot of this comes from how Sentry works with Kafka and Rabbit. That pain point is actively being addressed by moving to a virtualized system [1] though I am not sure how long it will take to actually adopt this. [1]: https://github.com/getsentry/taskbroker/ https://github.com/getsentry/taskbroker/
    - vanschelven 1y ago
      Hi Armin :-) I've seen you make this point on similar posts in the past, and I believe that you believe it. The counterpoint would be that "the purpose of a system is what it does"...
Weryj 1y ago
We self-host sentry in Hetzner, but with a high-end server. 96c, 512gb. It ends up only costing around $300 a month, however with the scale of events that it processes, the managed version would be in the 10's of thousands. The overhead at low volume is pretty high, but in the higher volumes (25M transactions/24h) it's a massive cost saving for us. Edit: There were just some initial headaches with needing to increase kafka partitions and add replications to the transaction processors, otherwise we didn't quite leverage the available compute and the backpressure would fill Redis up until OOM.
- vanschelven 1y ago
  Yeah I fully get how that's a volume where going self-hosted Sentry makes perfect sense at the bottom line and including any upkeep you might have. Bugsink's also quite scalable[0], but I wouldn't recommend it a 25M/day. [0] https://www.bugsink.com/scalable-and-reliable/ https://www.bugsink.com/scalable-and-reliable/
  - selcuka 1y ago
    > Bugsink's also quite scalable[0], but I wouldn't recommend it a 25M/day. Well, your homepage disagrees with this statement: > Bugsink can deal with millions of events per day on dirt cheap hardware
    - kelnos 1y ago
      To me, "millions" usually means less than 10M. 25M falls into "tens of millions" to me. But it's a very fuzzy way of quantifying something, and open to various interpretations.
      - selcuka 1y ago
        Fair point, but if it can cope with millions on "dirt cheap hardware" it logically follows that it can do 25M on more expensive hardware.
- pebble 1y ago
precommunicator 1y ago
We've just applied a helm chart a while back. It just works. We maybe had like a few incidents over the years, requiring stuff like Kafka queues to be wiped. The argument that you have to read a sh script doesn't make sense to me. Are you gonna read source code of any software is referenced in this script or any you download too? No? What's the difference between that and a bash script, at the end of the day both can do damage.
- xyzzy123 1y ago
  We used the helm chart but things didn't get updated often enough to keep our container security stuff happy. Helm is a huge pain in the butt if you have mitigation obligations because the overall supply chain for a 1-command install can involve several different parties, who all update things at different frequencies :/ So chart A includes subchart B, which consumes an image from party C, who haven't updated to foobar X yet. You either need to wait for 3 different people to update stuff to get mainline fixed, or you roll up your sleeves and start rebuilding things, hosting your own images and forking charts. At first you build 1 image and set a value but the problem grows over time. If you update independently you end up running version combinations of software that the OG vendor has never tested. This is not helm's fault of course; it's just the reality of deploying software with a lot of moving parts.
- vanschelven 1y ago
  Rereading that section, I'd agree it's probably not the best-argued point because it implies security concerns... I guess what I'm saying is: for something I'm setting up to keep around for a while, I'd like to know a bit what's in the package before I deploy it. In that sense, the shell script serves as a table of contents... and if the table of contents is 800 lines, that makes me wonder how many moving parts there are and how many things might break at inconvenient times because of that.
  - precommunicator 1y ago
    For me I would just run it on a clean cluster/VM somewhere (to be destroyed after that) just to see what happens. If you have no local resources to spare, an hour of even very high end (to save time) VMs/cluster at a provider e.g. AWS costs next to nothing
Brosper 1y ago
Maybe Sentry don't want you to have self-hosted version?
- neuroelectron 1y ago
  What gave you that impression?
  - mdaniel 1y ago
    I didn't downvote you but as one metric their docker-compose is 500 lines long https://github.com/getsentry/self-hosted/blob/25.4.0/docker-compose.yml#L500 https://github.com/getsentry/self-hosted/blob/25.4.0/docker-... as compared to back in the good-old days when it was basically python, redis, and postgres https://github.com/getsentry/self-hosted/blob/9.1.2/docker-compose.yml#L16-L19 https://github.com/getsentry/self-hosted/blob/9.1.2/docker-c...
- conradfr 1y ago
  I actually learn from this post that there is a self-hosted version...
  - Brosper 1y ago
    I talked with them. They just offer it but don't offer any support for it.
croes 1y ago
From the Sentry homepage > Application monitoring software considered "not bad" by 4 million developers. Sounds pretty bad to me
- gardnr 1y ago
  As a software developer I can say that it is not bad. It's pretty great, actually.
  - croes 1y ago
    But the infrastructure behind sounds complicated and fragile.
    - zeeg 1y ago
      No, its complicated and scales to servicing every customer in the world. Thats not the same thing. Doesnt mean the complaints about self-hosted arent valid, but "literally has to scale to the most insane volumes of data" and "is not good software" are two different things. We're building a cloud service at the end of the day - its a lot easier to optimize a multi-tenant install than it is a single-tenant install, and that complexity shows up in the self-hosted repo. Can't really avoid it.
mplantsheer 1y ago
The amount of shell script that needs to be executed to install is a bit of a no no for me. It also doesn't make sense to spin up a 16GB machine (minimum!) to track the errors on those 4-8GB VPS which are running my production services.
epolanski 1y ago
The biggest issue I have with these solutions is indeed local debugging. I use Sentry with most of my clients, and for effective debugging I need to spin my own Sentry in a Docker container which ends up being quite heavy on my machine especially when combined with Grafana and Prometheus. I'm really unhappy with virtually all monitoring/telemetry/tracking solutions. It really feels they are all designed to vendor lock you in their expensive cloud solutions and I really don't feel I'm getting my $s back at all. Worst of all those cloud vendors would rather add new features non-stop rather than honing what they currently have. Their sales are everywhere, I've seen two different clients getting Datadog sales people join private Slacks to evangelize their products. Both times I escalated to the CTO, both times I ended up suspecting someone in management had something to gain from pushing teams to adopt those solutions.
- vanschelven 1y ago
  I actually wrote about that scenario! Killing flies with hammers and all, but since I really like my hammer I actually do all my local development with my full-blown error tracker too: https://www.bugsink.com/blog/local-development/ https://www.bugsink.com/blog/local-development/
- delusional 1y ago
  Sentry's sales team is incredibly aggressive. I've seen multiple colleagues hijacked for sales presentations over a few months. It would not surprise me at all if they just asked random employees to be added to the company slack, and even less if those people then just did it. I can only commend the hustle on their part, but it does feel a little like a high pressure time share situation.
  - zeeg 1y ago
    If you - or anyone reading this - ever ends up in a situation where we came off as aggressive send me a direct email and I will take care of it. This is not something we believe in at Sentry, and while you cant manage everything, its important to us that we never become "of those companies" like so many other successful companies become. david at sentry.io
adamcharnock 1y ago
This absolutely mirrors my experience. Sentry was straightforward to deploy years ago, but now seems like one of the more egregious offenders in the, 'self host-able but increasingly prohibitively complex by design' category. As others have said, we've [0] found the only practical way to deploy this for our clients is Kuberentes + Helm chart, and that's on bare-metal servers (mostly Hetzner). It runs well if you can throw hardware and engineering time at it, which thankfully we can. But given the option we would love a simpler solution. [0]: https://lithus.eu https://lithus.eu
- wg0 1y ago
  And how do you install and maintain/upgrade kubernetes? Are you running databases also on kubernetes?
  - adamcharnock 1y ago
    In our case we have a collection of Ansible roles we use for the purpose. We run databases using the Stackgres operator either using logical replication on local fast NVMe dives, on top of OpenEBS/Mayastor replicated block-storage. But we specialise in this so that our clients don't have to. As much as I do actually love Kubernetes, the fact that the _easiest_ way to self-host Sentry is via Kubernetes is not a good sign. And choosing to spin up a Kubernetes cluster just to run Sentry would feel a lot like the lady who swallowed a fly[0]. [0]: https://en.wikipedia.org/wiki/There_Was_an_Old_Lady_Who_Swallowed_a_Fly https://en.wikipedia.org/wiki/There_Was_an_Old_Lady_Who_Swal...
    - apexalpha 1y ago
      Thanks for the poem, it seems pretty apt for IT in 2025. That said I would honestly prefer if the industry would just settle on K8s as our OS. I really do not see any benefit that sentry could bring on its own compared to a solid set of Helm charts for k8s.
    - baq 1y ago
      Does it really have to be a proper cluster, though? Can it be e.g. a single node k3s?
ghh 1y ago
This isn't okay - the author is selling their own alternative to Sentry, 'reusing' Sentry's open-source client SDK's, while spreading FUD about self-hosting Sentry. I've been self-hosting Sentry for over 10 years: Sentry is installed by running `git clone`, editing one or two configuration files, and running ./install.sh. It requires 16GB RAM if you enable the full feature set. That includes automatically recording user sessions on web or mobile for replay, and support for end-to-end tracing (so seeing which database queries your api asks for in response to a button tap in your mobile app). Sentry has a wonderful self-hosting team that's working hard to make Sentry's commercial feature set available, for free, to everyone that needs a mature error tracking solution. You can talk to them on discord and they listen and help.
- roywashere 1y ago
  Just for transparency, are you by any chance a member of this self-hosting team or a sentry employee? Is it a coincidence that your keybase name is georghendrik according to your profile, and the painter _Georg Hendrik_ Breitner painted a picture called "Sentry"? https://www.demesdagcollectie.nl/en/collection/hwm0055 https://www.demesdagcollectie.nl/en/collection/hwm0055
  - nevertoolate 1y ago
    I was born 400 years ago in the Highlands of Scotland. I am Immortal, and I am not alone. Now is the time of the Gathering, when the stroke of the sword will release the power of the Quickening. In the end, there can be only one.
  - Biganon 1y ago
    This is some Olympics level of badly coping with valid and welcome criticism. Georg Hendrik = "George Henry", pretty common name. The fact that Google returned a result when you searched "Georg Hendrik Sentry" should not be considered weird.
- vanschelven 1y ago
  TBH most of the FUD in the OP is straight from Sentry's own website. Regarding using the SDKs, I'm telling my users to take Sentry at their word when they wrote "Permission is hereby granted, free of charge [..] to deal in the Software without restriction, including without limitation the rights to use"
tanepiper 1y ago
This is also my opinion of Backstage. It's positioned as this easy-to-use open source tool for IDPs - but in reality, it's a product that needs a team to maintain it constantly.
- mtndew4brkfst 1y ago
  It has its own conference now! Definitely not a signal for something that is reliable set-it-and-forget-it tech.
fefe23 1y ago
I find it very funny that the post is basically complaining about fearmongering by the maker of Sentry to scare people into not hosting themselves. And then he does the exact same thing, on behalf of Sentry. I hope he got paid for this. Otherwise it would just be sad.
- Timshel 1y ago
  I doubt he would have been paid to promote a competing service ..
  - vanschelven 1y ago
    If that were to happen I think the blog post (guaranteed virality) would just write itself...
notpushkin 1y ago
I still love Sentry, but it’s so enormous now that it isn’t practical to self-host for smaller businesses. A “small” alternative is always great to see! I’m not sure how I feel about the license though (Polyform Shield, basically use-but-don’t-compete). It’s a totally valid choice – I just wish it would convert to FOSS at some point. (I understand the concern as I’ve had to make a similar decision when releasing https://lunni.dev/ https://lunni.dev/. I went with AGPL, but I’m still overthinking it :-)
mayli 1y ago
I am not sure why there isn't a even lightweight sentry endpoint that does - save post body to folders (use uuid as folder name to avoid spam) - dir listing, and count number of entries - render posted json to html, highlight stacktrace with js - download raw json - rotate, compress old entries. I give those requirements to LLM, and I get a pretty much working rust implementation after few tweaks. It uses <5M ram idle.
- hobofan 1y ago
  That covers the most basic feature that Sentry has (one-off error reporting, with rudimentary statistics). Sure, if that's enough for you then go ahead, but for most people using Sentry that's only the tip of the iceberg.
seanwilson 1y ago
> we don’t necessarily recommend self-hosting to everyone. In addition to existing hidden costs, as Sentry evolves, our self-hosted version will become more complex, demanding additional types of infrastructure. Any insights on why Sentry is so complex and needs so much resources? Is collecting, storing, and organizing errors messages and stack traces at scale difficult? Or it's the other features on top of this?
- eitland 1y ago
  Some ideas: - they had enough money that they never needed to think seriously about maintenence cost, and the sales process was strokg enough to keep customers arriving anyway (look to Oracle for another example of hopelessly complicated installation process but people keep using it anyway) - at some point someone realized this was actually a feature: the more complicated it got, the harder it became to self host. And from that perspective it is a win-win for the company: they can claim it is open source without being afraid that most people will choose to self host.
  - vanschelven 1y ago
    I'd say that the architecture that is desirable from the point of view of a large scale SaaS is very different than the one that's desirable from the point of view of a tool that just needs to work for a single organization. And since the SaaS is bringing in all the money, that's where the architecture follows. > actually a feature I would guess that for a few people people (e.g. the ones who made the scary visual of rising costs) this is explicitly so, but for most people it's more implied. i.e. I don't think anyone advanced their career with Sentry by making self-hosting easier.
    - BYK 1y ago
      Well I actually did as I was literally hired for that and worked on it for almost 3 years :)
- slyall 1y ago
  Anything gets complex at scale. This is the same software they us to host their SaaS system. Presumably has to scale to many thousands of customer and a huge number of events per second. They have all sorts of caching, autoscaling, distributed systems and other stuff thats complete overkill for all except that largest installation. Plus all sorts of software features only needed by a few customers and extra layers to be multi-customer. It's the difference between a hoop in your back yard and a NBA Stadium
deleted 1y ago
[deleted]
h1fra 1y ago
That's the inevitable evolution of any popular open-source product/company. Light and enterprise-ready is hardly compatible, and you can expect this from any project that has a cloud offering.
- fuzzfactor 1y ago
  Not only open source. I don't have servers like this but the author makes it easy to understand, and it applies to a lot of other things. The Comparative Costs picture does tell it all. The purpose of so much software is to reduce human and machine costs through time, and this apparently turned out to do just the opposite, apparently after long-term testing under real-world conditions. Could be an unsurmountable fundamental structure of technical debt or something like that which metastasizes. Then this, anyone could say about anything: >I’m not going to run a piece of software that requires 16GB of RAM, has a complex installation script, and is known to be a pain to maintain. I’m not going to recommend it to anyone else either. Easier said than done, sometimes it's the only option. It's a "script". Maybe that's why you have to "rehearse" it more than once before you barely get it right, and then you might have to really go the extra mile (and have a bit of good fortune) before you can achieve a "command performance". How do you think it feels these days to have to settle for this kind of thing in expensive proprietary software too?
madeofpalk 1y ago
It's wild that Sentry themselves are spreading literal FUD about about bad it is to self-host their own product. So confounding that they're so ready to shit on their own product.
elAhmo 1y ago
So many services are going this route. I remembered Reddit had an install script and almost everything worked out of the box 10-15 years ago on a Ubuntu VPS.
KronisLV 1y ago
A good APM tool I’ve been using for a few years is Apache Skywalking: https://skywalking.apache.org/ https://skywalking.apache.org/ It might not do everything Sentry does but it definitely has helped with tracking down some issues, even production ones and runs in a fairly manageable setup (in comparison to how long even the Sentry self-hosted Docker Compose file is). What’s more, if you want, you can even use regular PostgreSQL as the backing data store (might not be quite as efficient as ElasticSearch for a metrics use case, but also doesn’t eat your RAM like crazy).
- mdaniel 1y ago
  Any product that uses etcd is immediately off my radar[1]; it is nice of them to have a whole page dedicated to how to babysit the PoS https://skywalking.apache.org/docs/skywalking-banyandb/next/operation/cluster/#etcd-node-failure https://skywalking.apache.org/docs/skywalking-banyandb/next/... 1: yes, I'm a huge k8s fanboi and yes I long every day for them to allow me to swap out etcd for something sane
  - KronisLV 1y ago
    That’s kind of the beauty of it: you can opt for a different storage solution too https://skywalking.apache.org/docs/main/v10.2.0/en/setup/backend/backend-storage/ https://skywalking.apache.org/docs/main/v10.2.0/en/setup/bac... Personally, no hate towards their BanyanDB but after getting burnt by OrientDB in Sonatype Nexus, I very much prefer more widespread options.
afro88 1y ago
> First, there’s a signal being given about the complexity and fragility of the software. A 16GB RAM requirement just screams “this is a big, complex piece of software that will break in mysterious ways”. Not very scientific, I know, but based on experience. This lines up with my experience self hosting a headless BI service. In "developer mode" it takes maybe 1GB RAM. But as soon as you want to go prod you need multiple servers with 4+ cores and 16GB+ RAM that need a strongly consistent shared file store. Add confusing docs to the mix, mysterious breakages, incomplete logging and admin APIs, and a reliance on community templates for stuff like k8s deployment... it was very painful. I too gave up on self hosted.
- AdrianB1 1y ago
  In most corporate projects that I worked with, the day it went to production started the clock to disband the core team and optimizations were not on the table. When the product works, they move on. The only area where performance optimization is a mandatory step is video games, but that is where your clients are external. Most business software is in the "finally, but barely works" state and new features beat performance improvements. This is caused by short sighted management that need to deliver and move on. "Long term" is a contradiction with their business model. In this case "long term" means "after product launch".
BenGosub 1y ago
I worked in a startup around 2019-2020 where we started using Sentry for distributed tracing. It was a company in the IOT space and we had a lot of events going through. The bill started going in the $3-5000 range just from the tracing, so we decided to host it on our own. When looking at the sheer complexity of the project, I was flabbergasted. We need a name for managing the infrastructure of company based open source, self-hosted solutions. Often times, the right choice would be to choose a different, simpler, open source solution. In this case there are some neat distributed tracing solutions, that are easier to manage.
exiguus 1y ago
I'm a big fan of self-hosting, but it requires the right infrastructure. You need a robust virtualization solution like Proxmox, equipped with sufficient CPU cores and RAM. For instance, allocating 16GB of RAM shouldn't be an issue if your server has 128GB to share. In my experience, solutions like Mailcow, which involve multiple services and containers (such as SMTP, IMAP, Redis, SSO, webmail, Rspamd, etc.), work very well. I have extensive experience running these systems, performing backups, restoring data, and updating their ecosystems. Additionally, I've had a positive experience setting up and running a self-hosted Sentry instance with Docker for a project that spanned several years. However, this experience might be somewhat outdated, as it was a few years ago.
- npodbielski 1y ago
  I am running Mailcow on just some VM with docker. No Proxmox needed. So far it works.
  - exiguus 1y ago
    Sure, that topic was more about the 16GB of RAM needed for sentry. If you are already have shared RAM, 16GB sounds much more do-able.
azthecx 1y ago
At first glance it's not immediately obvious to me, why would you pick Sentry or Bugsink over something included in the Grafana stack? What's the use use
- mdaniel 1y ago
  I haven't used Sentry's obserability portions, but the error capture has very strong deduplication built into it, and can optionally integrate with your SCM, CI, and issue tracker, if one wishes. Thus, the flow can go like: - error happens, can be attributed to release 1.2.3 - every subsequent time that error happens to a different user, it can track who was affected by it, without opening a new error report - your project can opt-in to accepting end-user feedback on error: "please tell us what you were doing when this exploded, or feel free to rant and rave, we read them all" - it knows from the stack trace that the error is in src/kaboom/onoz.py line 55 - onoz.py:55 was last changed by claude@example.com last week, in PR #666 - sentry can comment upon said PR to advise the reviewers of the bad outcome - sentry can create a Jira with the relevant details - claude.manager@example.com can mark the bug as "fixed in the next release", which will cause sentry to suppress chirping about it until it sees a release 1.2.4 - if it happens again it will re-open the prior error report, marking it as a regression Unless you know something I don't, Grafana does *ABSOLUTELY NONE* of that
  - azthecx 1y ago
    Thanks for the detailed response
cuu508 1y ago
I was looking into moving away from managed Sentry for GDPR reasons. I was only using Sentry for notifications about unhandled exceptions in my app. All the fancy APM features I did not use. Grouping of similar exceptions, pretty display of tracebacks were nice but not essential. In my Django app I wrote a logging handler that stores the log records (including traceback) in a database table. I can inspect the log records through Django admin, and a cron job sends me daily emails saying "X new log records in the last 24 hours" so I know to check them out. And that's it :-) Of course, this does a lot less than Sentry, and has various limitations (e.g. what if the error is about the database being down...), but it fits my needs. BTW, IIUC, Sentry in its early beginnings was also doing just that – logging to database: https://github.com/dcramer/django-db-log https://github.com/dcramer/django-db-log
rwky 1y ago
I use both hosted and self hosted sentry. I prefer hosted it's less to manage but self hosted isn't too awful as long as you can budget the resources. If you just want the team sentry plan it's going to be cheaper to use hosted. I would only self host if you had to for legal/compliance reasons. In the many years I've been managing sentry we've only had it truly crap out once and that was an upgrade from 9 to their new date versioning basically the whole hosted method changed and it was just easier to start over from scratch.
linsomniac 1y ago
TL;DR: Not so much "gave up" as "never tried" because I was scared away by 16GB RAM requirement. FWIW, we've been self-hosting 2 instances (and purchase a third hosted instance), for, it looks like 8 years now, and it's had a few small bumps but hasn't been too bad. Our instances are running with 20GB of RAM, so pretty small. ~90% of updates have gone smoothly, which leaves 10% of updates that have had some sort of problem. Solutions have been found in the issue tracker in all our cases. We are a little afraid of doing updates, but do them every couple of months. Sentry is an amazing piece of software and it is fantastic that they offer a self-hosing version, and all things considered it is a fairly easy self-host.
- linsomniac 1y ago
  FYI: Just did the upgrade from 25.2.0 to 25.4.0 and it went smoothly.
- vanschelven 1y ago
  You're right, but so am I... :-) I "gave up" from the perspective of returning to Sentry after a couple of years, and finding an entirely different beast from the tool I loved before. At that point I indeed didn't make it past Sentry's own FUD.
1vuio0pswjnm7 1y ago
As a computer user I do not allow connections to sentry.io For example, Firefox tries to connect
- mdaniel 1y ago
  Heh, saying that in a thread about self hosting Sentry is pretty funny :-) --- IMHO, the RUM tooling[1] is the worst offender, tracking mouse movement and keystrokes. At least Sentry is usually just trying to report JS kabooms 1: https://docs.datadoghq.com/real_user_monitoring/session_replay/ https://docs.datadoghq.com/real_user_monitoring/session_repl... and https://docs.newrelic.com/docs/browser/browser-monitoring/browser-pro-features/session-replay/ https://docs.newrelic.com/docs/browser/browser-monitoring/br... et al
johnklos 1y ago
Software that's only available as x86 binaries and a whole system that needs 16 gigs of memory? It's not surprising that it comes from the corporate world and not the open source world. I respect the fact that the Sentry people are honest about their teetering tower.
m463 1y ago
> It’s a ton of docker containers. Things will fail randomly or maybe with a lot of traffic, don’t remember well. maybe they should put in a system to monitor the docker containers.
domysee 1y ago
HyperDX is a great alternative to Sentry, much easier to self-host, and also open source. It's relatively new and did take some tinkering to make it work properly, so I wrote a short article about it: https://weberdominik.com/blog/self-host-hyperdx https://weberdominik.com/blog/self-host-hyperdx But the feature set and user experience is great!

186 points by roywashere 1y ago

This is what usually happens with self-hosted software that offers a cloud paid alternative. If it's too easy to install and maintain, the cloud version would make no money. For my self-hosted analytics platform [0], I chose to only use the LAMP stack (PHP/MySQL), as it's one of the easiest to set up and maintain. Yes, it won't be able to track 100 million events per day, but most people don't have that amount of traffic anyway. It runs on the cheapest VPS, and can be installed in many different ways (adding more and more, if the platform is self-hosted only, my goal is to make installation as simple as possible, especially with the free trial, so customers can try it anywhere in a few minutes). Sorry if it sounds like a rant, but it's hard competing with huge corporations promising free "self-hosted options", only to use it as a lead-magnet for the cloud service. I saw this trend growing, even with smaller companies, where their self-hosted one-click app installs fail and they provide no support for it or have no intention to fix the installer. [0]: https://www.uxwizz.com https://www.uxwizz.com

vanschelven 1y ago

Hey, that's me! When I posted this myself on Reddit, I said the following: I've long held off on actually posting this article to a platform like this one (don't bash your competition and all that), but "isn't Sentry self-hosted?" _is_ one of the most asked questions I get, and multiple people have told me this blog-post actually explains the rationale for Bugsink better than the rest of the site, so there you have it.

yarekt 1y ago
Well done! I came to the same conclusion (with the exact same bewilderment steps) as I do love Sentry myself. I will definitely try Bugsink, it’s something i’ve been looking for ages. Feedback on competition bashing: sometimes they deserve it, they should really just come out and say it: “open sourcing our stuff isn’t working for us, we want to keep making money on the hosting”, and that would be ok
- zeeg 1y ago
  fwiw I was always pretty transparent about our priorities: https://blog.sentry.io/building-an-open-source-service/ https://blog.sentry.io/building-an-open-source-service/ We enable self-hosting because not everyone can use a cloud service (e.g. government regulation), otherwise we probably wouldn't even spend energy on it. We dont commercialize it at all, and likely never will. I strongly believe people should not run many systems themselves, and something that monitors your reliability is one such system. The lesson you learn building a venture backed company, and one that most folks miss: focus on growth, not cost-cutting. Self-hosting for many is a form of cost-cutting. We do invest in making it easier, and its 100% a valid complaint that the entire thing is awful today to self-host, and most people dont need a lot of the functionality we ship. Its not intentional by any means, its just really hard to enable a tiny-scale use-case while also enabling someone like Disney Plus.
  - yarekt 1y ago
    Despite my earlier comment, the way Sentry approaches open source is at least a little better than the majority, so its good that you at least provide an escape hatch that means heavy integration with Sentry isn't a complete vendor lock. On your second point, you're right that self-hosting is cost cutting, but I found that when a business is growing, the features that sentry offers are more of a nice to have along the way, not quite critical to its success (No point capturing errors if no-one's using your bloody thing). Its easy to rack up that monthly bill with nothing but nice to have hosted services. What I'd love to have is dumb versions of tools to self host initially when the requirements (and traffic) is very low. Kibana for example is a pig to self host, at one point it was taking up 25% of our production capacity. I found Loki is much better for simple cases in the beginning. Good example that strikes a balance is Grafana / Prometheus. Its practically impossible to run a software shop without them, and everywhere I worked went through the same phases: Chuck a set of random containers in prod -> Deploy helm chart -> Migrate to Thanos -> Move to hosted Grafana when user/teams management gets out of hand. Benefit is that absolutely everyone is familiar (and even likes) your tooling, I hope that's enough of a reason to give away a simpler offering.
miyuru 1y ago
Your project is awesome, you should do a show HN later.
- vanschelven 1y ago
  Thanks for the kind words! In fact I did one last week, but it got only a fraction of today's article's traction... I'll try again in whatever the prescribed interval is :-)
  - apitman 1y ago
    You can submit 2-3 times over a couple days.

vanschelven 1y ago

also in recent news: https://it-notes.dragas.net/2024/12/28/i-almost-died-for-a-full-sentry-database/ https://it-notes.dragas.net/2024/12/28/i-almost-died-for-a-f...

rmnclmnt 1y ago
> Code needs to be written properly; you can’t just waste money and resources endlessly to cover up inefficiencies. Quite rare to hear this wise line these days. An I guess with AI coding assistant, this is only the beginning of this kind of horror story
- nurettin 1y ago
  For me, the horror story started when people ditched optimal desktop apps for Electron because they knew js and css.
  - beng-nl 1y ago
    Perhaps we can make an AI translate electron apps into native apps…
  - poisonborz 1y ago
    So tired of hearing this trope. Electron is alright. Memory is cheap. Tell me a single better way to write cross-platform UI other than a worse version of Electron.
    - nurettin 1y ago
      Sure, but what do you think would count as better?
    - mdaniel 1y ago
      > Memory is cheap If you're running on commodity hardware, sure; if you happen to be a $1T company that solders 3x marked-up RAM then that's definitely not true https://www.apple.com/shop/buy-mac/macbook-pro/16-inch-space-black-standard-display-apple-m4-pro-with-14-core-cpu-and-20-core-gpu-24gb-memory-512gb https://www.apple.com/shop/buy-mac/macbook-pro/16-inch-space... is the entry model, and clicking the 48GB option dials the price up to $3k

stebian_dable 1y ago

FOSS Sentry fork GlitchTip keeps things more simple and self-hosting friendly. https://glitchtip.com/ https://glitchtip.com/

crimsonnoodle58 1y ago
+1 for Glitchtip. We also found the same problem as OP with self hosting sentry. Each release would unleash more containers and consume more memory until we couldn't run anything on the 32gb server except Sentry. We looked at both GlitchTip and BugSink and only settled on GlitchTip as it was maintained by a bigger team. Feature wise they were quite similar and both good alternatives. So far so good with GlitchTip. And thanks Op for making BugSink, the more alternatives the better.
vanschelven 1y ago
Indeed it does! Although with Bugsink (which is what came out of this origin story of annoyance) I'm aiming for _even more_ simple (1 Docker container at minimum, rather than 4), 30 seconds up and running etc.
- bufke 1y ago
  Hello, ping me (GlitchTip lead) if you want to collaborate. Your stack is also Django. I'd be open to simplifying - we could probably make Redis optional. I have an experimental script that runs celery and Django in one process. But I think Postgres is a must have. So that gets down to two. My hope is that GlitchTip works for super small use cases and scales with minimal thought required.
anonzzzies 1y ago
We run them both as we are evaluating glitchtip, but, at least for us, it has so many bugs vs sentry. But it's so much lighter so we try to stick with it.
- npodbielski 1y ago
  Can you say what kind of bugs? This application looks interesting; I want to try it instead of https://healthchecks.io/ https://healthchecks.io/

whatnow37373 1y ago

> I’m not going to run a piece of software that requires 16GB of RAM, has a complex installation script, and is known to be a pain to maintain. This has to be self-hosted eventually either by you or Sentry themselves so the full cost of this is coming down somewhere. The planet is round and all that and there's no getting away from these inefficiencies, but you can kick the proverbial can down the road. Also, they are incentivized to make the open product as hard to use as possible so color me surprised the self-hosted edition is hard to operate.

vanschelven 1y ago
> color me surprised last time a similar thing was pointed out on HN Armin Ronacher (former Sentry) came out and pointed at the following issue: https://github.com/getsentry/team-ospo/issues/232 https://github.com/getsentry/team-ospo/issues/232 but that is now closed with > We are no longer working on this effort for now
- the_mitsuhiko 1y ago
  I no longer work at Sentry but the complexity of operating Sentry is something that we still worked on when I was there. There is no secret incentive to make Sentry hard to operate. You have to consider that engineers run a version of Sentry on their machines too and any complexity ends up there too. One of the biggest pain points of running Sentry is the number of containers and a lot of this comes from how Sentry works with Kafka and Rabbit. That pain point is actively being addressed by moving to a virtualized system [1] though I am not sure how long it will take to actually adopt this. [1]: https://github.com/getsentry/taskbroker/ https://github.com/getsentry/taskbroker/
  - vanschelven 1y ago
    Hi Armin :-) I've seen you make this point on similar posts in the past, and I believe that you believe it. The counterpoint would be that "the purpose of a system is what it does"...

Weryj 1y ago

We self-host sentry in Hetzner, but with a high-end server. 96c, 512gb. It ends up only costing around $300 a month, however with the scale of events that it processes, the managed version would be in the 10's of thousands. The overhead at low volume is pretty high, but in the higher volumes (25M transactions/24h) it's a massive cost saving for us. Edit: There were just some initial headaches with needing to increase kafka partitions and add replications to the transaction processors, otherwise we didn't quite leverage the available compute and the backpressure would fill Redis up until OOM.

vanschelven 1y ago
Yeah I fully get how that's a volume where going self-hosted Sentry makes perfect sense at the bottom line and including any upkeep you might have. Bugsink's also quite scalable[0], but I wouldn't recommend it a 25M/day. [0] https://www.bugsink.com/scalable-and-reliable/ https://www.bugsink.com/scalable-and-reliable/
- selcuka 1y ago
  > Bugsink's also quite scalable[0], but I wouldn't recommend it a 25M/day. Well, your homepage disagrees with this statement: > Bugsink can deal with millions of events per day on dirt cheap hardware
  - kelnos 1y ago
    To me, "millions" usually means less than 10M. 25M falls into "tens of millions" to me. But it's a very fuzzy way of quantifying something, and open to various interpretations.
    - selcuka 1y ago
      Fair point, but if it can cope with millions on "dirt cheap hardware" it logically follows that it can do 25M on more expensive hardware.
pebble 1y ago

precommunicator 1y ago

We've just applied a helm chart a while back. It just works. We maybe had like a few incidents over the years, requiring stuff like Kafka queues to be wiped. The argument that you have to read a sh script doesn't make sense to me. Are you gonna read source code of any software is referenced in this script or any you download too? No? What's the difference between that and a bash script, at the end of the day both can do damage.

xyzzy123 1y ago
We used the helm chart but things didn't get updated often enough to keep our container security stuff happy. Helm is a huge pain in the butt if you have mitigation obligations because the overall supply chain for a 1-command install can involve several different parties, who all update things at different frequencies :/ So chart A includes subchart B, which consumes an image from party C, who haven't updated to foobar X yet. You either need to wait for 3 different people to update stuff to get mainline fixed, or you roll up your sleeves and start rebuilding things, hosting your own images and forking charts. At first you build 1 image and set a value but the problem grows over time. If you update independently you end up running version combinations of software that the OG vendor has never tested. This is not helm's fault of course; it's just the reality of deploying software with a lot of moving parts.
vanschelven 1y ago
Rereading that section, I'd agree it's probably not the best-argued point because it implies security concerns... I guess what I'm saying is: for something I'm setting up to keep around for a while, I'd like to know a bit what's in the package before I deploy it. In that sense, the shell script serves as a table of contents... and if the table of contents is 800 lines, that makes me wonder how many moving parts there are and how many things might break at inconvenient times because of that.
- precommunicator 1y ago
  For me I would just run it on a clean cluster/VM somewhere (to be destroyed after that) just to see what happens. If you have no local resources to spare, an hour of even very high end (to save time) VMs/cluster at a provider e.g. AWS costs next to nothing

Brosper 1y ago

Maybe Sentry don't want you to have self-hosted version?

neuroelectron 1y ago
What gave you that impression?
- mdaniel 1y ago
  I didn't downvote you but as one metric their docker-compose is 500 lines long https://github.com/getsentry/self-hosted/blob/25.4.0/docker-compose.yml#L500 https://github.com/getsentry/self-hosted/blob/25.4.0/docker-... as compared to back in the good-old days when it was basically python, redis, and postgres https://github.com/getsentry/self-hosted/blob/9.1.2/docker-compose.yml#L16-L19 https://github.com/getsentry/self-hosted/blob/9.1.2/docker-c...
conradfr 1y ago
I actually learn from this post that there is a self-hosted version...
- Brosper 1y ago
  I talked with them. They just offer it but don't offer any support for it.

croes 1y ago

From the Sentry homepage > Application monitoring software considered "not bad" by 4 million developers. Sounds pretty bad to me

gardnr 1y ago
As a software developer I can say that it is not bad. It's pretty great, actually.
- croes 1y ago
  But the infrastructure behind sounds complicated and fragile.
  - zeeg 1y ago
    No, its complicated and scales to servicing every customer in the world. Thats not the same thing. Doesnt mean the complaints about self-hosted arent valid, but "literally has to scale to the most insane volumes of data" and "is not good software" are two different things. We're building a cloud service at the end of the day - its a lot easier to optimize a multi-tenant install than it is a single-tenant install, and that complexity shows up in the self-hosted repo. Can't really avoid it.

mplantsheer 1y ago

The amount of shell script that needs to be executed to install is a bit of a no no for me. It also doesn't make sense to spin up a 16GB machine (minimum!) to track the errors on those 4-8GB VPS which are running my production services.

epolanski 1y ago

The biggest issue I have with these solutions is indeed local debugging. I use Sentry with most of my clients, and for effective debugging I need to spin my own Sentry in a Docker container which ends up being quite heavy on my machine especially when combined with Grafana and Prometheus. I'm really unhappy with virtually all monitoring/telemetry/tracking solutions. It really feels they are all designed to vendor lock you in their expensive cloud solutions and I really don't feel I'm getting my $s back at all. Worst of all those cloud vendors would rather add new features non-stop rather than honing what they currently have. Their sales are everywhere, I've seen two different clients getting Datadog sales people join private Slacks to evangelize their products. Both times I escalated to the CTO, both times I ended up suspecting someone in management had something to gain from pushing teams to adopt those solutions.

vanschelven 1y ago
I actually wrote about that scenario! Killing flies with hammers and all, but since I really like my hammer I actually do all my local development with my full-blown error tracker too: https://www.bugsink.com/blog/local-development/ https://www.bugsink.com/blog/local-development/
delusional 1y ago
Sentry's sales team is incredibly aggressive. I've seen multiple colleagues hijacked for sales presentations over a few months. It would not surprise me at all if they just asked random employees to be added to the company slack, and even less if those people then just did it. I can only commend the hustle on their part, but it does feel a little like a high pressure time share situation.
- zeeg 1y ago
  If you - or anyone reading this - ever ends up in a situation where we came off as aggressive send me a direct email and I will take care of it. This is not something we believe in at Sentry, and while you cant manage everything, its important to us that we never become "of those companies" like so many other successful companies become. david at sentry.io

adamcharnock 1y ago

This absolutely mirrors my experience. Sentry was straightforward to deploy years ago, but now seems like one of the more egregious offenders in the, 'self host-able but increasingly prohibitively complex by design' category. As others have said, we've [0] found the only practical way to deploy this for our clients is Kuberentes + Helm chart, and that's on bare-metal servers (mostly Hetzner). It runs well if you can throw hardware and engineering time at it, which thankfully we can. But given the option we would love a simpler solution. [0]: https://lithus.eu https://lithus.eu

wg0 1y ago
And how do you install and maintain/upgrade kubernetes? Are you running databases also on kubernetes?
- adamcharnock 1y ago
  In our case we have a collection of Ansible roles we use for the purpose. We run databases using the Stackgres operator either using logical replication on local fast NVMe dives, on top of OpenEBS/Mayastor replicated block-storage. But we specialise in this so that our clients don't have to. As much as I do actually love Kubernetes, the fact that the _easiest_ way to self-host Sentry is via Kubernetes is not a good sign. And choosing to spin up a Kubernetes cluster just to run Sentry would feel a lot like the lady who swallowed a fly[0]. [0]: https://en.wikipedia.org/wiki/There_Was_an_Old_Lady_Who_Swallowed_a_Fly https://en.wikipedia.org/wiki/There_Was_an_Old_Lady_Who_Swal...
  - apexalpha 1y ago
    Thanks for the poem, it seems pretty apt for IT in 2025. That said I would honestly prefer if the industry would just settle on K8s as our OS. I really do not see any benefit that sentry could bring on its own compared to a solid set of Helm charts for k8s.
  - baq 1y ago
    Does it really have to be a proper cluster, though? Can it be e.g. a single node k3s?

ghh 1y ago

This isn't okay - the author is selling their own alternative to Sentry, 'reusing' Sentry's open-source client SDK's, while spreading FUD about self-hosting Sentry. I've been self-hosting Sentry for over 10 years: Sentry is installed by running `git clone`, editing one or two configuration files, and running ./install.sh. It requires 16GB RAM if you enable the full feature set. That includes automatically recording user sessions on web or mobile for replay, and support for end-to-end tracing (so seeing which database queries your api asks for in response to a button tap in your mobile app). Sentry has a wonderful self-hosting team that's working hard to make Sentry's commercial feature set available, for free, to everyone that needs a mature error tracking solution. You can talk to them on discord and they listen and help.

roywashere 1y ago
Just for transparency, are you by any chance a member of this self-hosting team or a sentry employee? Is it a coincidence that your keybase name is georghendrik according to your profile, and the painter _Georg Hendrik_ Breitner painted a picture called "Sentry"? https://www.demesdagcollectie.nl/en/collection/hwm0055 https://www.demesdagcollectie.nl/en/collection/hwm0055
- nevertoolate 1y ago
  I was born 400 years ago in the Highlands of Scotland. I am Immortal, and I am not alone. Now is the time of the Gathering, when the stroke of the sword will release the power of the Quickening. In the end, there can be only one.
- Biganon 1y ago
  This is some Olympics level of badly coping with valid and welcome criticism. Georg Hendrik = "George Henry", pretty common name. The fact that Google returned a result when you searched "Georg Hendrik Sentry" should not be considered weird.
vanschelven 1y ago
TBH most of the FUD in the OP is straight from Sentry's own website. Regarding using the SDKs, I'm telling my users to take Sentry at their word when they wrote "Permission is hereby granted, free of charge [..] to deal in the Software without restriction, including without limitation the rights to use"

tanepiper 1y ago

This is also my opinion of Backstage. It's positioned as this easy-to-use open source tool for IDPs - but in reality, it's a product that needs a team to maintain it constantly.

mtndew4brkfst 1y ago
It has its own conference now! Definitely not a signal for something that is reliable set-it-and-forget-it tech.

fefe23 1y ago

I find it very funny that the post is basically complaining about fearmongering by the maker of Sentry to scare people into not hosting themselves. And then he does the exact same thing, on behalf of Sentry. I hope he got paid for this. Otherwise it would just be sad.

Timshel 1y ago
I doubt he would have been paid to promote a competing service ..
- vanschelven 1y ago
  If that were to happen I think the blog post (guaranteed virality) would just write itself...

notpushkin 1y ago

I still love Sentry, but it’s so enormous now that it isn’t practical to self-host for smaller businesses. A “small” alternative is always great to see! I’m not sure how I feel about the license though (Polyform Shield, basically use-but-don’t-compete). It’s a totally valid choice – I just wish it would convert to FOSS at some point. (I understand the concern as I’ve had to make a similar decision when releasing https://lunni.dev/ https://lunni.dev/. I went with AGPL, but I’m still overthinking it :-)

mayli 1y ago

I am not sure why there isn't a even lightweight sentry endpoint that does - save post body to folders (use uuid as folder name to avoid spam) - dir listing, and count number of entries - render posted json to html, highlight stacktrace with js - download raw json - rotate, compress old entries. I give those requirements to LLM, and I get a pretty much working rust implementation after few tweaks. It uses <5M ram idle.

hobofan 1y ago
That covers the most basic feature that Sentry has (one-off error reporting, with rudimentary statistics). Sure, if that's enough for you then go ahead, but for most people using Sentry that's only the tip of the iceberg.

seanwilson 1y ago

> we don’t necessarily recommend self-hosting to everyone. In addition to existing hidden costs, as Sentry evolves, our self-hosted version will become more complex, demanding additional types of infrastructure. Any insights on why Sentry is so complex and needs so much resources? Is collecting, storing, and organizing errors messages and stack traces at scale difficult? Or it's the other features on top of this?

eitland 1y ago
Some ideas: - they had enough money that they never needed to think seriously about maintenence cost, and the sales process was strokg enough to keep customers arriving anyway (look to Oracle for another example of hopelessly complicated installation process but people keep using it anyway) - at some point someone realized this was actually a feature: the more complicated it got, the harder it became to self host. And from that perspective it is a win-win for the company: they can claim it is open source without being afraid that most people will choose to self host.
- vanschelven 1y ago
  I'd say that the architecture that is desirable from the point of view of a large scale SaaS is very different than the one that's desirable from the point of view of a tool that just needs to work for a single organization. And since the SaaS is bringing in all the money, that's where the architecture follows. > actually a feature I would guess that for a few people people (e.g. the ones who made the scary visual of rising costs) this is explicitly so, but for most people it's more implied. i.e. I don't think anyone advanced their career with Sentry by making self-hosting easier.
  - BYK 1y ago
    Well I actually did as I was literally hired for that and worked on it for almost 3 years :)
slyall 1y ago
Anything gets complex at scale. This is the same software they us to host their SaaS system. Presumably has to scale to many thousands of customer and a huge number of events per second. They have all sorts of caching, autoscaling, distributed systems and other stuff thats complete overkill for all except that largest installation. Plus all sorts of software features only needed by a few customers and extra layers to be multi-customer. It's the difference between a hoop in your back yard and a NBA Stadium

deleted 1y ago

[deleted]

h1fra 1y ago

That's the inevitable evolution of any popular open-source product/company. Light and enterprise-ready is hardly compatible, and you can expect this from any project that has a cloud offering.

fuzzfactor 1y ago
Not only open source. I don't have servers like this but the author makes it easy to understand, and it applies to a lot of other things. The Comparative Costs picture does tell it all. The purpose of so much software is to reduce human and machine costs through time, and this apparently turned out to do just the opposite, apparently after long-term testing under real-world conditions. Could be an unsurmountable fundamental structure of technical debt or something like that which metastasizes. Then this, anyone could say about anything: >I’m not going to run a piece of software that requires 16GB of RAM, has a complex installation script, and is known to be a pain to maintain. I’m not going to recommend it to anyone else either. Easier said than done, sometimes it's the only option. It's a "script". Maybe that's why you have to "rehearse" it more than once before you barely get it right, and then you might have to really go the extra mile (and have a bit of good fortune) before you can achieve a "command performance". How do you think it feels these days to have to settle for this kind of thing in expensive proprietary software too?

madeofpalk 1y ago

It's wild that Sentry themselves are spreading literal FUD about about bad it is to self-host their own product. So confounding that they're so ready to shit on their own product.

elAhmo 1y ago

So many services are going this route. I remembered Reddit had an install script and almost everything worked out of the box 10-15 years ago on a Ubuntu VPS.

KronisLV 1y ago

A good APM tool I’ve been using for a few years is Apache Skywalking: https://skywalking.apache.org/ https://skywalking.apache.org/ It might not do everything Sentry does but it definitely has helped with tracking down some issues, even production ones and runs in a fairly manageable setup (in comparison to how long even the Sentry self-hosted Docker Compose file is). What’s more, if you want, you can even use regular PostgreSQL as the backing data store (might not be quite as efficient as ElasticSearch for a metrics use case, but also doesn’t eat your RAM like crazy).

mdaniel 1y ago
Any product that uses etcd is immediately off my radar[1]; it is nice of them to have a whole page dedicated to how to babysit the PoS https://skywalking.apache.org/docs/skywalking-banyandb/next/operation/cluster/#etcd-node-failure https://skywalking.apache.org/docs/skywalking-banyandb/next/... 1: yes, I'm a huge k8s fanboi and yes I long every day for them to allow me to swap out etcd for something sane
- KronisLV 1y ago
  That’s kind of the beauty of it: you can opt for a different storage solution too https://skywalking.apache.org/docs/main/v10.2.0/en/setup/backend/backend-storage/ https://skywalking.apache.org/docs/main/v10.2.0/en/setup/bac... Personally, no hate towards their BanyanDB but after getting burnt by OrientDB in Sonatype Nexus, I very much prefer more widespread options.

afro88 1y ago

> First, there’s a signal being given about the complexity and fragility of the software. A 16GB RAM requirement just screams “this is a big, complex piece of software that will break in mysterious ways”. Not very scientific, I know, but based on experience. This lines up with my experience self hosting a headless BI service. In "developer mode" it takes maybe 1GB RAM. But as soon as you want to go prod you need multiple servers with 4+ cores and 16GB+ RAM that need a strongly consistent shared file store. Add confusing docs to the mix, mysterious breakages, incomplete logging and admin APIs, and a reliance on community templates for stuff like k8s deployment... it was very painful. I too gave up on self hosted.

AdrianB1 1y ago
In most corporate projects that I worked with, the day it went to production started the clock to disband the core team and optimizations were not on the table. When the product works, they move on. The only area where performance optimization is a mandatory step is video games, but that is where your clients are external. Most business software is in the "finally, but barely works" state and new features beat performance improvements. This is caused by short sighted management that need to deliver and move on. "Long term" is a contradiction with their business model. In this case "long term" means "after product launch".

BenGosub 1y ago

I worked in a startup around 2019-2020 where we started using Sentry for distributed tracing. It was a company in the IOT space and we had a lot of events going through. The bill started going in the $3-5000 range just from the tracing, so we decided to host it on our own. When looking at the sheer complexity of the project, I was flabbergasted. We need a name for managing the infrastructure of company based open source, self-hosted solutions. Often times, the right choice would be to choose a different, simpler, open source solution. In this case there are some neat distributed tracing solutions, that are easier to manage.

exiguus 1y ago

I'm a big fan of self-hosting, but it requires the right infrastructure. You need a robust virtualization solution like Proxmox, equipped with sufficient CPU cores and RAM. For instance, allocating 16GB of RAM shouldn't be an issue if your server has 128GB to share. In my experience, solutions like Mailcow, which involve multiple services and containers (such as SMTP, IMAP, Redis, SSO, webmail, Rspamd, etc.), work very well. I have extensive experience running these systems, performing backups, restoring data, and updating their ecosystems. Additionally, I've had a positive experience setting up and running a self-hosted Sentry instance with Docker for a project that spanned several years. However, this experience might be somewhat outdated, as it was a few years ago.

npodbielski 1y ago
I am running Mailcow on just some VM with docker. No Proxmox needed. So far it works.
- exiguus 1y ago
  Sure, that topic was more about the 16GB of RAM needed for sentry. If you are already have shared RAM, 16GB sounds much more do-able.

azthecx 1y ago

At first glance it's not immediately obvious to me, why would you pick Sentry or Bugsink over something included in the Grafana stack? What's the use use

mdaniel 1y ago
I haven't used Sentry's obserability portions, but the error capture has very strong deduplication built into it, and can optionally integrate with your SCM, CI, and issue tracker, if one wishes. Thus, the flow can go like: - error happens, can be attributed to release 1.2.3 - every subsequent time that error happens to a different user, it can track who was affected by it, without opening a new error report - your project can opt-in to accepting end-user feedback on error: "please tell us what you were doing when this exploded, or feel free to rant and rave, we read them all" - it knows from the stack trace that the error is in src/kaboom/onoz.py line 55 - onoz.py:55 was last changed by claude@example.com last week, in PR #666 - sentry can comment upon said PR to advise the reviewers of the bad outcome - sentry can create a Jira with the relevant details - claude.manager@example.com can mark the bug as "fixed in the next release", which will cause sentry to suppress chirping about it until it sees a release 1.2.4 - if it happens again it will re-open the prior error report, marking it as a regression Unless you know something I don't, Grafana does *ABSOLUTELY NONE* of that
- azthecx 1y ago
  Thanks for the detailed response

cuu508 1y ago

I was looking into moving away from managed Sentry for GDPR reasons. I was only using Sentry for notifications about unhandled exceptions in my app. All the fancy APM features I did not use. Grouping of similar exceptions, pretty display of tracebacks were nice but not essential. In my Django app I wrote a logging handler that stores the log records (including traceback) in a database table. I can inspect the log records through Django admin, and a cron job sends me daily emails saying "X new log records in the last 24 hours" so I know to check them out. And that's it :-) Of course, this does a lot less than Sentry, and has various limitations (e.g. what if the error is about the database being down...), but it fits my needs. BTW, IIUC, Sentry in its early beginnings was also doing just that – logging to database: https://github.com/dcramer/django-db-log https://github.com/dcramer/django-db-log

rwky 1y ago

I use both hosted and self hosted sentry. I prefer hosted it's less to manage but self hosted isn't too awful as long as you can budget the resources. If you just want the team sentry plan it's going to be cheaper to use hosted. I would only self host if you had to for legal/compliance reasons. In the many years I've been managing sentry we've only had it truly crap out once and that was an upgrade from 9 to their new date versioning basically the whole hosted method changed and it was just easier to start over from scratch.

linsomniac 1y ago

TL;DR: Not so much "gave up" as "never tried" because I was scared away by 16GB RAM requirement. FWIW, we've been self-hosting 2 instances (and purchase a third hosted instance), for, it looks like 8 years now, and it's had a few small bumps but hasn't been too bad. Our instances are running with 20GB of RAM, so pretty small. ~90% of updates have gone smoothly, which leaves 10% of updates that have had some sort of problem. Solutions have been found in the issue tracker in all our cases. We are a little afraid of doing updates, but do them every couple of months. Sentry is an amazing piece of software and it is fantastic that they offer a self-hosing version, and all things considered it is a fairly easy self-host.

linsomniac 1y ago
FYI: Just did the upgrade from 25.2.0 to 25.4.0 and it went smoothly.
vanschelven 1y ago
You're right, but so am I... :-) I "gave up" from the perspective of returning to Sentry after a couple of years, and finding an entirely different beast from the tool I loved before. At that point I indeed didn't make it past Sentry's own FUD.

1vuio0pswjnm7 1y ago

As a computer user I do not allow connections to sentry.io For example, Firefox tries to connect

mdaniel 1y ago
Heh, saying that in a thread about self hosting Sentry is pretty funny :-) --- IMHO, the RUM tooling[1] is the worst offender, tracking mouse movement and keystrokes. At least Sentry is usually just trying to report JS kabooms 1: https://docs.datadoghq.com/real_user_monitoring/session_replay/ https://docs.datadoghq.com/real_user_monitoring/session_repl... and https://docs.newrelic.com/docs/browser/browser-monitoring/browser-pro-features/session-replay/ https://docs.newrelic.com/docs/browser/browser-monitoring/br... et al

johnklos 1y ago

Software that's only available as x86 binaries and a whole system that needs 16 gigs of memory? It's not surprising that it comes from the corporate world and not the open source world. I respect the fact that the Sentry people are honest about their teetering tower.

m463 1y ago

> It’s a ton of docker containers. Things will fail randomly or maybe with a lot of traffic, don’t remember well. maybe they should put in a system to monitor the docker containers.

domysee 1y ago

HyperDX is a great alternative to Sentry, much easier to self-host, and also open source. It's relatively new and did take some tinkering to make it work properly, so I wrote a short article about it: https://weberdominik.com/blog/self-host-hyperdx https://weberdominik.com/blog/self-host-hyperdx But the feature set and user experience is great!