6 ms·

Anecdotally, we've seen a number of larger "open-source alternative to X" projects posted on HN of late that are technically self-deployable, but require so muc

Anecdotally, we've seen a number of larger "open-source alternative to X" projects posted on HN of late that are technically self-deployable, but require so much up-front knowledge that it's not actually accessible to those who might truly be liberated by such software. I mean no slander or disrespect to anyone involved, but there was a DataDog alternative posted sometime in the last few weeks that had a docker-compose with like 15 containers in it. Required running a few different Typescript servers, a Clickhouse instance, Redis, MySQL, the lot of it. I'm sure it was a fully-featured service that made adequate use of those resources, but it also reminded me of why people pay out the wazoo for DataDog: nobody wants to manage all that stuff! EDIT: the repo linked in the GP contains 3 instances of what you could call databases: MariaDB, Mongo, and Redis. There doesn't appear to be any explanation in the deployment docs for why all three are necessary.

abouolia 3y ago
Totally agree, nobody can manage that amount of containers, it doesn't mean if it was self-hosted would excuse the deployment is not necessary to be easy, always the deployment of self-hosted should be easy with sample CLIs. and also providing pre-configured instances on the most popular cloud services like on DigitalOcean to deploy the app with sample clicks (instead of CLIs) that would great additional for non-techincal people and that's what we're seeking for, in terms of 3 instance actually we used Mongo for Agenda.js but we want to get rid of it ASAP.
KronisLV 3y ago
> I mean no slander or disrespect to anyone involved, but there was a DataDog alternative posted sometime in the last few weeks that had a docker-compose with like 15 containers in it. Reminds me of Sentry: https://develop.sentry.dev/self-hosted/ https://develop.sentry.dev/self-hosted/ This is their example docker-compose for self-hosting: https://github.com/getsentry/self-hosted/blob/master/docker-compose.yml https://github.com/getsentry/self-hosted/blob/master/docker-... It has: - exim4 (smtp) - memcached - Redis - PostgreSQL - Zookeeper - Kafka - Clickhouse - geoipupdate - 18 containers for something called snuba (integrates with Clickhouse?) - 2 symbolicator containers - web app - cron - 18 other utility containers - Nginx - relay - 2 vroom containers Meanwhile, as far as the other options in the APM space go, this is how Apache Skywalking looks like: - web app - back end API - database (Elasticsearch/OpenSearch, PostgreSQL, MariaDB/MySQL or something else) I'm glad that generally it's good enough for my needs, because I can't imagine actually having to self-host Sentry. Then again, something like Graylog does log shipping better and without a doubt Sentry has lots of nice features... as long as you don't have to run it yourself, because many people out there won't realistically be able to do so. > EDIT: the repo linked in the GP contains 3 instances of what you could call databases: MariaDB, Mongo, and Redis. That's not too bad, though, but seems like an interesting choice (both a regular RDBMS and a NoSQL one; Redis would probably be useful for caching etc. for either).
- aidos 3y ago
  Sentry is definitely a bit heavyweight on the services. Though, it makes sense since you’re effectively running everything they run for their saas business built for a whole other scale. The trade off is that it’s far more complex than what anybody needs if they just want to run sentry locally. It used to be a lot simpler but now it’s pretty much impossible to even understand how they store data without spending a long time studying it. To be fair to them though, their migrations and releases have generally been solid and reliable.
- mekster 3y ago
  What are you talking about? Running Sentry isn't hard despite crazy number of containers as long as you do it their way and use their docker-compose.yml. The problem is running it on 4GB instance with 4GB swap is bare minimum which costs quite a bit to give it a dedicated instance. But you can't beat its features. We should be glad it's provided for free and self hosted version isn't limited. Can't live without it.
- yebyen 3y ago
  You know what skywalking doesn't have though, apparently, is Ruby support! I would love to understand a bit better why this one gets left off the list so often. "There's just nobody interested in putting those two things together" I guess, it's surprising to me though, because Ruby is pretty popular AFAIK.
that_guy_iain 3y ago
> Anecdotally, we've seen a number of larger "open-source alternative to X" projects posted on HN of late that are technically self-deployable, but require so much up-front knowledge that it's not actually accessible to those who might truly be liberated by such software. Any self-hosted application really needs someone technical unless you're providing something like the DigitalOcean deploy button. Docker-compose requires technical knowledge to run locally. Sure running "docker compose" is easy but installing it often isn't. As someone quite technical I'm honestly not sure how that would work for deploying on to AWS or GCP even though I know it's possible. I would have to look into it. Ansible scripts require technical knowledge. Again running the command is easy but installing ansible isn't. How DigitalOcean's deploy button is really easy as is shown in my demo video - https://www.youtube.com/watch?v=cbInyGtqLCs&t=1s https://www.youtube.com/watch?v=cbInyGtqLCs&t=1s. Note there is 10 minutes of my app demo inbetween starting the process and me using a deployed version. You can even try it out at https://github.com/BillaBear/billabear https://github.com/BillaBear/billabear it's literally super easy. It's also silly easy to actually build. "DigitalOcean's app platform is so good it makes BillaBear looks good since the deployment is so easy" is literally what I've said to people.
mikeshi42 3y ago
> I mean no slander or disrespect to anyone involved, but there was a DataDog alternative posted sometime in the last few weeks that had a docker-compose with like 15 containers in it. If this was us [1] (an OSS Datadog alternative posted last week :D ), we do have quite a bit (11, 2 are optional) but we're working on bringing it down. We currently split our ingestion pipeline into 3 independently scalable bits, but we can probably bring it down into 1 for any small-scale deployment. Otherwise we do have the standard need for a cache (redis), db (mongo), main storage (clickhouse), and then the standard API server + frontend, and a separate task to run alerts. There's likely a few more things we can merge together, but it comes at the expense of making it unscalable to workloads typical to a company and divergence in the code base, which overall doesn't seem like the right tradeoff. Imo the real concern as someone that personally owns too many tiny VM instances is resource footprint - which can be tuned down to ~1GB of memory for us (depending on server load). Feel free to open an issue too if you think there's something we can adjust there as well. [1] https://github.com/hyperdxio/hyperdx https://github.com/hyperdxio/hyperdx

abouolia 3y ago

Totally agree, nobody can manage that amount of containers, it doesn't mean if it was self-hosted would excuse the deployment is not necessary to be easy, always the deployment of self-hosted should be easy with sample CLIs. and also providing pre-configured instances on the most popular cloud services like on DigitalOcean to deploy the app with sample clicks (instead of CLIs) that would great additional for non-techincal people and that's what we're seeking for, in terms of 3 instance actually we used Mongo for Agenda.js but we want to get rid of it ASAP.

KronisLV 3y ago

> I mean no slander or disrespect to anyone involved, but there was a DataDog alternative posted sometime in the last few weeks that had a docker-compose with like 15 containers in it. Reminds me of Sentry: https://develop.sentry.dev/self-hosted/ https://develop.sentry.dev/self-hosted/ This is their example docker-compose for self-hosting: https://github.com/getsentry/self-hosted/blob/master/docker-compose.yml https://github.com/getsentry/self-hosted/blob/master/docker-... It has: - exim4 (smtp) - memcached - Redis - PostgreSQL - Zookeeper - Kafka - Clickhouse - geoipupdate - 18 containers for something called snuba (integrates with Clickhouse?) - 2 symbolicator containers - web app - cron - 18 other utility containers - Nginx - relay - 2 vroom containers Meanwhile, as far as the other options in the APM space go, this is how Apache Skywalking looks like: - web app - back end API - database (Elasticsearch/OpenSearch, PostgreSQL, MariaDB/MySQL or something else) I'm glad that generally it's good enough for my needs, because I can't imagine actually having to self-host Sentry. Then again, something like Graylog does log shipping better and without a doubt Sentry has lots of nice features... as long as you don't have to run it yourself, because many people out there won't realistically be able to do so. > EDIT: the repo linked in the GP contains 3 instances of what you could call databases: MariaDB, Mongo, and Redis. That's not too bad, though, but seems like an interesting choice (both a regular RDBMS and a NoSQL one; Redis would probably be useful for caching etc. for either).

aidos 3y ago
Sentry is definitely a bit heavyweight on the services. Though, it makes sense since you’re effectively running everything they run for their saas business built for a whole other scale. The trade off is that it’s far more complex than what anybody needs if they just want to run sentry locally. It used to be a lot simpler but now it’s pretty much impossible to even understand how they store data without spending a long time studying it. To be fair to them though, their migrations and releases have generally been solid and reliable.
mekster 3y ago
What are you talking about? Running Sentry isn't hard despite crazy number of containers as long as you do it their way and use their docker-compose.yml. The problem is running it on 4GB instance with 4GB swap is bare minimum which costs quite a bit to give it a dedicated instance. But you can't beat its features. We should be glad it's provided for free and self hosted version isn't limited. Can't live without it.
yebyen 3y ago
You know what skywalking doesn't have though, apparently, is Ruby support! I would love to understand a bit better why this one gets left off the list so often. "There's just nobody interested in putting those two things together" I guess, it's surprising to me though, because Ruby is pretty popular AFAIK.

that_guy_iain 3y ago

> Anecdotally, we've seen a number of larger "open-source alternative to X" projects posted on HN of late that are technically self-deployable, but require so much up-front knowledge that it's not actually accessible to those who might truly be liberated by such software. Any self-hosted application really needs someone technical unless you're providing something like the DigitalOcean deploy button. Docker-compose requires technical knowledge to run locally. Sure running "docker compose" is easy but installing it often isn't. As someone quite technical I'm honestly not sure how that would work for deploying on to AWS or GCP even though I know it's possible. I would have to look into it. Ansible scripts require technical knowledge. Again running the command is easy but installing ansible isn't. How DigitalOcean's deploy button is really easy as is shown in my demo video - https://www.youtube.com/watch?v=cbInyGtqLCs&t=1s https://www.youtube.com/watch?v=cbInyGtqLCs&t=1s. Note there is 10 minutes of my app demo inbetween starting the process and me using a deployed version. You can even try it out at https://github.com/BillaBear/billabear https://github.com/BillaBear/billabear it's literally super easy. It's also silly easy to actually build. "DigitalOcean's app platform is so good it makes BillaBear looks good since the deployment is so easy" is literally what I've said to people.

mikeshi42 3y ago

> I mean no slander or disrespect to anyone involved, but there was a DataDog alternative posted sometime in the last few weeks that had a docker-compose with like 15 containers in it. If this was us [1] (an OSS Datadog alternative posted last week :D ), we do have quite a bit (11, 2 are optional) but we're working on bringing it down. We currently split our ingestion pipeline into 3 independently scalable bits, but we can probably bring it down into 1 for any small-scale deployment. Otherwise we do have the standard need for a cache (redis), db (mongo), main storage (clickhouse), and then the standard API server + frontend, and a separate task to run alerts. There's likely a few more things we can merge together, but it comes at the expense of making it unscalable to workloads typical to a company and divergence in the code base, which overall doesn't seem like the right tradeoff. Imo the real concern as someone that personally owns too many tiny VM instances is resource footprint - which can be tuned down to ~1GB of memory for us (depending on server load). Feel free to open an issue too if you think there's something we can adjust there as well. [1] https://github.com/hyperdxio/hyperdx https://github.com/hyperdxio/hyperdx