4 ms·

Anyone know more lightweight alternative to (ELK) Elastic Stack? I found https://vector.dev https://vector.dev but it seems to be only the "L" part.

webo 6y ago
Have you looked into Google Cloud Logging (Stackdriver)? It's the most affordable and decent-enough solution we've found. The only issue is querying can be slow on large volumes.
- spyspy 6y ago
  Have you tried creating a sink from stackdriver to bigquery?
  - webo 6y ago
    Hi, not really because my understanding is bigquery requires structured data but Stackdriver just expects a arbitrary json payload which can be queried from. Do you have any experience?
ehnto 6y ago
I found Lucene's base library really easy to use without the configuration/infrastructure overhead of Elasticsearch, but haven't experienced it at scale: https://lucene.apache.org/ https://lucene.apache.org/
- rzzzt 6y ago
  Solr is the equivalent-ish of ES, if you are looking for a search server instead of a library that can be embedded: https://lucene.apache.org/solr/ https://lucene.apache.org/solr/
Xylakant 6y ago
Promtail/Loki https://github.com/grafana/loki https://github.com/grafana/loki is an alternative to elk, but while it seems more lightweight, it definitely is less featureful. The integration with grafana/prometheus seems nice, but I've only toyed with it, not used in production.
johnx123-up 6y ago
What about https://github.com/meilisearch/MeiliSearch https://github.com/meilisearch/MeiliSearch ?
- wdb 6y ago
  I am looking into this. Do you have experience with it?
  - johnx123-up 6y ago
    It is good. I can't find any CDC for Postgres for the incremental sync. And so I had to use the bulk update/sync and that causes performance issues occasionally. Also, some Algolia features are not available yet https://github.com/meilisearch/instant-meilisearch/issues/215 https://github.com/meilisearch/instant-meilisearch/issues/21...
FridgeSeal 6y ago
ClickHouse will happily replace the ElasticSearch bit, and there’s a few open source dashboards you could use as a kibana stand in: - Metabase (with ClickHouse plug-in) - Superset - Grafana

webo 6y ago

Have you looked into Google Cloud Logging (Stackdriver)? It's the most affordable and decent-enough solution we've found. The only issue is querying can be slow on large volumes.

spyspy 6y ago
Have you tried creating a sink from stackdriver to bigquery?
- webo 6y ago
  Hi, not really because my understanding is bigquery requires structured data but Stackdriver just expects a arbitrary json payload which can be queried from. Do you have any experience?

ehnto 6y ago

I found Lucene's base library really easy to use without the configuration/infrastructure overhead of Elasticsearch, but haven't experienced it at scale: https://lucene.apache.org/ https://lucene.apache.org/

rzzzt 6y ago
Solr is the equivalent-ish of ES, if you are looking for a search server instead of a library that can be embedded: https://lucene.apache.org/solr/ https://lucene.apache.org/solr/

Xylakant 6y ago

Promtail/Loki https://github.com/grafana/loki https://github.com/grafana/loki is an alternative to elk, but while it seems more lightweight, it definitely is less featureful. The integration with grafana/prometheus seems nice, but I've only toyed with it, not used in production.

johnx123-up 6y ago

What about https://github.com/meilisearch/MeiliSearch https://github.com/meilisearch/MeiliSearch ?

wdb 6y ago
I am looking into this. Do you have experience with it?
- johnx123-up 6y ago
  It is good. I can't find any CDC for Postgres for the incremental sync. And so I had to use the bulk update/sync and that causes performance issues occasionally. Also, some Algolia features are not available yet https://github.com/meilisearch/instant-meilisearch/issues/215 https://github.com/meilisearch/instant-meilisearch/issues/21...

FridgeSeal 6y ago

ClickHouse will happily replace the ElasticSearch bit, and there’s a few open source dashboards you could use as a kibana stand in: - Metabase (with ClickHouse plug-in) - Superset - Grafana