4 ms·
This is obviously a dumb question, being in this thread and all but what's the difference between LogStash, Elastic Search and Lucene? I only ever saw one amazi
by animalnewbie 7y ago
This is obviously a dumb question, being in this thread and all but what's the difference between LogStash, Elastic Search and Lucene? I only ever saw one amazing demo once with real time search and I remember hearing all 3 names. My guess is LogStash stores and the other two retrieve?
- gjulianm 7y agoAFAIK, Elasticsearch is the database system. Apache Lucene is the engine that Elasticsearch uses to perform searches, and the Logstash is the system that retrieves (or receives) data, processes it and gets it into ES.
- itronitron 7y agoLucene is the core search engine, it can be used on its own and has a lot of depth in its many capabilities. ElasticSearch (and SolrCloud >> LucidWorks Fusion Server) add a distributed architecture that leverages Lucene's capabilities. LogStash helps ES easily deal with log files, and it has been one of the main marketing drivers used by elastic which is ironic since most of Lucene's capabilities are more useful on human generated text as opposed to machine generated logs.
- tilolebo 7y agoLogstash accepts events from different types of sources, processes them (there are tons of inputs, filters and outputs available), converts them to Elasticsearch-compatible format and sends them to the cluster. It is usually used as a central component. For logs and metrics reading, one can use Elastic Beats,such as Filebeat for logfiles. Beats are lightweight agents written in Golang. Recent versions of Elasticsearch also allow to perform the processing step directly in Elasticsearch, using ingest nodes and pipelines (my experience with it wasn't great). In that case Logstash isn't needed.