4 ms·

Elasticsearch, Lucene under the hood, implements an inverted index which is an extremely fast data structure for text search. ES has clustering as a primary fea

Elasticsearch, Lucene under the hood, implements an inverted index which is an extremely fast data structure for text search. ES has clustering as a primary feature too and many search features that can significantly improve relevance that you won't find in MySQL and most other databases.

arminiusreturns 6y ago
How would you say it stands up to splunk these days?
lolive 6y ago
Plus it does not accept joins. So you basically have to denormalize all your data before injecting into Elastic. It helps speedup things. But is a headache to manage on a day to day basis.
- rpedela 6y ago
  Yeah. What I do is create a view that does all the joins then the middleware just needs to do "SELECT * FROM my_view". If the DB has good JSON support, I will also convert the data into an ES index request with SQL so the middleware becomes even simpler.
  - lolive 6y ago
    Let’s say you have a mostly read-only DB (otherwise things are different). Does it work when the view is insanely big? [i am not an expert at DBs, so my vision of a big DB might amuse you, but let’s say I have millions of rows to assemble as a view].
devy 6y ago
Have you tried Toshi[1] or MeiliSearch[2]. I wonder how it would compare in terms of operational costs (monthly cloud hosting bill) at the current data size. [1]: https://news.ycombinator.com/item?id=18895655 https://news.ycombinator.com/item?id=18895655 [2]: https://news.ycombinator.com/item?id=22685831 https://news.ycombinator.com/item?id=22685831
- jabo 6y ago
  Do you have the structured dataset somewhere? I’d love to index it in Typesense [1] and see how it does. I recently tried a 32M songs dataset [2] and it works great, so I’m on the lookout for larger datasets to benchmark with. [1] https://news.ycombinator.com/item?id=22181437 https://news.ycombinator.com/item?id=22181437 [2] https://songs-search.typesense.org/ https://songs-search.typesense.org/

arminiusreturns 6y ago

How would you say it stands up to splunk these days?

lolive 6y ago

Plus it does not accept joins. So you basically have to denormalize all your data before injecting into Elastic. It helps speedup things. But is a headache to manage on a day to day basis.

rpedela 6y ago
Yeah. What I do is create a view that does all the joins then the middleware just needs to do "SELECT * FROM my_view". If the DB has good JSON support, I will also convert the data into an ES index request with SQL so the middleware becomes even simpler.
- lolive 6y ago
  Let’s say you have a mostly read-only DB (otherwise things are different). Does it work when the view is insanely big? [i am not an expert at DBs, so my vision of a big DB might amuse you, but let’s say I have millions of rows to assemble as a view].

devy 6y ago

Have you tried Toshi[1] or MeiliSearch[2]. I wonder how it would compare in terms of operational costs (monthly cloud hosting bill) at the current data size. [1]: https://news.ycombinator.com/item?id=18895655 https://news.ycombinator.com/item?id=18895655 [2]: https://news.ycombinator.com/item?id=22685831 https://news.ycombinator.com/item?id=22685831

jabo 6y ago
Do you have the structured dataset somewhere? I’d love to index it in Typesense [1] and see how it does. I recently tried a 32M songs dataset [2] and it works great, so I’m on the lookout for larger datasets to benchmark with. [1] https://news.ycombinator.com/item?id=22181437 https://news.ycombinator.com/item?id=22181437 [2] https://songs-search.typesense.org/ https://songs-search.typesense.org/