4 ms·

From the the reddit link[1] Sorry, I'm just curious. It says MySQL 8 and Elasticsearch 7.8. I don't have much experience in elasticsearch, I wanted to know ho

From the the reddit link[1] Sorry, I'm just curious. It says MySQL 8 and Elasticsearch 7.8. I don't have much experience in elasticsearch, I wanted to know how does elasticsearch makes it faster? Is it like an extension that makes it faster? Or Elasticsearch has its own data store that consumes data from the database and magically makes it faster? Thanks. [1]https://www.reddit.com/r/programming/comments/jg4rkv/how_a_search_engine_with_400_million_records/ https://www.reddit.com/r/programming/comments/jg4rkv/how_a_s...

richardbarosky 6y ago
Elasticsearch is a search platform. A "database" but meant for search stuff. It's not part of MySQL.
nerdponx 6y ago
https://news.ycombinator.com/item?id=25152925 https://news.ycombinator.com/item?id=25152925
rpedela 6y ago
Elasticsearch, Lucene under the hood, implements an inverted index which is an extremely fast data structure for text search. ES has clustering as a primary feature too and many search features that can significantly improve relevance that you won't find in MySQL and most other databases.
- arminiusreturns 6y ago
  How would you say it stands up to splunk these days?
- lolive 6y ago
  Plus it does not accept joins. So you basically have to denormalize all your data before injecting into Elastic. It helps speedup things. But is a headache to manage on a day to day basis.
  - rpedela 6y ago
    Yeah. What I do is create a view that does all the joins then the middleware just needs to do "SELECT * FROM my_view". If the DB has good JSON support, I will also convert the data into an ES index request with SQL so the middleware becomes even simpler.
    - lolive 6y ago
      Let’s say you have a mostly read-only DB (otherwise things are different). Does it work when the view is insanely big? [i am not an expert at DBs, so my vision of a big DB might amuse you, but let’s say I have millions of rows to assemble as a view].
- devy 6y ago
  Have you tried Toshi[1] or MeiliSearch[2]. I wonder how it would compare in terms of operational costs (monthly cloud hosting bill) at the current data size. [1]: https://news.ycombinator.com/item?id=18895655 https://news.ycombinator.com/item?id=18895655 [2]: https://news.ycombinator.com/item?id=22685831 https://news.ycombinator.com/item?id=22685831
  - jabo 6y ago
    Do you have the structured dataset somewhere? I’d love to index it in Typesense [1] and see how it does. I recently tried a 32M songs dataset [2] and it works great, so I’m on the lookout for larger datasets to benchmark with. [1] https://news.ycombinator.com/item?id=22181437 https://news.ycombinator.com/item?id=22181437 [2] https://songs-search.typesense.org/ https://songs-search.typesense.org/
scottydelta 6y ago
The gist is Elasticsearch is a full-index database. Whatever data goes in gets indexed as compared to only indexing certain fields in MySQL on which you perform search frequently. Think of Elasticsearch as MongoDB + full-indexing. It's a document storage with blazing fast search and aggregation.

richardbarosky 6y ago

Elasticsearch is a search platform. A "database" but meant for search stuff. It's not part of MySQL.

nerdponx 6y ago

https://news.ycombinator.com/item?id=25152925 https://news.ycombinator.com/item?id=25152925

rpedela 6y ago

Elasticsearch, Lucene under the hood, implements an inverted index which is an extremely fast data structure for text search. ES has clustering as a primary feature too and many search features that can significantly improve relevance that you won't find in MySQL and most other databases.

arminiusreturns 6y ago
How would you say it stands up to splunk these days?
lolive 6y ago
Plus it does not accept joins. So you basically have to denormalize all your data before injecting into Elastic. It helps speedup things. But is a headache to manage on a day to day basis.
- rpedela 6y ago
  Yeah. What I do is create a view that does all the joins then the middleware just needs to do "SELECT * FROM my_view". If the DB has good JSON support, I will also convert the data into an ES index request with SQL so the middleware becomes even simpler.
  - lolive 6y ago
    Let’s say you have a mostly read-only DB (otherwise things are different). Does it work when the view is insanely big? [i am not an expert at DBs, so my vision of a big DB might amuse you, but let’s say I have millions of rows to assemble as a view].
devy 6y ago
Have you tried Toshi[1] or MeiliSearch[2]. I wonder how it would compare in terms of operational costs (monthly cloud hosting bill) at the current data size. [1]: https://news.ycombinator.com/item?id=18895655 https://news.ycombinator.com/item?id=18895655 [2]: https://news.ycombinator.com/item?id=22685831 https://news.ycombinator.com/item?id=22685831
- jabo 6y ago
  Do you have the structured dataset somewhere? I’d love to index it in Typesense [1] and see how it does. I recently tried a 32M songs dataset [2] and it works great, so I’m on the lookout for larger datasets to benchmark with. [1] https://news.ycombinator.com/item?id=22181437 https://news.ycombinator.com/item?id=22181437 [2] https://songs-search.typesense.org/ https://songs-search.typesense.org/

scottydelta 6y ago

The gist is Elasticsearch is a full-index database. Whatever data goes in gets indexed as compared to only indexing certain fields in MySQL on which you perform search frequently. Think of Elasticsearch as MongoDB + full-indexing. It's a document storage with blazing fast search and aggregation.