7 ms·
As the author, it may be good that I clarify the goals behind this post. First I'm glad that this post made HN first page for a second time. I encourage to re
by rachbelaid 12y ago
As the author, it may be good that I clarify the goals behind this post.
First I'm glad that this post made HN first page for a second time.
I encourage to read this comment because it summarizes well few of the missing features in Postgres about search. https://news.ycombinator.com/item?id=8715624 https://news.ycombinator.com/item?id=8715624
It's not the first time that I get similar feebacks and I understand that "Enough" is subjective notion and maybe I should've pick an other title like "Postgres full-text search for basic search need".
Just to clarify, this post is not against ES or SOLR. This tools are great and really powerfull but obviously this tools will better than Postgres Search. ES / SOLR are tools build to be anly around document store and search and on the otherside the search in Postgres is only one feature in many.
I think there is multiple reason when you would want to give Postgres search a go:
- You have no experience with ElasticSearch or SOLR
- The project is still early stage and you don't want to invest time/money in a secondary datastore (eg: ES cluster)
- You want to give quickly a search feature (maybe not perfect) in the hand of your user
- Team knowledge, you may want to be careful before introducing a new tech to your team and ensure that people understand it. There is nothing worst than introducing a tech and being the only one who can maintain it.
The tech world is big and doesn't only apply to Startup usecase, I worked in very different environment: Financial institution, Open Source company, Startup and Service company. A thing that I learn is that you don't always have flexibility. In some company having new servers is long process and even if I hate you have to do with this constraint then you have 2 choices front of you: You do nothing and wait or you move yourself to offer to your users a workaround solution without any extra dependency ... That's what Postgres search can be in some usecases.
From my experience, if you are already using Postgres and you find the right documentation (this post try to describe a start to end example) then I think you could get a search working in 1 day of work without any extra servers or resources.
I don't have any experience with SOLR so I will have to pick ES, but if you never used it then you are going to have loads of "fun":
- Sync. ElasticSearch IMHO cannot be used as a cannonical datasource so you will need to figure out a sync strategy. Some people may argue on that but I don't think that ES started with this goal in mind. They may change direction now but still I wouldn't store my data only in ES.
http://aphyr.com/posts/317-call-me-maybe-elasticsearch http://aphyr.com/posts/317-call-me-maybe-elasticsearch
- Hosting. In the case that you are not using a hosted ES service then you will need to learn to host and build a cluster and also monitor it.
- Learning to use it. After reading ~50pages of doc then you start to figure out how to use type mapping, custom analyser and the 42 ways to do a search query.
I can only speak from my experience but it took me ~2weeks to get my head around all this point without any previous experience.
The project for which I used PG search and wrote this blog post recently moved to ES. More or less, 1y later and I'm glad that the PG solution was good enought until now. In our case we needed to support more languages: Japanese, .. and I also wanted to get rid of a bunch of triggers.
The results are good and very fast but for english the results are quite similar to Postgres solution. I think that's mainly because I'm using much only ES basic feature, but we have loads of room for improvement which is a good position to be.
Now there is few other usecases where you may want to consider PG search:
- you are using MySQL and you are using the builtin then moving to Postgres can give your search a bit more power
- Side project using Postgres and you want to start a search feature.
- Search on machine output, case where analyser doesn't really matter and you don't need stemming. Lucene features for tokenization and stemming are great.
To summarize, I don't think that PG search is a silver bullet but it may be "Good Enough" for your need to get you started. Spend 20min and read the post and then maybe it's also good enough for you. On the other hand if you know already ES/SOLR and you got the resources then it become less relevant to use Postgres Search. ES/SOLR (tools base on Lucene) are amazing and really feature rich, if you have important search need and long term plan about search then you should look into them.
If you are interrested about the PG search and you want to know more about it than I suggest you to have a look to this blog posts. I think that the author did an amazing job.
http://shisaa.jp/postset/postgresql-full-text-search-part-1.html http://shisaa.jp/postset/postgresql-full-text-search-part-1....
http://shisaa.jp/postset/postgresql-full-text-search-part-2.html http://shisaa.jp/postset/postgresql-full-text-search-part-2....
http://shisaa.jp/postset/postgresql-full-text-search-part-3.html http://shisaa.jp/postset/postgresql-full-text-search-part-3....
I don't know if people will be interrested but I thought few times about writing a following blog post "When Postgres search is not Enough" to describe how to integrate Postgres with ElasticSearch with JDBC River to pull content from the DB.. Let me know if it's something that you will be interrested about.
Sorry for the numerous grammatical mistakes but I'm not a native english speaker and I don't want to spend too much time reviewing my comment grammar
Thanks again to upvote this post on the 1st page of HN
- percept 12y agoThanks for the article and sharing so much of your experience.
- aaronem 12y agoThanks for the article, and I'd certainly be interested in seeing "When Postgres Search is Not Enough" -- it'd make an excellent companion piece to the one you've already written.
- wuliwong 12y agoYou just posted this today? This isn't the first time your article has been on the front page of HN. I actually still have your article "saved" in a tab in chrome from the last time it was on HN. It made quite an impression on me, I'm in the process of moving our dev environments from sqlite to postgres, so that we can start adding some of this functionality. Thanks.
- rachbelaid 12y agoTo be honest, I'm quite surprise to see that somebody posted this article again.. I don't know HN rules when you can post again something. I'm not going to complain about it :)
- why-el 12y agoI think you can repost about a year. It could have changed tough.
- wuliwong 12y agoI don't know when it was posted, but it definitely was less than one year ago. My main data point is that it still is living in an open tab in Chrome. I have a couple of sites which allow submitting links. I know people can get around my requirement that they are unique pretty easily. I'm sure HN is more sophisticated but I think that just having different url parameters might qualify as a different link? Sometimes a url parameter change really does point to a different page, so it might not be possible to eliminate duplicates that have the same url with just different params.