4 ms·
I recently had to build a product search functionality. I was able to pick up nearly all important features of Elasticsearch in about 2 days and had live-indexi
by bladecatcher 10y ago
I recently had to build a product search functionality. I was able to pick up nearly all important features of Elasticsearch in about 2 days and had live-indexing and search up and running within 2 more days. Of course, this wasn't production level scalable code. But it was fairly easy to hook a Logical Decoding Output Plugin on Postgres, which would stream database mutations (in form of JSON) to a Kafka cluster, from where the Elasticsearch layer will ingest the data and index/update it appropriately.
- alexnewman 10y agowhat logical decoder?
- bladecatcher 10y agoPostgres 9.4+ lets you use the write-ahead log and "replication slots" using which you can create streams of database changes that can be replayed to a client in the order they were made on the origin server [1] EDIT: In case the question was which library did I use, I used BottledWater and it's worked great for me so far [2] [1] https://www.postgresql.org/docs/9.4/static/logicaldecoding-explanation.html https://www.postgresql.org/docs/9.4/static/logicaldecoding-e... [2] https://github.com/confluentinc/bottledwater-pg https://github.com/confluentinc/bottledwater-pg