4 ms·

Do you see any particular reasons to use or not use ZomboDB [1]? It claims to lets you use ElasticSearch from PG seamlessly e.g. it manages coherency of which r

Do you see any particular reasons to use or not use ZomboDB [1]? It claims to lets you use ElasticSearch from PG seamlessly e.g. it manages coherency of which results ought to be returned according to the current transaction. (I've never quite ended up needing to use ES but it's always seemed to me I'd be likely to need ZomboDB if I did.) [1] https://github.com/zombodb/zombodb https://github.com/zombodb/zombodb

mixmastamyk 5y ago
You can do anything, anything at all, at https://zombo.com/ https://zombo.com/ "The only limit, is yourself…"
- habibur 5y ago
  It's still around?
  - mixmastamyk 5y ago
    Klicken-Sie Link.
  - mumblemumble 5y ago
    That seems like one for the philosophers. If you completely rewrite a Flash site in HTML5, but it looks the same and has the same URL, is it still the same site?
    - da_chicken 5y ago
      So, YouTube?
- FpUser 5y ago
  This one is my favorite for many many years. Good for relaxing.
Conlectus 5y ago
Though I have only a recreational interest in datastores, I would be pretty wary of a service that claims to strap a Consistant-Unavailable database (Postgres) to an Inconsistent-Available database (ElasticSearch) in a useful way. Doing so would require ElasticSearch to reach consensus on every read/write, which would remove most of the point of a distributed cluster. Despite this, ZomboDB's documentation says "complex aggregate queries can be answered in parallel across your ElasticSearch cluster". They also claim that transactions will abort if ElasticSearch runs into network trouble, but the ElasticSearch documentation notes that writes during network partitions don't wait for confirmation of success[1], so I'm not sure how they would be able to detect that. In short: I'll wait for the Jepsen analysis. [1] https://www.elastic.co/blog/tracking-in-sync-shard-copies#:~:text=chooses%20write%20availability https://www.elastic.co/blog/tracking-in-sync-shard-copies#:~...
- zombodb 5y ago
  > Doing so would require ElasticSearch to reach consensus on every read/write ZomboDB only requires that ES have a view of its index that's consistent with the active Postgres transaction snapshot. ZDB handles this by ensuring that the ES index is fully refreshed after writes. This doesn't necessarily make ZDB great for high-update loads, but that's not ZDB's target usage. > They also claim that transactions will abort if ElasticSearch runs into network trouble... I had to search my own repo to see where I make this claim. I don't. I do note that network failures between PG & ES will cause the active Postgres xact to abort. On top of that, any error that ES is capable of reporting back to the client will cause the PG xact to abort -- ensuring consistency between the two. Because the ES index is properly refreshed as it relates to the active Postgres transaction, all of ES' aggregate search functions are capable of providing proper MVCC-correct results, using the parallelism provided by the ES cluster. I don't have the time to detail everything that ZDB does to project Postgres xact snapshots on top of ES, but the above two points are the easy ones to solve.
  - mrslave 5y ago
    ZomboDB scratches an itch in a way I find fascinating, though I have yet to do more with it than shoehorn it into a prototype that was a bit square-peg-in-a-round-hole. It is in my catalogue of technologies I hope exploit someday. And I hope you're enjoying the work and making some $$$ too. While I'm here might I ask, are you finding the hosted PostgreSQL services (AWS, Azure, etc.) growing or shrinking your market opportunities? Also, does it play nice with Citus?

mixmastamyk 5y ago

You can do anything, anything at all, at https://zombo.com/ https://zombo.com/ "The only limit, is yourself…"

habibur 5y ago
It's still around?
- mixmastamyk 5y ago
  Klicken-Sie Link.
- mumblemumble 5y ago
  That seems like one for the philosophers. If you completely rewrite a Flash site in HTML5, but it looks the same and has the same URL, is it still the same site?
  - da_chicken 5y ago
    So, YouTube?
FpUser 5y ago
This one is my favorite for many many years. Good for relaxing.

Conlectus 5y ago

Though I have only a recreational interest in datastores, I would be pretty wary of a service that claims to strap a Consistant-Unavailable database (Postgres) to an Inconsistent-Available database (ElasticSearch) in a useful way. Doing so would require ElasticSearch to reach consensus on every read/write, which would remove most of the point of a distributed cluster. Despite this, ZomboDB's documentation says "complex aggregate queries can be answered in parallel across your ElasticSearch cluster". They also claim that transactions will abort if ElasticSearch runs into network trouble, but the ElasticSearch documentation notes that writes during network partitions don't wait for confirmation of success[1], so I'm not sure how they would be able to detect that. In short: I'll wait for the Jepsen analysis. [1] https://www.elastic.co/blog/tracking-in-sync-shard-copies#:~:text=chooses%20write%20availability https://www.elastic.co/blog/tracking-in-sync-shard-copies#:~...

zombodb 5y ago
> Doing so would require ElasticSearch to reach consensus on every read/write ZomboDB only requires that ES have a view of its index that's consistent with the active Postgres transaction snapshot. ZDB handles this by ensuring that the ES index is fully refreshed after writes. This doesn't necessarily make ZDB great for high-update loads, but that's not ZDB's target usage. > They also claim that transactions will abort if ElasticSearch runs into network trouble... I had to search my own repo to see where I make this claim. I don't. I do note that network failures between PG & ES will cause the active Postgres xact to abort. On top of that, any error that ES is capable of reporting back to the client will cause the PG xact to abort -- ensuring consistency between the two. Because the ES index is properly refreshed as it relates to the active Postgres transaction, all of ES' aggregate search functions are capable of providing proper MVCC-correct results, using the parallelism provided by the ES cluster. I don't have the time to detail everything that ZDB does to project Postgres xact snapshots on top of ES, but the above two points are the easy ones to solve.
- mrslave 5y ago
  ZomboDB scratches an itch in a way I find fascinating, though I have yet to do more with it than shoehorn it into a prototype that was a bit square-peg-in-a-round-hole. It is in my catalogue of technologies I hope exploit someday. And I hope you're enjoying the work and making some $$$ too. While I'm here might I ask, are you finding the hosted PostgreSQL services (AWS, Azure, etc.) growing or shrinking your market opportunities? Also, does it play nice with Citus?