4 ms·

I would LOVE a layer built on top of postgres for reactive db events. +1 If it also configured into graphql like you were mentioning at one point, even sexier.

by jbhatab 10y ago

lobster_johnson 10y ago
We're preparing for an open source release of exactly that -- a document-oriented data store on top of Postgres and Elasticsearch, with transactions, joins, optional schemas, a query language, fine-grained document patches and change feeds. Email me if you want a ping when it's available.
- floatboth 10y ago
  Why Elasticsearch when Postgres has full-text search built in? It's one of my favorite Postgres features, I basically never have to bother with Elasticsearch/Solr/etc.
  - lobster_johnson 10y ago
    Good question. I hope to create a backend that actually stores data in Postgres, too. We don't really use Elasticsearch because of the fulltext support, although we do use that, too. Our document store is split into two parts: (1) A highly transactional data store (which uses Postgres) which stores master data and where everything is strict; and (2) an eventually-consistent search index (which uses Elasticsearch) where everything is expendable and queries are less strict. This has the benefit of allowing extreme horizontal scalability on the read path, without impacting the performance of the write path, but at the cost of less consistency. By dividing the two, we can control the flow of data into the write path; for example, some clients do batch imports that are indexed more slowly than real-time updates. The challenge with layering the search index on top of Postgres is how to represent the data. The document store manages the schemas for you, so we know to some extent what the data is, but also supports either partially or completely open schemas (where the allowed fields are either partially validated or completely schemaless). We could build tables dynamically from schemas, or we could denormalize the data into a less efficient [id, field, type, value] table, or we could index JSONB documents (GIN, not as efficient as B-trees on normal columns afaik, and limited in some ways). There are a bunch of options.
ruslan_talpa 10y ago
http://graphqlapi.com http://graphqlapi.com on top of OpenResty/PostgREST/PostgreSQL/RabbitMQ

I would LOVE a layer built on top of postgres for reactive db events. +1 If it also configured into graphql like you were mentioning at one point, even sexier.

by jbhatab 10y ago

I would LOVE a layer built on top of postgres for reactive db events. +1 If it also configured into graphql like you were mentioning at one point, even sexier.

lobster_johnson 10y ago

We're preparing for an open source release of exactly that -- a document-oriented data store on top of Postgres and Elasticsearch, with transactions, joins, optional schemas, a query language, fine-grained document patches and change feeds. Email me if you want a ping when it's available.

floatboth 10y ago
Why Elasticsearch when Postgres has full-text search built in? It's one of my favorite Postgres features, I basically never have to bother with Elasticsearch/Solr/etc.
- lobster_johnson 10y ago
  Good question. I hope to create a backend that actually stores data in Postgres, too. We don't really use Elasticsearch because of the fulltext support, although we do use that, too. Our document store is split into two parts: (1) A highly transactional data store (which uses Postgres) which stores master data and where everything is strict; and (2) an eventually-consistent search index (which uses Elasticsearch) where everything is expendable and queries are less strict. This has the benefit of allowing extreme horizontal scalability on the read path, without impacting the performance of the write path, but at the cost of less consistency. By dividing the two, we can control the flow of data into the write path; for example, some clients do batch imports that are indexed more slowly than real-time updates. The challenge with layering the search index on top of Postgres is how to represent the data. The document store manages the schemas for you, so we know to some extent what the data is, but also supports either partially or completely open schemas (where the allowed fields are either partially validated or completely schemaless). We could build tables dynamically from schemas, or we could denormalize the data into a less efficient [id, field, type, value] table, or we could index JSONB documents (GIN, not as efficient as B-trees on normal columns afaik, and limited in some ways). There are a bunch of options.

ruslan_talpa 10y ago

http://graphqlapi.com http://graphqlapi.com on top of OpenResty/PostgREST/PostgreSQL/RabbitMQ