3 ms·

Thanks for the questions! > How is it better than SQL or other document querying representations? SQL is a nice language for querying relational data, but Ref

Thanks for the questions! > How is it better than SQL or other document querying representations? SQL is a nice language for querying relational data, but RefineDB deals with nested documents. I think translation from SQL is possible for the subset of RefineDB schemas where documents are not nested though. And RefineDB has types, and I’m not aware of any other query language that deal with types (let me know if I'm wrong). > What is a concurrent data flow virtual machine? The query language (RefineAsm) is a functional language with immutable semantics where values cannot be modified after construction. So a RefineAsm program can be encoded as a data flow graph where nodes are instructions and edges are data dependencies, and instructions that don’t depend on each other can be evaluated concurrently. Read accesses to a distributed key-value store usually incur at least a few hundred microseconds of latency, so by evaluating instructions concurrently the total latency can be minimized. Somewhat similar to what a SQL query planner does. > What is the contract to make this layer talk with the underlying stores A storage plan is generated from the schema (the schema is the type of the storage plan). The storage plan describes how paths of document accesses should map to the underlying key-value space. There are detailed description and examples of how this mapping is done at https://github.com/losfair/RefineDB/blob/main/design/storage.md https://github.com/losfair/RefineDB/blob/main/design/storage....

unraveller 5y ago
>other query language that deal with types EdgeDB has a similar strongly typed schema [1], see their edgeQL/graphQL ability. Their typed schema also facilitates computed references and computed anything, which is crucial for nested items as it keeps your DB calls much simpler. They make SQL style queries for nested data easy. On the topic of great QL i really love what deeprQL[2] brought to the table but it's JS only. [1] https://www.edgedb.com/docs/edgeql/index https://www.edgedb.com/docs/edgeql/index [2] https://github.com/deeprjs/deepr https://github.com/deeprjs/deepr
- cormacrelf 5y ago
  This is a bit off-topic, but I would really like to see a good query optimiser for GraphQL. Last I checked the industry standard solution to the "N+1 selects" problem that nested queries trigger was to add a delay and attempt to batch those N calls into one. I expect EdgeDB's GraphQL implementation just translates to EdgeQL/SQL and can simply optimise that. (Probably Hasura does this too.) But I have yet to see anything near that in a GraphQL server implementation for any programming language. From the GraphQL FAQ: > Accessing and manipulating data is performed with arbitrary functions called resolvers. ... Generally, these resolver functions should delegate to a business logic layer responsible for communicating with the various underlying data sources. These data sources could be remote APIs, databases, local cache, and nearly anything else your programming language can access. Somehow the lets-implement-this-syntax-in-100-popular-languages part of being a new language means these GraphQL server libraries just give you bare bones "get by id", "get all" interfaces to implement, and you're stuck with their dumb processing. This looks like a pretty standard experience of implementing GraphQL, and having to clear out a dozen jaw-droppingly dumb bugs introduced because the libraries do literally nothing for you: https://www.toptal.com/graphql/graphql-internal-api-optimization https://www.toptal.com/graphql/graphql-internal-api-optimiza.... Being a language, of course it must describe what the queries actually do in abstract, naive terms. Just like a programming language standard. But undefined behaviours are meant to be exploited, not used as a prototype for every implementation on Earth. I don't think GraphQL implementations are doing this nearly enough, considering that the goal is to build an SQL-like for arbitrary data backends. SQL databases ALL have optimisers. It's simply non-negotiable. For GraphQL, do whatever you need -- ask me which collections are available as tables in the same database so you can reorganise a query and send them as one "codegen unit", ask me to estimate the cost of using a resolver, ask me to build and keep indexes. That's the job, right?
  - 1st1 5y ago
    > I expect EdgeDB's GraphQL implementation just translates to EdgeQL/SQL and can simply optimise that. Yeah, that's what we do. Any GraphQL query can easily be compiled to an EdgeQL (as it's a superset language) query, which then can be compiled to one SQL query.
LukeEF 5y ago
TerminusDB's WOQL deals with types. As with RefineDB, TDB is designed to work with nested documents.
StreamBright 5y ago
>> SQL is a nice language for querying relational data, but RefineDB deals with nested documents. Can't you just expand the nested documents into a non-nested version and use references?
e12e 5y ago
> RefineDB has types, and I’m not aware of any other query language that deal with types (let me know if I'm wrong). In what sense is SQL not typed?

unraveller 5y ago

>other query language that deal with types EdgeDB has a similar strongly typed schema [1], see their edgeQL/graphQL ability. Their typed schema also facilitates computed references and computed anything, which is crucial for nested items as it keeps your DB calls much simpler. They make SQL style queries for nested data easy. On the topic of great QL i really love what deeprQL[2] brought to the table but it's JS only. [1] https://www.edgedb.com/docs/edgeql/index https://www.edgedb.com/docs/edgeql/index [2] https://github.com/deeprjs/deepr https://github.com/deeprjs/deepr

cormacrelf 5y ago
This is a bit off-topic, but I would really like to see a good query optimiser for GraphQL. Last I checked the industry standard solution to the "N+1 selects" problem that nested queries trigger was to add a delay and attempt to batch those N calls into one. I expect EdgeDB's GraphQL implementation just translates to EdgeQL/SQL and can simply optimise that. (Probably Hasura does this too.) But I have yet to see anything near that in a GraphQL server implementation for any programming language. From the GraphQL FAQ: > Accessing and manipulating data is performed with arbitrary functions called resolvers. ... Generally, these resolver functions should delegate to a business logic layer responsible for communicating with the various underlying data sources. These data sources could be remote APIs, databases, local cache, and nearly anything else your programming language can access. Somehow the lets-implement-this-syntax-in-100-popular-languages part of being a new language means these GraphQL server libraries just give you bare bones "get by id", "get all" interfaces to implement, and you're stuck with their dumb processing. This looks like a pretty standard experience of implementing GraphQL, and having to clear out a dozen jaw-droppingly dumb bugs introduced because the libraries do literally nothing for you: https://www.toptal.com/graphql/graphql-internal-api-optimization https://www.toptal.com/graphql/graphql-internal-api-optimiza.... Being a language, of course it must describe what the queries actually do in abstract, naive terms. Just like a programming language standard. But undefined behaviours are meant to be exploited, not used as a prototype for every implementation on Earth. I don't think GraphQL implementations are doing this nearly enough, considering that the goal is to build an SQL-like for arbitrary data backends. SQL databases ALL have optimisers. It's simply non-negotiable. For GraphQL, do whatever you need -- ask me which collections are available as tables in the same database so you can reorganise a query and send them as one "codegen unit", ask me to estimate the cost of using a resolver, ask me to build and keep indexes. That's the job, right?
- 1st1 5y ago
  > I expect EdgeDB's GraphQL implementation just translates to EdgeQL/SQL and can simply optimise that. Yeah, that's what we do. Any GraphQL query can easily be compiled to an EdgeQL (as it's a superset language) query, which then can be compiled to one SQL query.

LukeEF 5y ago

TerminusDB's WOQL deals with types. As with RefineDB, TDB is designed to work with nested documents.

StreamBright 5y ago

>> SQL is a nice language for querying relational data, but RefineDB deals with nested documents. Can't you just expand the nested documents into a non-nested version and use references?

e12e 5y ago

> RefineDB has types, and I’m not aware of any other query language that deal with types (let me know if I'm wrong). In what sense is SQL not typed?