7 ms·

JavaScript query language to perform complex object searches

52 points by jviotti 11y ago

wppick 11y ago
Looks similar to sequelize http://docs.sequelizejs.com/en/latest/ http://docs.sequelizejs.com/en/latest/
- amirouche 11y ago
  Looks similar to sequelize query API, direct link: http://docs.sequelizejs.com/en/latest/docs/querying/ http://docs.sequelizejs.com/en/latest/docs/querying/ This kind of things looks better in homoiconic languages. I'm wondering how that approach compares to rho/rewriting calculus.
- jviotti 11y ago
  Looks cool, I haven't heard of it. However I don't think I could use their querying language separately from the ORM. Our use case for Queryl is to filter array of objects based on different criteria, but I guess an extra benefit is that the query language is completely decoupled from its usage, so anyone can use it in their own projects for different needs.
faldore 11y ago
I use lodash
lodash_happy 11y ago
how's this better than just using lodash?
- jviotti 11y ago
  The project heavily relies on lodash for it's implementation actually, so you can of course do it straight with lodash, however the benefit is that you don't have to write any code to produce complex searches (with multiple nested levels of $or and $and), which would require a lot of conditionals if done straight with JavaScript. Another benefit (which was the reason behind the implementation), is that we can stringify the search object to easily persist and share it, which would not be that easy with plain code.
MalcolmDwyer 11y ago
Along similar lines, I just made this library a couple weeks ago to describe data validations programmatically (i.e., one JSON to validate another JSON document). https://github.com/MalcolmDwyer/json-predicate https://github.com/MalcolmDwyer/json-predicate
jlas 11y ago
Looks similar to MongoDB query syntax [1]. Is it inspired by that? What are similarities/differences? [1] http://docs.mongodb.org/master/tutorial/query-documents/ http://docs.mongodb.org/master/tutorial/query-documents/
- WhitneyLand 11y ago
  Yes it looks very similar, having built a query builder for it: http://youtu.be/fZ3_NGLKE2U http://youtu.be/fZ3_NGLKE2U
- jviotti 11y ago
  Yeah, it was inspired by MongoDB, but then deviates in some ways: - Conjunction and disjunction operations are defined by arrays in MongoDB, but as Objects in Queryl. For example: { $or: [ { foo: { $gt: 100 } }, { bar: { $gt: 200 } } ] } vs: { $or: { $gt: { foo: 100, bar: 200 } } } - As you can notice in the above example, in MongoDB, you declare the property and the specific query inside, while in Queryl is the opposite: you declare the operation, and then the property. - Of course, Queryl is still in it's infancy, and doesn't support as many operations as MongoDB does (but will be added eventually). There are probably more differences that I'm not spotting due to lack of in depth knowledge in MongoDB queries.
grumblestumble 11y ago
looks very similar to siftjs: https://github.com/crcn/sift.js/tree/master https://github.com/crcn/sift.js/tree/master
lgas 11y ago
Also similar to https://github.com/raine/ramda-cli https://github.com/raine/ramda-cli which was posted recently.
tcdent 11y ago
This could use a touch more intelligence to cut down on a lot of the noise. You could eliminate `$equal` and `$match` by looking for regex types and strings. You could also take hints from Django and add operations to the keys to eliminate instantiating a new object for `$not`, `$gt` and `$lt`. So, instead of: queryl.match({ $or: { $equal: { foo: 'bar' }, $and: { $not: { $match: { foo: /^baz/ } }, $gt: { bar: 3 } } }, ... ...it would look like: queryl.match({ $or: { foo: 'bar', $and: { foo__not: /^baz/, bar__gt: 3 } }, ... Seems like this syntax is popular; is there something about this that doesn't work when serializing, or other uses that need the pure-object approach?
- deleted 11y ago
  [deleted]
- jviotti 11y ago
  Hey there! Your second example was the original idea when building Queryl, however that required me to always inspect the properties of an object, and based on the key, check if it is a Queryl operation (e.g: starts with $ and it's recognised), or a simple property to match for equality. Also, it had the edge case in which a user's object properties could collide with the operations defined by Queryl, which (I believe, didn't write tests for it) doesn't happen with the current approach.
jestar_jokin 11y ago
An alternative approach is to use XPath-inspired DSLs that work on JS or JSON object structures. These tend to work on strings, rather than declarative query objects. Examples: JSPath: https://github.com/dfilatov/jspath https://github.com/dfilatov/jspath JSONPath: https://github.com/s3u/JSONPath https://github.com/s3u/JSONPath (Since you don't have type safety in JS, I don't think you lose much by using strings, unless you want to wrap it in TypeScript.)
- smizell 11y ago
  Or you could actually use XPath :) https://www-01.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/com.ibm.dp.doc/json_jsonxconversionexample.html https://www-01.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/...
smizell 11y ago
It always feels like we keep reinventing Lisp. You could do something like: ['contains?', ['list', 1, 2, 3], 1] Instead of: { $contain: { foo: 1 } }, { foo: [ 1, 2, 3 ] }
- jviotti 11y ago
  Haha definitely! You could write a small module that compiles the lisp-like syntax that you mention above to a Queryl object.
voxic11 11y ago
I have been enjoying https://linqjs.codeplex.com/ https://linqjs.codeplex.com/
smizell 11y ago
I've been working on library that creates a queryable structure for trees like this. I prefer writing normal JS than to having some query language for it. var obj = minim.toElement({ a: 1, b: [ { c: 1, d: 2, e: [ { f: 5 } ], f: 6 } ] }); // Results are [5, 6] var results = obj.find(function(value, key) { return key.equals('f'); }); See https://github.com/refractproject/minim https://github.com/refractproject/minim
smizell 11y ago
I've been working on library that creates a DOM-like structure for trees like this. I prefer writing normal JS than to having some query language for it. var obj = minim.toElement({ a: 1, b: [ { c: 1, d: 2, e: [ { f: 5 } ], f: 6 } ] }); // Results are [5, 6] var results = obj.find(function(value, key) { return key.equals('f'); }); See https://github.com/refractproject/minim https://github.com/refractproject/minim
plaguuuuuu 11y ago
Can anyone explain why you might need to store queries as JSON data with horrifying syntax, instead of just writing plain functions that search/filter/whatever? Or if you really have to, using a sensible DSL to do the job instead?
- jviotti 11y ago
  Hey there, Objects (or DSL) are easier to encode/decode than functions. Also notice that Queryl is a very low level way of defining queries, and can easily become horrifying and unmaintainable with very complex queries. The idea was to provide the low level, verbose way of defining this, and provide a user friendly frontend (as a DSL, probably) that compiles to it.

52 points by jviotti 11y ago

wppick 11y ago

Looks similar to sequelize http://docs.sequelizejs.com/en/latest/ http://docs.sequelizejs.com/en/latest/

amirouche 11y ago
Looks similar to sequelize query API, direct link: http://docs.sequelizejs.com/en/latest/docs/querying/ http://docs.sequelizejs.com/en/latest/docs/querying/ This kind of things looks better in homoiconic languages. I'm wondering how that approach compares to rho/rewriting calculus.
jviotti 11y ago
Looks cool, I haven't heard of it. However I don't think I could use their querying language separately from the ORM. Our use case for Queryl is to filter array of objects based on different criteria, but I guess an extra benefit is that the query language is completely decoupled from its usage, so anyone can use it in their own projects for different needs.

faldore 11y ago

I use lodash

lodash_happy 11y ago

how's this better than just using lodash?

jviotti 11y ago
The project heavily relies on lodash for it's implementation actually, so you can of course do it straight with lodash, however the benefit is that you don't have to write any code to produce complex searches (with multiple nested levels of $or and $and), which would require a lot of conditionals if done straight with JavaScript. Another benefit (which was the reason behind the implementation), is that we can stringify the search object to easily persist and share it, which would not be that easy with plain code.

MalcolmDwyer 11y ago

Along similar lines, I just made this library a couple weeks ago to describe data validations programmatically (i.e., one JSON to validate another JSON document). https://github.com/MalcolmDwyer/json-predicate https://github.com/MalcolmDwyer/json-predicate

jlas 11y ago

Looks similar to MongoDB query syntax [1]. Is it inspired by that? What are similarities/differences? [1] http://docs.mongodb.org/master/tutorial/query-documents/ http://docs.mongodb.org/master/tutorial/query-documents/

WhitneyLand 11y ago
Yes it looks very similar, having built a query builder for it: http://youtu.be/fZ3_NGLKE2U http://youtu.be/fZ3_NGLKE2U
jviotti 11y ago
Yeah, it was inspired by MongoDB, but then deviates in some ways: - Conjunction and disjunction operations are defined by arrays in MongoDB, but as Objects in Queryl. For example: { $or: [ { foo: { $gt: 100 } }, { bar: { $gt: 200 } } ] } vs: { $or: { $gt: { foo: 100, bar: 200 } } } - As you can notice in the above example, in MongoDB, you declare the property and the specific query inside, while in Queryl is the opposite: you declare the operation, and then the property. - Of course, Queryl is still in it's infancy, and doesn't support as many operations as MongoDB does (but will be added eventually). There are probably more differences that I'm not spotting due to lack of in depth knowledge in MongoDB queries.

grumblestumble 11y ago

looks very similar to siftjs: https://github.com/crcn/sift.js/tree/master https://github.com/crcn/sift.js/tree/master

lgas 11y ago

Also similar to https://github.com/raine/ramda-cli https://github.com/raine/ramda-cli which was posted recently.

tcdent 11y ago

This could use a touch more intelligence to cut down on a lot of the noise. You could eliminate `$equal` and `$match` by looking for regex types and strings. You could also take hints from Django and add operations to the keys to eliminate instantiating a new object for `$not`, `$gt` and `$lt`. So, instead of: queryl.match({ $or: { $equal: { foo: 'bar' }, $and: { $not: { $match: { foo: /^baz/ } }, $gt: { bar: 3 } } }, ... ...it would look like: queryl.match({ $or: { foo: 'bar', $and: { foo__not: /^baz/, bar__gt: 3 } }, ... Seems like this syntax is popular; is there something about this that doesn't work when serializing, or other uses that need the pure-object approach?

deleted 11y ago
[deleted]
jviotti 11y ago
Hey there! Your second example was the original idea when building Queryl, however that required me to always inspect the properties of an object, and based on the key, check if it is a Queryl operation (e.g: starts with $ and it's recognised), or a simple property to match for equality. Also, it had the edge case in which a user's object properties could collide with the operations defined by Queryl, which (I believe, didn't write tests for it) doesn't happen with the current approach.

jestar_jokin 11y ago

An alternative approach is to use XPath-inspired DSLs that work on JS or JSON object structures. These tend to work on strings, rather than declarative query objects. Examples: JSPath: https://github.com/dfilatov/jspath https://github.com/dfilatov/jspath JSONPath: https://github.com/s3u/JSONPath https://github.com/s3u/JSONPath (Since you don't have type safety in JS, I don't think you lose much by using strings, unless you want to wrap it in TypeScript.)

smizell 11y ago
Or you could actually use XPath :) https://www-01.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/com.ibm.dp.doc/json_jsonxconversionexample.html https://www-01.ibm.com/support/knowledgecenter/SS9H2Y_7.1.0/...

smizell 11y ago

It always feels like we keep reinventing Lisp. You could do something like: ['contains?', ['list', 1, 2, 3], 1] Instead of: { $contain: { foo: 1 } }, { foo: [ 1, 2, 3 ] }

jviotti 11y ago
Haha definitely! You could write a small module that compiles the lisp-like syntax that you mention above to a Queryl object.

voxic11 11y ago

I have been enjoying https://linqjs.codeplex.com/ https://linqjs.codeplex.com/

smizell 11y ago

I've been working on library that creates a queryable structure for trees like this. I prefer writing normal JS than to having some query language for it. var obj = minim.toElement({ a: 1, b: [ { c: 1, d: 2, e: [ { f: 5 } ], f: 6 } ] }); // Results are [5, 6] var results = obj.find(function(value, key) { return key.equals('f'); }); See https://github.com/refractproject/minim https://github.com/refractproject/minim

smizell 11y ago

I've been working on library that creates a DOM-like structure for trees like this. I prefer writing normal JS than to having some query language for it. var obj = minim.toElement({ a: 1, b: [ { c: 1, d: 2, e: [ { f: 5 } ], f: 6 } ] }); // Results are [5, 6] var results = obj.find(function(value, key) { return key.equals('f'); }); See https://github.com/refractproject/minim https://github.com/refractproject/minim

plaguuuuuu 11y ago

Can anyone explain why you might need to store queries as JSON data with horrifying syntax, instead of just writing plain functions that search/filter/whatever? Or if you really have to, using a sensible DSL to do the job instead?

jviotti 11y ago
Hey there, Objects (or DSL) are easier to encode/decode than functions. Also notice that Queryl is a very low level way of defining queries, and can easily become horrifying and unmaintainable with very complex queries. The idea was to provide the low level, verbose way of defining this, and provide a user friendly frontend (as a DSL, probably) that compiles to it.