3 ms·

This comment is full of hyperbole. Writing a precedence-climbing SQL parser should take a few days. I know because I've done it. I don't know where you're getti

by t8sr 3y ago

This comment is full of hyperbole. Writing a precedence-climbing SQL parser should take a few days. I know because I've done it. I don't know where you're getting "months of work" - mine ended up being like 600 lines of Python. And I don't know what you mean by "no sane way to comprehend SQL" - I guess the millions of data people in the industry are just insane? Cobbling it together with YAML or JSON is a reasonable trade off if you're in a hurry, but I don't understand how we got to the point where we're throwing out estimates like "writing a parser is months of work" and "it's a PhD project to render some glyphs [1]" 1: https://news.ycombinator.com/item?id=28743687 https://news.ycombinator.com/item?id=28743687

8note 3y ago
As far as "comprehend" goes, id think about getting typed responses to sql where I put in an arbitrary query and get a typed output. Generally, I've only seen ORMs create the query from the object, rather than the object from the query
- t8sr 3y ago
  I don’t understand the problem - is it that you don’t know the result type from just the query? (You need to consider the underlying schema) Or are you talking about something different?
frognumber 3y ago
I'm getting months of work because I need a production-grade system: - Reasonable performance - Test infrastructure - Pretty comprehensive SQL support. A lot of computation happens in the queries. - Maintainable - Documented Most things can be hacked together quickly, but that's different from correct production-grade code.
- t8sr 3y ago
  I was kind of assuming you had those things for a YAML based frontend, and just wanted to implement SQL support. I can see that if your YAML solution doesn’t have a way to express GROUP BY, so the backend doesn’t support it, then of course that’ll be extra work, but then that’s IMO a different feature. SQL itself is a tiny language - a parser that transforms it into your YAML based AST really would be pretty small. Here’s the one I made many years ago: https://github.com/google/dotty/blob/master/efilter/parsers/dottysql/parser.py https://github.com/google/dotty/blob/master/efilter/parsers/... It’s not the best quality code, and it doesn’t implement SQL92, but we did run it in production.
  - frognumber 3y ago
    My current JSON-based system doesn't implement full SQL semantics. We definitely don't have (or actually need) GROUP BY. Doing full SQL would require ASTs, a query optimizer, and similar. Right now, it's SQL-like, but the implementation is really quite dumb. What I actually want is the whole dotty / efilter system, only with things like documentation. We really do care about performance, though. From your code, this would not work: api.apply("SELECT name FROM users WHERE age > 10", vars={"users": ({"age": 10, "name": "Bob"}, {"age": 20, "name": "Alice"}, {"age": 30, "name": "Eve"})) It would really need to be: query = api.compile("SELECT name FROM users WHERE age > 10") query(vars={"users": ({"age": 10, "name": "Bob"}, {"age": 20, "name": "Alice"}, {"age": 30, "name": "Eve"}))) Or more likely, vars would be a closure which would allow it to interact with the proper data stores. As a footnote, that looks like a really nice project. I wish it were supported, maintained, and finished.

by t8sr 3y ago

8note 3y ago

As far as "comprehend" goes, id think about getting typed responses to sql where I put in an arbitrary query and get a typed output. Generally, I've only seen ORMs create the query from the object, rather than the object from the query

t8sr 3y ago
I don’t understand the problem - is it that you don’t know the result type from just the query? (You need to consider the underlying schema) Or are you talking about something different?

frognumber 3y ago

I'm getting months of work because I need a production-grade system: - Reasonable performance - Test infrastructure - Pretty comprehensive SQL support. A lot of computation happens in the queries. - Maintainable - Documented Most things can be hacked together quickly, but that's different from correct production-grade code.

t8sr 3y ago
I was kind of assuming you had those things for a YAML based frontend, and just wanted to implement SQL support. I can see that if your YAML solution doesn’t have a way to express GROUP BY, so the backend doesn’t support it, then of course that’ll be extra work, but then that’s IMO a different feature. SQL itself is a tiny language - a parser that transforms it into your YAML based AST really would be pretty small. Here’s the one I made many years ago: https://github.com/google/dotty/blob/master/efilter/parsers/dottysql/parser.py https://github.com/google/dotty/blob/master/efilter/parsers/... It’s not the best quality code, and it doesn’t implement SQL92, but we did run it in production.
- frognumber 3y ago
  My current JSON-based system doesn't implement full SQL semantics. We definitely don't have (or actually need) GROUP BY. Doing full SQL would require ASTs, a query optimizer, and similar. Right now, it's SQL-like, but the implementation is really quite dumb. What I actually want is the whole dotty / efilter system, only with things like documentation. We really do care about performance, though. From your code, this would not work: api.apply("SELECT name FROM users WHERE age > 10", vars={"users": ({"age": 10, "name": "Bob"}, {"age": 20, "name": "Alice"}, {"age": 30, "name": "Eve"})) It would really need to be: query = api.compile("SELECT name FROM users WHERE age > 10") query(vars={"users": ({"age": 10, "name": "Bob"}, {"age": 20, "name": "Alice"}, {"age": 30, "name": "Eve"}))) Or more likely, vars would be a closure which would allow it to interact with the proper data stores. As a footnote, that looks like a really nice project. I wish it were supported, maintained, and finished.