4 ms·
The parser.py [1] has only 1.6k lines. And it is hand-written parser. This size is amazing if it's really capable, but I intuitively doubt it. For example, duck
by flakiness 4y ago
The parser.py [1] has only 1.6k lines. And it is hand-written parser. This size is amazing if it's really capable, but I intuitively doubt it. For example, duckdb's select.y [2] has 3700 lines, and this is only for SELECT. ZetaSQL's grammar file [3] is almost 10k lines.
The SQL is a monstrous language. Is there any trick that keeps the code simple?
[1] https://github.com/tobymao/sqlglot/blob/main/sqlglot/parser.py https://github.com/tobymao/sqlglot/blob/main/sqlglot/parser....
[2] https://github.com/duckdb/duckdb/blob/master/third_party/libpg_query/grammar/statements/select.y https://github.com/duckdb/duckdb/blob/master/third_party/lib...
[3] https://github.com/google/zetasql/blob/master/zetasql/parser/bison_parser.y https://github.com/google/zetasql/blob/master/zetasql/parser...
- ramraj07 4y agoThe parser.py seems extremely terse, not a line of doc strings or typing. Perhaps that explains?
- zasdffaa 4y agoNo, SQL isd big an ugly. I wonder if it can do select ... (select ...) as some_alias, ... from ... also, can't see SOME/ANY/ALL
- captaintobs 4y agoSQLGlot supports nested queries and EXISTS. SOME, ANY, and ALL aren't fully supported yet but it's only a couple lines to add support since I already support EXISTS. https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/identity.sql#L100 https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/... https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/identity.sql#L154 https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/...
- zasdffaa 4y agoI'm impressed.
- captaintobs 4y agoI've added support now for any, all, and some. Some is identical to Any so I always default to Any. You can see what it took to add this in this commit. https://github.com/tobymao/sqlglot/commit/ab49a3a2964bdff3a3423b4b3cb360987ce07da3 https://github.com/tobymao/sqlglot/commit/ab49a3a2964bdff3a3...
- einpoklum 4y agoIf you look at DuckDB's select.py, you'll notice very "generous" use of newlines, e.g. putting just a ";" on a line, or just "{", or just "}" etc. Partially explains the difference I suppose.
- codetrotter 4y agoNitpick: The file is called select.y, not select.py Was confused seeing these mentions of a select.py and why there would be any noticeable amount of semicolons in a Python file.
- flakiness 4y agoGood catch! Fixed.
- hardware2win 4y agoMaybe they're just good at computers The code looks good
- captaintobs 4y agoYou should try it out and see. Here are the test cases https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/identity.sql https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/... I can even parse and optimize TPC-H https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/optimizer/tpc-h/tpc-h.sql https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/.... SQLGlot's parser handles a superset of SQL, so it's much more flexible than any one engine.