5 ms·

> SQL is great at expressing simple needs very well, but in more advanced cases, it can get "mind-twistingly complicated" even for a mathematically included bra

by mrtimo 2y ago

> SQL is great at expressing simple needs very well, but in more advanced cases, it can get "mind-twistingly complicated" even for a mathematically included brain. I have found Malloy[1] to be easier to read and write, even for really complex queries. Malloy complies to SQL. As an example check out [2]. Malloy can query .csv files directly (enabled by duckdb). You can also join .csv files. This makes getting started a breeze for beginners, and is a boon for data scientists. Much easier than Pandas in many cases. With the visualization capabilities it has built in, Malloy sort-of competes with Tableau and PowerBI. Most people only write SQL to get data out of databases, for these people, Malloy is an interesting tool to consider. [1] https://www.malloydata.dev/ https://www.malloydata.dev/ [2] https://docs.malloydata.dev/blog/2023-10-26-malloy-bump-chart/ https://docs.malloydata.dev/blog/2023-10-26-malloy-bump-char... Not affiliated with Malloy, I've just played with it for a while and been impressed. It's also MIT licensed.

myaccountonhn 2y ago
There’s also recutils for those that have simpler data storage needs.
mikpanko 2y ago
Malloy is great. Why do you think it is not taking off if it is a clear significant improvement on SQL and even compiles to it?
- bdcravens 2y ago
  Limited support https://docs.malloydata.dev/documentation/ https://docs.malloydata.dev/documentation/ "Malloy currently works with SQL databases BigQuery, Postgres, and querying Parquet and CSV via DuckDB."
- emmanueloga_ 2y ago
  Languages like Malloy or PRQL require some upfront investment to learn and setup. Additionally, I think both Malloy and PRQL are query oriented, so you still need to learn a decent amount of SQL to interact with your database (for updates). I'm guessing most people would rather spend effort in solving their immediate problems with SQL rather than bet on a newer and less known technology, even if has a lot of promise.
- arp242 2y ago
  Looking at this for a bit, there's a few reasons: For starters, according to that link it only left "experimental status" in Oct 2023; it's pretty new. Although their GitHub still describes it as "an experimental language". That doesn't exactly inspire confidence for long-term production use. The second problem is that while imperfect, everyone and their dog knows at least the SQL basics. There's a lot of value in that. Thirdly it's written in TypeScript, so anyone not using TypeScript will either have to run some sort of "generate" step, or have to rewrite it in $language_of_choice. Both are painful. This is not a TypeScript problem: you will have that with any language (although with languages that compile to a binary it's a bit less painful, albeit still painful). Lastly, it doesn't work for all SQL flavours: just BigQuery and PostgreSQL. That's pretty limited. All of that is assuming the SQL it generates performs as well as "native" SQL, and that all of this can be reasonably debugged if something goes wrong.
- mrtimo 2y ago
  I see Malloy as an analysis tool like Pandas, PowerBI, Tableau, or Looker. It seems like most people write SQL to get "All the data" and then take it to an analysis tool for further study. With Malloy you can do the analysis on your data lake directly for 80% of your questions.

by mrtimo 2y ago

myaccountonhn 2y ago

There’s also recutils for those that have simpler data storage needs.

mikpanko 2y ago

Malloy is great. Why do you think it is not taking off if it is a clear significant improvement on SQL and even compiles to it?

bdcravens 2y ago
Limited support https://docs.malloydata.dev/documentation/ https://docs.malloydata.dev/documentation/ "Malloy currently works with SQL databases BigQuery, Postgres, and querying Parquet and CSV via DuckDB."
emmanueloga_ 2y ago
Languages like Malloy or PRQL require some upfront investment to learn and setup. Additionally, I think both Malloy and PRQL are query oriented, so you still need to learn a decent amount of SQL to interact with your database (for updates). I'm guessing most people would rather spend effort in solving their immediate problems with SQL rather than bet on a newer and less known technology, even if has a lot of promise.
arp242 2y ago
Looking at this for a bit, there's a few reasons: For starters, according to that link it only left "experimental status" in Oct 2023; it's pretty new. Although their GitHub still describes it as "an experimental language". That doesn't exactly inspire confidence for long-term production use. The second problem is that while imperfect, everyone and their dog knows at least the SQL basics. There's a lot of value in that. Thirdly it's written in TypeScript, so anyone not using TypeScript will either have to run some sort of "generate" step, or have to rewrite it in $language_of_choice. Both are painful. This is not a TypeScript problem: you will have that with any language (although with languages that compile to a binary it's a bit less painful, albeit still painful). Lastly, it doesn't work for all SQL flavours: just BigQuery and PostgreSQL. That's pretty limited. All of that is assuming the SQL it generates performs as well as "native" SQL, and that all of this can be reasonably debugged if something goes wrong.
mrtimo 2y ago
I see Malloy as an analysis tool like Pandas, PowerBI, Tableau, or Looker. It seems like most people write SQL to get "All the data" and then take it to an analysis tool for further study. With Malloy you can do the analysis on your data lake directly for 80% of your questions.