4 ms·

Have a look at PRQL (www.prql-lang.org)!

by snthpy 4y ago

nmstoker 4y ago
Thanks for the reminder. I knew this was reminding me of something! Here's a previous discussion on PRQL: https://news.ycombinator.com/item?id=31897430 https://news.ycombinator.com/item?id=31897430
- snthpy 4y ago
  Full disclosure: I'm a PRQL contributor. There will be a big PRQL release in January which makes that previous discussion look quite dated. Best source for current state is the prql-lang.org website and the github repo (github.com/PRQL/prql). Also try the playground - there you can live queries on real data in your browser.
  - cbsmith 4y ago
    Hey, maybe you can help me with something about PRQL that I've never bothered to investigate (and I'll confess this is laziness on my part). If newlines just extend a pipeline, what separates pipelines? Do semicolons separate pipelines just like they separate queries in SQL?
    - snthpy 4y ago
      I don't recall us discussing that before. You can have named pipelines which basically become CTEs, like table employees_usa = (from employees | filter country=='USA') but ultimately there is usually just one "main" pipeline/query per file/input. Because PRQL only targets queries / DQL at the moment, I'm not sure what the sense multiple pipelines per file would be at this point. You're welcome to open an issue or start a discussion on the Github repo about your use case though, we'd love to hear about it!
      - usgroup 4y ago
        Does PRQL allow for a shared library? Ie can fragments be saved and reused?
        snthpy 4y ago
        That's definitely on the roadmap but we're not quite there yet. With the next release you will be able to create functions that are reusable pipeline segments (e.g. something like "top 5 items per group"). This works at the moment but is currently undocumented. Do expect there to be breaking changes on this still though as there are some issues around resolving names and namespaces etc... that we still need to work out. There is a (currently rather small) stdlib of functions but beyond that we still need to think about a good way to share code and create reusable libraries.
        usgroup 4y ago
        I think a composable SQL alternative would be great but if doesn't compose I personally don't see too much value in just an alternative syntax.
  - civilized 4y ago
    Thank you for giving so much credit to dplyr in your FAQ. The influence is obvious and I think dplyr deserves more credit for its revolutionary design, which has influenced a whole new generation of data manipulation tools. (I am not affiliated with dplyr/R/RStudio in any way, just a huge fan.)
SigmundA 4y ago
Nice reminds me of Linq, which looks like they acknowledge as one of their inspirations. SQL relational servers should really move more toward this kind of language. I always thought just building a pure .Net DB server using its types and language model would be very productive rather than translating to SQL, like PRQL or Entity Framework.
- manigandham 4y ago
  Have you seen RavenDB: https://ravendb.net/ https://ravendb.net/ It's a nosql/multimodal document store written in .NET and supports LINQ-like syntax.
  - SigmundA 4y ago
    Yes not my cup of tea. Had some poor experience with it in the past and don't really like its design philosophy unless somethings changed. Stores data as BSON I believe, which is better than text JSON I guess, but pretty ineffecient compared to something schema based that doesn't have to store keys with values for every every column/property. Didn't like the async indexing with stale results, nice to have as an option but the default should be sync/consistent. Written in .Net but not really using the .Net data type/model, more of a json db, used to use Windows only ESENT data engine but I think they got around to building their own K/V in .Net at some point. I am thinking something more like https://velocitydb.com https://velocitydb.com
    - manigandham 4y ago
      RavenDB has changed massively and your assumptions are about 6 years and 2 major versions out of date. I suggest trying it again as none of those are issues anymore. > ".Net but not really using the .Net data type/model" What's that mean? The .NET driver has seamless object persistence.
      - SigmundA 4y ago
        That means at the storage level it's storing a JSON data model not a .Net one. That means the length of a property name matters because its stored with every document rather than having the type info stored in a schema and the individual object instances only storing the values, similar to how a relational db stores columns names in a schema and rows only have data. This can save significant space and overhead. If I am designing a .Net database from scratch with Linq as its target query language then I would want the storage engine to understand and take advantage of types to optimize storage with minimal translation rather than having to serialize/deserialize from .Net to BSON/JSON with key overhead included.
usgroup 4y ago
Thinking about why the PRQL syntax is as it is, does it have something to do with chaining CTEs together? I roughly see how given a pipeline with discrete components you could translate it into a collection of CTEs which lead into each other.

Have a look at PRQL (www.prql-lang.org)!

by snthpy 4y ago

Have a look at PRQL (www.prql-lang.org)!

Thanks for the reminder. I knew this was reminding me of something! Here's a previous discussion on PRQL: https://news.ycombinator.com/item?id=31897430 https://news.ycombinator.com/item?id=31897430

snthpy 4y ago
Full disclosure: I'm a PRQL contributor. There will be a big PRQL release in January which makes that previous discussion look quite dated. Best source for current state is the prql-lang.org website and the github repo (github.com/PRQL/prql). Also try the playground - there you can live queries on real data in your browser.
- cbsmith 4y ago
  Hey, maybe you can help me with something about PRQL that I've never bothered to investigate (and I'll confess this is laziness on my part). If newlines just extend a pipeline, what separates pipelines? Do semicolons separate pipelines just like they separate queries in SQL?
  - snthpy 4y ago
    I don't recall us discussing that before. You can have named pipelines which basically become CTEs, like table employees_usa = (from employees | filter country=='USA') but ultimately there is usually just one "main" pipeline/query per file/input. Because PRQL only targets queries / DQL at the moment, I'm not sure what the sense multiple pipelines per file would be at this point. You're welcome to open an issue or start a discussion on the Github repo about your use case though, we'd love to hear about it!
    - usgroup 4y ago
      Does PRQL allow for a shared library? Ie can fragments be saved and reused?
      - snthpy 4y ago
        That's definitely on the roadmap but we're not quite there yet. With the next release you will be able to create functions that are reusable pipeline segments (e.g. something like "top 5 items per group"). This works at the moment but is currently undocumented. Do expect there to be breaking changes on this still though as there are some issues around resolving names and namespaces etc... that we still need to work out. There is a (currently rather small) stdlib of functions but beyond that we still need to think about a good way to share code and create reusable libraries.
        usgroup 4y ago
        I think a composable SQL alternative would be great but if doesn't compose I personally don't see too much value in just an alternative syntax.
- civilized 4y ago
  Thank you for giving so much credit to dplyr in your FAQ. The influence is obvious and I think dplyr deserves more credit for its revolutionary design, which has influenced a whole new generation of data manipulation tools. (I am not affiliated with dplyr/R/RStudio in any way, just a huge fan.)

SigmundA 4y ago

Nice reminds me of Linq, which looks like they acknowledge as one of their inspirations. SQL relational servers should really move more toward this kind of language. I always thought just building a pure .Net DB server using its types and language model would be very productive rather than translating to SQL, like PRQL or Entity Framework.

manigandham 4y ago
Have you seen RavenDB: https://ravendb.net/ https://ravendb.net/ It's a nosql/multimodal document store written in .NET and supports LINQ-like syntax.
- SigmundA 4y ago
  Yes not my cup of tea. Had some poor experience with it in the past and don't really like its design philosophy unless somethings changed. Stores data as BSON I believe, which is better than text JSON I guess, but pretty ineffecient compared to something schema based that doesn't have to store keys with values for every every column/property. Didn't like the async indexing with stale results, nice to have as an option but the default should be sync/consistent. Written in .Net but not really using the .Net data type/model, more of a json db, used to use Windows only ESENT data engine but I think they got around to building their own K/V in .Net at some point. I am thinking something more like https://velocitydb.com https://velocitydb.com
  - manigandham 4y ago
    RavenDB has changed massively and your assumptions are about 6 years and 2 major versions out of date. I suggest trying it again as none of those are issues anymore. > ".Net but not really using the .Net data type/model" What's that mean? The .NET driver has seamless object persistence.
    - SigmundA 4y ago
      That means at the storage level it's storing a JSON data model not a .Net one. That means the length of a property name matters because its stored with every document rather than having the type info stored in a schema and the individual object instances only storing the values, similar to how a relational db stores columns names in a schema and rows only have data. This can save significant space and overhead. If I am designing a .Net database from scratch with Linq as its target query language then I would want the storage engine to understand and take advantage of types to optimize storage with minimal translation rather than having to serialize/deserialize from .Net to BSON/JSON with key overhead included.

usgroup 4y ago

Thinking about why the PRQL syntax is as it is, does it have something to do with chaining CTEs together? I roughly see how given a pipeline with discrete components you could translate it into a collection of CTEs which lead into each other.