3 ms·

Hi can you tell us a bit more about this SQL-related project? I'm rather practically interested.

by knz42 9y ago

arnon 9y ago
I work for www.sqream.com, and our main product is SQream DB. SQream DB is a GPU SQL database for analytics. Everything was written in-house - from the SQL parser all the way down to the storage layer. We're designed to deal with sizes from a few terabytes to hundreds of terabytes, and we use the GPU to do most of the work. It is the compiler (written in Haskell) however, that decides about specific optimizations and deals with generating the query plan. Our SQL parser and typechecker is actually BSD3 licensed, if it's interesting: https://github.com/jakewheat/hssqlppp https://github.com/jakewheat/hssqlppp
- shusson 9y ago
  Any chance SQream DB will be open sourced or a demo version? It seems every Column based GPU DB I find is behind a paywall and targeting enterprise levels.
  - arnon 9y ago
    I think it's unlikely at this point... We too are currently during some large projects for enterprises... We will be releasing a community version on AWS and Azure in the near future which should be cheap or free, other than the instance cost on the respective cloud provider.
- Veedrac 9y ago
  > www.sqream.com I don't like having to be that guy, but your landing page hijacks scrolling (which always causes problems; I noticed first because it broke ctrl-scroll zooming), downloads a 30MB video in a loop, forever, takes a significant amount of CPU even when the video isn't in view and the tab is not active, and despite having almost no content manages to take far more memory than any other non-email tab I have open.
  - arnon 9y ago
    Our website is... Yeah... I know... :/ I've passed on your comments in any case, but know that we're in the process of rebuilding the website from scratch.
dllthomas 9y ago
If you want more generic SQL tooling in Haskell, my project was just recently open sourced (https://github.com/uber/queryparser https://github.com/uber/queryparser). It currently has support for Vertica, Hive, and Presto; adding support for more dialects isn't complicated. I'm working on cleaning it up for hackage.
- arnon 9y ago
  Very interesting! Do you do any sort of typechecking in this? FWIW, HsSqlPpp also supports various dialects. In SQream DB we use a custom dialect, while HsSqlPpp is mostly Postgres.
  - dllthomas 9y ago
    Typechecking of the SQL statements? There was an internal branch where I was experimenting with that, but it didn't get landed before I left, and isn't in the squashed version. It was relatively simplistic - along the lines of "things tested for equality must share a type". It was also focused on domain-relevant types, not SQL type (/representation). > FWIW, HsSqlPpp also supports various dialects. In SQream DB we use a custom dialect, while HsSqlPpp is mostly Postgres. Nice. It'll be interesting to see where our implementation choices differed, and what we can learn from each other :)

Hi can you tell us a bit more about this SQL-related project? I'm rather practically interested.

by knz42 9y ago

Hi can you tell us a bit more about this SQL-related project? I'm rather practically interested.

I work for www.sqream.com, and our main product is SQream DB. SQream DB is a GPU SQL database for analytics. Everything was written in-house - from the SQL parser all the way down to the storage layer. We're designed to deal with sizes from a few terabytes to hundreds of terabytes, and we use the GPU to do most of the work. It is the compiler (written in Haskell) however, that decides about specific optimizations and deals with generating the query plan. Our SQL parser and typechecker is actually BSD3 licensed, if it's interesting: https://github.com/jakewheat/hssqlppp https://github.com/jakewheat/hssqlppp

shusson 9y ago
Any chance SQream DB will be open sourced or a demo version? It seems every Column based GPU DB I find is behind a paywall and targeting enterprise levels.
- arnon 9y ago
  I think it's unlikely at this point... We too are currently during some large projects for enterprises... We will be releasing a community version on AWS and Azure in the near future which should be cheap or free, other than the instance cost on the respective cloud provider.
Veedrac 9y ago
> www.sqream.com I don't like having to be that guy, but your landing page hijacks scrolling (which always causes problems; I noticed first because it broke ctrl-scroll zooming), downloads a 30MB video in a loop, forever, takes a significant amount of CPU even when the video isn't in view and the tab is not active, and despite having almost no content manages to take far more memory than any other non-email tab I have open.
- arnon 9y ago
  Our website is... Yeah... I know... :/ I've passed on your comments in any case, but know that we're in the process of rebuilding the website from scratch.

dllthomas 9y ago

If you want more generic SQL tooling in Haskell, my project was just recently open sourced (https://github.com/uber/queryparser https://github.com/uber/queryparser). It currently has support for Vertica, Hive, and Presto; adding support for more dialects isn't complicated. I'm working on cleaning it up for hackage.

arnon 9y ago
Very interesting! Do you do any sort of typechecking in this? FWIW, HsSqlPpp also supports various dialects. In SQream DB we use a custom dialect, while HsSqlPpp is mostly Postgres.
- dllthomas 9y ago
  Typechecking of the SQL statements? There was an internal branch where I was experimenting with that, but it didn't get landed before I left, and isn't in the squashed version. It was relatively simplistic - along the lines of "things tested for equality must share a type". It was also focused on domain-relevant types, not SQL type (/representation). > FWIW, HsSqlPpp also supports various dialects. In SQream DB we use a custom dialect, while HsSqlPpp is mostly Postgres. Nice. It'll be interesting to see where our implementation choices differed, and what we can learn from each other :)