3 ms·
I'm not familiar with Ecto, dplyr, or DBT, but I would love an ML-like language to replace SQL. I'm imagining being able to pass a table (or any table-oriented
by alphanumeric0 4y ago
I'm not familiar with Ecto, dplyr, or DBT, but I would love an ML-like language to replace SQL. I'm imagining being able to pass a table (or any table-oriented data, like a sub-query) to functions that would type-check columns and would return table-oriented or scalar data. I'm not sure if this is actually possible in practice, but one can dream.
For instance, a "top 10" function that could be re-used on any table (apologies for my pseudo types and code):
selectTop10 : Column -> Table -> Table
selectTop10 orderByColumn table =
SQL.selectAllFrom table
|> orderDescBy orderByColumn
|> limit 10
limit : Int -> Table -> Table
limit n rows =
SQL.limitBy n rows
orderDescBy : Column -> Table -> Table
orderDescBy orderByColumn rows =
SQL.orderBy [orderByColumn] SQL.Ordering.Desc rows
- chrisjc 4y agoNot 100% sure about what you're suggesting, but wouldn't it be easier to pass your functions to your table/sub-query? And that's exactly what you're able to do in most of the modern data warehouse services such as Snowflake. Inferences can be contained within internal/external user defined functions. This is very reminiscent of made the big-data/map-reduce movement so notable, sending your query to the data instead of moving your data to the query. Sending your model to the data, instead of sending the data to the model.
- alphanumeric0 4y agoI think I see what you're saying, and in my idea I'm suggesting the same - sending the function(s) to the data.
- rawgabbit 4y agoI don't quite follow what you are saying. In some SQL engines you can use row_number() function and derived tables (or CTE) to get top 10. SELECT Id,col1,col2 FROM ( SELECT Id ,col1 ,col2 ,row_number () over (partition by columnkey1, columnkey2 order by anycolumnwilldohere desc) as _row FROM _table ) as anytablealiaswilldohere WHERE _row <=10
- alphanumeric0 4y agoMy idea is you wouldn't have to cut and paste implementations like this. You would be able to just call the function with a column name and table name.