4 ms·

I saw a project in similar space- Malloy- go by recently, in a post, What Happened to the Semantic Layer, which I thought nicely setup the space & problem. htt

I saw a project in similar space- Malloy- go by recently, in a post, What Happened to the Semantic Layer, which I thought nicely setup the space & problem. https://carlineng.com/?postid=semantic-layer#blog https://carlineng.com/?postid=semantic-layer#blog https://news.ycombinator.com/item?id=35715410 https://news.ycombinator.com/item?id=35715410 They indeed mentioned Cube.js. Bunch of Malloy links over time: https://hn.algolia.com/?dateRange=all&query=Malloy&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&query=Malloy&sort=byDa... And cube.js, https://hn.algolia.com/?dateRange=all&query=cube.js&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&query=cube.js&sort=byD... It's all very interesting to me because I get the sense these folks feel like we haven't figured out to to make meaning from the data we have. As a developer, that rings true; the developee is the agent responsible for understanding what we have in sql now & how to eek out meaning, how to structure more info in. The information architectures are so occluded & concealed, so stoggily low level. I love these attempts to try to help us think more about our data.

adolph 4y ago
My operating theory is that in app centric development the data store is a component of the application. As a result, much of the metadata that changes data into information (data with meaning) is stored in app code and config. Upon ETL to reporting, warehouse, lake, or the like, the app’s semantic layer is lost.
- rektide 4y ago
  It's a crying shame how little of the data-core makes it's way out of applications. For both the app's owning entity & especially for the users. I paid respects[1] recently yet again to Window Manager Improved Improved (wmii), which kept it's state in a 9p filesystem any user could easily browse & modify. The state is still a component of the app/window manager, but it's at least malleable to all. [1] https://news.ycombinator.com/item?id=35768686 https://news.ycombinator.com/item?id=35768686
igorlukanin 4y ago
So good you brought Malloy here. I like it quite a bit because the folks really try to innovate (heck, they even have their own data querying syntax to replace SQL). But what I like even more — being part of the Cube team — that the "cons" of existing solutions that Carlin mentions in his blog are actually already solved by Cube. With Cube, Data exploration, ideation on the data model, querying, and bringing the insights all the way down to BI tools or data apps takes minutes rather than hours or days. Done, case closed :-)
- carlineng 4y ago
  My understanding of Cube is that iterating on the data model requires the user to (1) write SQL to develop a metric (2) edit YAML or JS config to incorporate the new metric (3) issue API request to Cube server and (4) compare results to raw SQL. Am I mistaken? Does Cube offer a smoother way to do this exploration/iteration?
  - igorlukanin 4y ago
    Hey, Carlin! Nice to see you here in comments! (Waving "hi" to the Malloy team.) Usually, the experience would look like this: one directly develops the data model in YAML (with only bits of SQL, if needed) and instantly explores metrics. No need to start with SQL in a separate tool/place (1), no need to use the API to check metrics (2) (for that, we have Playground, an interactive UI tool), and, thus, no need to compare results to raw SQL (4). You iterate but changing the data model and seeing the metrics in an instant, quite similar to how you work with Malloy, if I may.

adolph 4y ago

My operating theory is that in app centric development the data store is a component of the application. As a result, much of the metadata that changes data into information (data with meaning) is stored in app code and config. Upon ETL to reporting, warehouse, lake, or the like, the app’s semantic layer is lost.

rektide 4y ago
It's a crying shame how little of the data-core makes it's way out of applications. For both the app's owning entity & especially for the users. I paid respects[1] recently yet again to Window Manager Improved Improved (wmii), which kept it's state in a 9p filesystem any user could easily browse & modify. The state is still a component of the app/window manager, but it's at least malleable to all. [1] https://news.ycombinator.com/item?id=35768686 https://news.ycombinator.com/item?id=35768686

igorlukanin 4y ago

So good you brought Malloy here. I like it quite a bit because the folks really try to innovate (heck, they even have their own data querying syntax to replace SQL). But what I like even more — being part of the Cube team — that the "cons" of existing solutions that Carlin mentions in his blog are actually already solved by Cube. With Cube, Data exploration, ideation on the data model, querying, and bringing the insights all the way down to BI tools or data apps takes minutes rather than hours or days. Done, case closed :-)

carlineng 4y ago
My understanding of Cube is that iterating on the data model requires the user to (1) write SQL to develop a metric (2) edit YAML or JS config to incorporate the new metric (3) issue API request to Cube server and (4) compare results to raw SQL. Am I mistaken? Does Cube offer a smoother way to do this exploration/iteration?
- igorlukanin 4y ago
  Hey, Carlin! Nice to see you here in comments! (Waving "hi" to the Malloy team.) Usually, the experience would look like this: one directly develops the data model in YAML (with only bits of SQL, if needed) and instantly explores metrics. No need to start with SQL in a separate tool/place (1), no need to use the API to check metrics (2) (for that, we have Playground, an interactive UI tool), and, thus, no need to compare results to raw SQL (4). You iterate but changing the data model and seeing the metrics in an instant, quite similar to how you work with Malloy, if I may.