4 ms·

Their pykx integration is going a long way to fix some of the gaps in: - charting - machine learning/statsmodels - html processing/webscrapes Because for ex

Their pykx integration is going a long way to fix some of the gaps in: - charting - machine learning/statsmodels - html processing/webscrapes Because for example you can just open a Jupyter Notebook and do: import pykx as kx df = kx.q(“select from foo where bar”) plt.plot(df[“x”], df[“y”]) It’s truly an incredibly seamless and powerful integration. You get the best of both worlds and it may be the saving feature of the product in the next 10 years

nivertech 2y ago
I think this will only work with regular qSQL on a specific database node, i.e. RDB, IDB, HDB[1]. It will be much harder for a mortal Python developer to use Functional qSQL[2] which will join/merge/aggregate data from all these nodes. The join/merge/aggregation is usually application-specific and done on some kind of gateway node(s). Querying each of them is slightly different, with different keys and secondary indices, and requires using a parse tree (AST) of a query. --- [1] RDB - RAM DB (recent in-memory data), IDB (Intraday DB - recent data which doesn't fit into RAM), HDB - Historical DB (usually partitioned by date or other time-based or integral column). [2] https://code.kx.com/q/basics/funsql/ https://code.kx.com/q/basics/funsql/
- Jorge1o1 2y ago
  That’s accurate enough. I think the workflow was more built for a q dev occasionally dipping into python rather than the other way around. I think you touch on something really interesting which is the kink in the kdb+ learning curve when you go from really simple functions,tables, etc. to actually building a performant kdb architecture.
- qkdb1 2y ago
  Will be interesting to see what comes of some of the things that are being put on their roadmap https://code.kx.com/pykx/2.5/roadmap.html#upcoming-changes https://code.kx.com/pykx/2.5/roadmap.html#upcoming-changes seems to be moving in a direction of an API similar to Polars

nivertech 2y ago

I think this will only work with regular qSQL on a specific database node, i.e. RDB, IDB, HDB[1]. It will be much harder for a mortal Python developer to use Functional qSQL[2] which will join/merge/aggregate data from all these nodes. The join/merge/aggregation is usually application-specific and done on some kind of gateway node(s). Querying each of them is slightly different, with different keys and secondary indices, and requires using a parse tree (AST) of a query. --- [1] RDB - RAM DB (recent in-memory data), IDB (Intraday DB - recent data which doesn't fit into RAM), HDB - Historical DB (usually partitioned by date or other time-based or integral column). [2] https://code.kx.com/q/basics/funsql/ https://code.kx.com/q/basics/funsql/

Jorge1o1 2y ago
That’s accurate enough. I think the workflow was more built for a q dev occasionally dipping into python rather than the other way around. I think you touch on something really interesting which is the kink in the kdb+ learning curve when you go from really simple functions,tables, etc. to actually building a performant kdb architecture.
qkdb1 2y ago
Will be interesting to see what comes of some of the things that are being put on their roadmap https://code.kx.com/pykx/2.5/roadmap.html#upcoming-changes https://code.kx.com/pykx/2.5/roadmap.html#upcoming-changes seems to be moving in a direction of an API similar to Polars