4 ms·
Actually, it is a novel approach so I am not aware of anything similar. Initially, I implemented it for data wrangling (various potentially complex transformati
by asavinov 10y ago
Actually, it is a novel approach so I am not aware of anything similar. Initially, I implemented it for data wrangling (various potentially complex transformations: https://en.wikipedia.org/wiki/Data_wrangling https://en.wikipedia.org/wiki/Data_wrangling) and it can be found here: https://bitbucket.org/conceptoriented/dce-java https://bitbucket.org/conceptoriented/dce-java. The main point is that instead of defining a flow (graph) of tables we define a flow (graph) of columns. It can be used where Apache Spark is used.
StreamCommandr essentially relies on the same principles and changes only external API and some internal processing. The idea is that any data table is like a stream of records so that we add new records and delete outdated records. Simultaniously, we evaluate other columns and tables by performaing potentially complex computations which are difficult to do in a record flow.