2 ms·
> it could be applied to tasks where Spark is normally used How does it relate to Spark? Is it able to read from HDFS, translate to Spark map-reduce jobs or in
by techno_modus 9y ago
> it could be applied to tasks where Spark is normally used
How does it relate to Spark? Is it able to read from HDFS, translate to Spark map-reduce jobs or integrate somehow with Spark jobs?
- asavinov 9y agoIt could be a technological (column-oriented) alternative to what map-reduce in general and Spark with RDDs in particular are doing. It might be easier to define a data processing workflow as a number of column definitions rather than as a graph of collection transformations. And such a column transformation workflow could be also more efficient at run time. Here is my implementation of this column-oriented approach as a Java library for server-side transformations: https://bitbucket.org/conceptoriented/dc-core https://bitbucket.org/conceptoriented/dc-core