4 ms·
I would say that the goals align much more than the current approaches, which are somewhat orthogonal. My focus in developing Elemental has been on expanding fr
by math_and_stuff 11y ago
I would say that the goals align much more than the current approaches, which are somewhat orthogonal. My focus in developing Elemental has been on expanding from the scope of libraries like ScaLAPACK (which focused on distributed dense linear algebra) into more general algorithms (e.g., Interior Point Methods and lattice reduction) and modern programming practices (using templates to support arbitrary-precision versions of the above).
Arguably the biggest weakness of Elemental is its lack of focus on dynamic scheduling at an intranodal level, though this can to a large degree be pushed into a local BLAS layer. One of the big items on my wish list would be proper integration of intranodal dynamic scheduling with the current high-performance internodal static scheduling.
The current TensorFlow releases seem to be primarily focused on intranodal dynamic scheduling; there has been a substantial amount of work in this area from the PLASMA [1] and (less recently) SuperMatrix [2] projects.
[1] http://icl.cs.utk.edu/plasma/ http://icl.cs.utk.edu/plasma/
[2] http://www.cs.utexas.edu/users/flame/pubs/SuperMatrixTR.pdf http://www.cs.utexas.edu/users/flame/pubs/SuperMatrixTR.pdf
- Gtifn 11y agoHow about dask's distrubted array? Do you know how the architecture and goals compare to Elemental?
- deleted 11y ago[deleted]
- math_and_stuff 11y agoDistributed [1] is very new and seems to have similar core architectural goals as TensorFlow. But perhaps I'm being too politically correct: both make use of very course-grain parallelism relative to a typical distributed-memory linear algebra library (e.g., the current communication mechanisms of both are likely to be too course-grain to efficiently support distributed dense matrix inversion or eigensolvers; not that this is likely to be a design goal of either). [1] http://matthewrocklin.com/blog/work/2015/06/23/Distributed/ http://matthewrocklin.com/blog/work/2015/06/23/Distributed/
- Gtifn 11y agoI think distributed's scheduler has seen much iteration since then and is now based on tornado. I wonder if that changes your assessment. Also what do you think of Julia's native distributed capabilities and this library here: https://github.com/shashi/ComputeFramework.jl https://github.com/shashi/ComputeFramework.jl