3 ms·
Wow, defensive much? Afraid of a little competition maybe? Anyway, DSSTNE itself is not in any way locked to any cloud service whatsoever. It is an Apache-li
by scottlegrand2 10y ago
Wow, defensive much? Afraid of a little competition maybe?
Anyway, DSSTNE itself is not in any way locked to any cloud service whatsoever. It is an Apache-licensed library with dependencies on a C++11 compiler, a 7.0 or later CUDA Toolkit and Kepler or better GPU, a C++11-friendly MPI library, netcdf, and libjsoncpp. That's it. Please stop insinuating otherwise.
The article here shows how one could use DSSTNE with Spark for recommendations at scale. Speaking of which, how's your sparse data and model-parallel training support? Because as the author of DSSTNE, the poor support for these features in other frameworks was what forced to us to "roll our own" code in the first place. Everyone else was optimizing for ImageNet-winning CNNs (including NVIDIA). And there's nothing wrong with that, $25M+ companies have been built from that, but it just wasn't the use case here.
Finally, cuSparse is cuSlow for datasets at Amazon. DSSTNE's hand-coded sparse kernels stomp on cuSparse in the same way that Neon's convolution kernels stomp on cuDNN. And there's nothing wrong with that either. CuSparse is a great choice for other sparse data problems, just not Amazon's (and I suspect many other companies) recommendations problems.
- deleted 10y ago[deleted]