The DGX is impressive if incremental progress. MPI, OpenMP and SIMD are different APIs where CUDA is one, but their problems still need solving, eg inter-node,

The DGX is impressive if incremental progress. MPI, OpenMP and SIMD are different APIs where CUDA is one, but their problems still need solving, eg inter-node, intra-node/inter-core, and intra-core/lane. It still takes a domain expert to map a problem into those levels effectively. CUDA automates this for tree of GPUs but only do this in a handful of relatively straightforward problems.