6 ms·
It is, because some problems can be distributed to 1000s of GPUs and done in parallel, and some can't, because the answer to one calculation depends on the answ
by zhte415 10y ago
It is, because some problems can be distributed to 1000s of GPUs and done in parallel, and some can't, because the answer to one calculation depends on the answer to another calculation. You could combine the computing power of Azure, AWS and Google, and be pretty disappointed because of all the waiting time due to latency from one data center to another That's when the system's architecture - software and hardware - becomes important.
- dekhn 10y agoMany problems are very latency tolerant. Unless you have an algorithm which is truly latency intolerant, I argue you are best served not investing in low-latency interconnect because it costs so much. When I ran Exacycle, we distributed protein folding, protein design, drug discovery, telescope design, and other problems globally. We never had an issue with latency, because these problems all partition really well. People who claim supercomputers are "necessary" for these problems typically construct problems that are well-matched to supercomputers (for example, running molecular dynamics on huge proteins) but they tend not to have very high scientific value. In my experience, partitioning to minimize communication has always increased my total scientific throughput, while programming to supercomputers has always reduced it.