5 ms·

For me, the second-biggest obstacle would be data size and correlated, data transfer speed. I regularly analyse GBs to TBs of data on supercomputers which have

For me, the second-biggest obstacle would be data size and correlated, data transfer speed. I regularly analyse GBs to TBs of data on supercomputers which have PBs of free space, regular consumer PCs don't have the harddrive space for that kind of thing. And even if they have, they may not have the transfer speeds to make copying my data there feasible - by the time copying has finished, I may have already run the analysis on a slower CPU

HWR_14 5y ago
Are the PBs of free space necessary? I thought training a ML model on a terabyte of data was computation expensive, but not storage intensive.
- a_bonobo 5y ago
  Sometimes! Not for the GPU part, but to make the summary data for the ML data. For example, genome sequencing data and intermediate results are easily in the TB area of space, but the resulting table of genomic variants (k-mers like n-grams in NLP) is only a few hundred GB.
  - HWR_14 5y ago
    I'm confused from your post where the petabytes of storage come in.
    - freeqaz 5y ago
      I'm guessing that there are just many data sets, and each one is in the TBs of size.
vladvasiliu 5y ago
I don't work with ML, so maybe this is stupid, but wouldn't a "regular consumer PC" have orders of magnitude less GPU power than a "supercomputer"? I'm assuming the latter has some form of GPU farm, whereas I don't know if you can fit more that 3-4 fat GPUs in the former. So if you'd be able to split the workload such that it would be able to run on consumer PCs in a reasonable time, wouldn't that also split the storage requirements the same way? Or, if instead you're OK with waiting for ages for the regular GPU to do its thing, is transfer speed that much of an issue?
- tuvan 5y ago
  Yes there are models that are trained on hundreds of GPUs but from my limited experience in scientific computing, most of the time researchers run their programs on a single node because going multi-gpu or multi-cpu requires somewhat large code changes and a single node is "good enough" for their use case or they come from a heavy science background and don't even know how to utilize multi node architecture. Their main benefit from using a cluster is 100% uptime, large storage, and large memory. I've been to multiple research institutes where there is an institute-wide HPC and researchers share it, that way no one needs any kind of high end computer and can just connect to the cluster. This service can help in that area if researchers can somehow schedule a job from their low-end laptop and get the results when the job is done.

HWR_14 5y ago

Are the PBs of free space necessary? I thought training a ML model on a terabyte of data was computation expensive, but not storage intensive.

a_bonobo 5y ago
Sometimes! Not for the GPU part, but to make the summary data for the ML data. For example, genome sequencing data and intermediate results are easily in the TB area of space, but the resulting table of genomic variants (k-mers like n-grams in NLP) is only a few hundred GB.
- HWR_14 5y ago
  I'm confused from your post where the petabytes of storage come in.
  - freeqaz 5y ago
    I'm guessing that there are just many data sets, and each one is in the TBs of size.

vladvasiliu 5y ago

I don't work with ML, so maybe this is stupid, but wouldn't a "regular consumer PC" have orders of magnitude less GPU power than a "supercomputer"? I'm assuming the latter has some form of GPU farm, whereas I don't know if you can fit more that 3-4 fat GPUs in the former. So if you'd be able to split the workload such that it would be able to run on consumer PCs in a reasonable time, wouldn't that also split the storage requirements the same way? Or, if instead you're OK with waiting for ages for the regular GPU to do its thing, is transfer speed that much of an issue?

tuvan 5y ago
Yes there are models that are trained on hundreds of GPUs but from my limited experience in scientific computing, most of the time researchers run their programs on a single node because going multi-gpu or multi-cpu requires somewhat large code changes and a single node is "good enough" for their use case or they come from a heavy science background and don't even know how to utilize multi node architecture. Their main benefit from using a cluster is 100% uptime, large storage, and large memory. I've been to multiple research institutes where there is an institute-wide HPC and researchers share it, that way no one needs any kind of high end computer and can just connect to the cluster. This service can help in that area if researchers can somehow schedule a job from their low-end laptop and get the results when the job is done.