5 ms·

It's far from a nonsense purchase for Deep Learning professionals. At https://lambdalabs.com https://lambdalabs.com nearly every one of our customers is calling

It's far from a nonsense purchase for Deep Learning professionals. At https://lambdalabs.com https://lambdalabs.com nearly every one of our customers is calling us up interested in the 2080Ti for Deep Learning. The reality is that very few people training neural networks need more than 11GB.

bitL 8y ago
Then most likely they would not be successful in their businesses; training older models or newer ones with a batch of 1 is not a recipe for success these days when you more like need 100s of GPUs running slight modifications to models in parallel to find the one that works.
- sgt101 8y ago
  We used to call this model fishing and regard models that came from places that did it with fear and suspicion (due to sudden failure and poor performance in production) What has changed that people think this is a wise approach?
  - bitL 8y ago
    AutoML and automated architecture/hyperparameter search used by FAANG, producing things like AmoebaNet-D, that beat all previous state-of-art results.
  - matt4077 8y ago
    They don't suddenly fail any more, and performance in production is fine... This is a rather empirical science right now. If forced to speculate, I'd say far larger datasets are part of the answer. If you can afford to hold back 30%+ of your data for (multi-tranche) verification the difference between testing and production becomes a problem for the philosophers.
    - sgt101 8y ago
      Hmm this isn't my experience. My experience is that highly complex models tend to reflect accurately the ground truth in the current dataset, but not the domain theory that generated it, so when things change (like, prevailing weather conditions, political alignments, interest rates, data usage) which move the distribution (but not the domain theory) they do fail. The question is : are you trading poorer compression for more fidelity or are you learning the system that generates the data?
- sabalaba 8y ago
  Most people I know who publish in NIPS/ICLR train their models using between 1-4 GPUs under their desk or 8 GPUs in a server. I would argue that these folks make competitive models and "are successful in their business" with only 1-8 GPUs. While having access to hundreds of GPUs helps you explore model space more quickly, it's not a hard pre-requisite to success in Deep Learning.
  - eanzenberg 8y ago
    Yup, state of the art doesn't require GPU farms, just a couple in a consumer desktop will do.
    - bitL 8y ago
      Lookup AmoebaNet-D if you think you don't need a GPU farm these days. Those were times when 1-2 GPUs were enough...
      - jefft255 8y ago
        You know, there is more to deep learning research than meta-learning/architecture exploration. Sure you can explore the hyper-parameter space faster with 500 GPUs and get yet again a 0.05% better test score on ImageNet (or more I don't actually know), but there are other ways to do something meaningful in DL without using such compute power.
        bitL 8y ago
        That's a fair point and I agree. It's just sometimes difficult to beat automated exploration; as a standard company you probably don't have access to top-end researchers/practitioners, just average ones, and those might get a significant boost by trading smartness for brute force and run many models in parallel in evolutionary fashion. When you e.g. look at how Google's internal system constructs loss functions and how many combinations do they try, one has to have an unexpected idea to beat their results, and that idea can be usually quickly incorporated into their platform, raising the bar for individual researchers. At Facebook they basically press a few buttons and select a few checkboxes, then wait until best model is selected, leading to frustration among researchers.
jlg23 8y ago
Without anything beyond claims what reality™ is, your response unfortunately just reads like an ad for your company.
- sabalaba 8y ago
  "Reality", in our case, is based on conversations with a few dozen customers who use our products as well as the actively publishing researchers we know personally. The majority of sampled customers are training convnets of some fashion, the most common being Yolo/SSD and ResNet.
  - nabla9 8y ago
    Do these people who you talk with know the specs? Do they know if tensor cores in the these processors are just good for interference or are they similar to the more pricey models (floating point precision)?
  - lawrenceyan 8y ago
    Most people I know are just using Google Cloud. Directly integrated with Tensorflow, and way more scalable. I can run 10 GPUs on my model training runs and finish in an hour now when they used to take at least 2 or 3 days, and it took absolutely no work on my end. It’s been absolutely wonderful for my productivity. The price doesn’t matter either because compared to how much the people are being paid at these sorts of companies, it really doesn’t matter. The boost in efficiency is so much more important.
    - ericd 8y ago
      Does Google Cloud run anything other than Tensorflow well? Specifically, I'm wondering about PyTorch.
      - lawrenceyan 8y ago
        GCP is pretty tightly integrated/optimizes for Tensorflow. That’s why scaling GPU amounts wasn’t a hassle for example since I was already using the TFEstimator framework. I’m pretty sure for the next Tensorflow 3.0 update, they’re rethinking towards a Pytorch style with more dynamic computational graphs.

bitL 8y ago

Then most likely they would not be successful in their businesses; training older models or newer ones with a batch of 1 is not a recipe for success these days when you more like need 100s of GPUs running slight modifications to models in parallel to find the one that works.

sgt101 8y ago
We used to call this model fishing and regard models that came from places that did it with fear and suspicion (due to sudden failure and poor performance in production) What has changed that people think this is a wise approach?
- bitL 8y ago
  AutoML and automated architecture/hyperparameter search used by FAANG, producing things like AmoebaNet-D, that beat all previous state-of-art results.
- matt4077 8y ago
  They don't suddenly fail any more, and performance in production is fine... This is a rather empirical science right now. If forced to speculate, I'd say far larger datasets are part of the answer. If you can afford to hold back 30%+ of your data for (multi-tranche) verification the difference between testing and production becomes a problem for the philosophers.
  - sgt101 8y ago
    Hmm this isn't my experience. My experience is that highly complex models tend to reflect accurately the ground truth in the current dataset, but not the domain theory that generated it, so when things change (like, prevailing weather conditions, political alignments, interest rates, data usage) which move the distribution (but not the domain theory) they do fail. The question is : are you trading poorer compression for more fidelity or are you learning the system that generates the data?
sabalaba 8y ago
Most people I know who publish in NIPS/ICLR train their models using between 1-4 GPUs under their desk or 8 GPUs in a server. I would argue that these folks make competitive models and "are successful in their business" with only 1-8 GPUs. While having access to hundreds of GPUs helps you explore model space more quickly, it's not a hard pre-requisite to success in Deep Learning.
- eanzenberg 8y ago
  Yup, state of the art doesn't require GPU farms, just a couple in a consumer desktop will do.
  - bitL 8y ago
    Lookup AmoebaNet-D if you think you don't need a GPU farm these days. Those were times when 1-2 GPUs were enough...
    - jefft255 8y ago
      You know, there is more to deep learning research than meta-learning/architecture exploration. Sure you can explore the hyper-parameter space faster with 500 GPUs and get yet again a 0.05% better test score on ImageNet (or more I don't actually know), but there are other ways to do something meaningful in DL without using such compute power.
      - bitL 8y ago
        That's a fair point and I agree. It's just sometimes difficult to beat automated exploration; as a standard company you probably don't have access to top-end researchers/practitioners, just average ones, and those might get a significant boost by trading smartness for brute force and run many models in parallel in evolutionary fashion. When you e.g. look at how Google's internal system constructs loss functions and how many combinations do they try, one has to have an unexpected idea to beat their results, and that idea can be usually quickly incorporated into their platform, raising the bar for individual researchers. At Facebook they basically press a few buttons and select a few checkboxes, then wait until best model is selected, leading to frustration among researchers.

jlg23 8y ago

Without anything beyond claims what reality™ is, your response unfortunately just reads like an ad for your company.

sabalaba 8y ago
"Reality", in our case, is based on conversations with a few dozen customers who use our products as well as the actively publishing researchers we know personally. The majority of sampled customers are training convnets of some fashion, the most common being Yolo/SSD and ResNet.
- nabla9 8y ago
  Do these people who you talk with know the specs? Do they know if tensor cores in the these processors are just good for interference or are they similar to the more pricey models (floating point precision)?
- lawrenceyan 8y ago
  Most people I know are just using Google Cloud. Directly integrated with Tensorflow, and way more scalable. I can run 10 GPUs on my model training runs and finish in an hour now when they used to take at least 2 or 3 days, and it took absolutely no work on my end. It’s been absolutely wonderful for my productivity. The price doesn’t matter either because compared to how much the people are being paid at these sorts of companies, it really doesn’t matter. The boost in efficiency is so much more important.
  - ericd 8y ago
    Does Google Cloud run anything other than Tensorflow well? Specifically, I'm wondering about PyTorch.
    - lawrenceyan 8y ago
      GCP is pretty tightly integrated/optimizes for Tensorflow. That’s why scaling GPU amounts wasn’t a hassle for example since I was already using the TFEstimator framework. I’m pretty sure for the next Tensorflow 3.0 update, they’re rethinking towards a Pytorch style with more dynamic computational graphs.