5 ms·

Perhaps more efficient memory use is the answer instead. This article has mentioned most of the published techniques. As for the 1080 Ti, a better option might

Perhaps more efficient memory use is the answer instead. This article has mentioned most of the published techniques. As for the 1080 Ti, a better option might be the P100, which can have up to 16GB.

jacquesm 10y ago
> As for the 1080 Ti, a better option might be the P100, which can have up to 16GB. $4K or so, vs $700?
- deepnotderp 10y ago
  Most of the major labs I've seen pay six figures easily to their top researchers. Good compute is one of the biggest reasons why researchers would want to work somewhere. Skimping on compute seems foolish at that stage. Here's a half-joking, half-serious take on the matter: https://twitter.com/deliprao/status/842636496666419200 https://twitter.com/deliprao/status/842636496666419200
  - taneq 10y ago
    Yeah but is that P100 better than five or six 1080Ti cards?
    - deepnotderp 10y ago
      Because parallelization is so difficult, yes.
      - milesrout 10y ago
        If you are using a GPU, parallelisation is not difficult.
        jacquesm 10y ago
        I believe you may be missing something. Even on a GPU there are synchronization points in some algorithms and data needs to be passed from one processor to another once you spill over the boundary of what can be held in the RAM. Also, some parts of an algorithm may not be parallelizable at all. GPU parallelization is only 'not difficult' if: - all your data fits on a single GPU - your code is embarassingly parallel - for the total duration of one computation Bonus if you do can use the output of one computation as the input of the next. In all other kinds of computations the usual bottle-necks apply. https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law
        deepnotderp 10y ago
        I was talking about multi-GPU parallelization. Multi GPU SGD parallelization is not easy. Single GPU parallelization is trivial of course.
  - seanmcdirmid 10y ago
    > Most of the major labs I've seen pay six figures easily to their top researchers. Even a normal programmer easily makes six figures, I hope they are doing better than that given the demand in this space.
    - deepnotderp 10y ago
      That was a conservative estimate.
      - posterboy 10y ago
        I thought you might have meant monthly
gwern 10y ago
A better option... if someone else is paying for it. It is literally ~10x the price of a 1080ti ($700 vs a suggested $7000).
- BrailleHunting 10y ago
  The P100 has much better double-precision performance, but it's single-precision perf doesn't justify the cost in most instances. The DGX1 (w 8 P100's) is another example of an overpriced, turnkey money extraction "solution." Also worth noting it's hard to get a Ti non-founder edition, which has a hefty premium on it for the privilege of being first. The regular 1080 non-FE cards are probably the best bang/buck right now. Folking doing ML/AI/DL, or any HPC really, gotta profile their apps to find bottlenecks and root out resource waste: single prec math? ram? algorithm complexity? storage iops? network latency? etc. because throwing money at a problem is no substitute for having a clue how to use money, time and electricity wisely.
  - llukas 10y ago
    Compared to what it can replace DGX-1 is not overpriced in any way. Feel free to invest your time and money into building and debugging issues on your own system. Companies which pay big $$$ to their data scientists have different incentives than regular users - you do not want to waste time of your expensive staff on chasing hardware/software issues.
    - Roritharr 10y ago
      129000$ for the DGX-1, for anyone interested. Saves you debugging your way to a workable 8*1080ti, when you look at it from this angle. Not sure if being able to swap out the cards is what makes building them yourself worth it in addition to the cost saving.
      - modeless 10y ago
        P100 has double rate FP16 which makes it twice as fast, so make that 16x 1080Ti. Also those won't fit into one machine, so now you're building a cluster, and what are you going to use for the interconnect? You won't get linear scaling, so make that more like 24x 1080Tis plus extra development effort for scaling on a cluster, if your problem even scales that way. Now account for the power usage...

Perhaps more efficient memory use is the answer instead. This article has mentioned most of the published techniques. As for the 1080 Ti, a better option might be the P100, which can have up to 16GB.

jacquesm 10y ago

> As for the 1080 Ti, a better option might be the P100, which can have up to 16GB. $4K or so, vs $700?

deepnotderp 10y ago
Most of the major labs I've seen pay six figures easily to their top researchers. Good compute is one of the biggest reasons why researchers would want to work somewhere. Skimping on compute seems foolish at that stage. Here's a half-joking, half-serious take on the matter: https://twitter.com/deliprao/status/842636496666419200 https://twitter.com/deliprao/status/842636496666419200
- taneq 10y ago
  Yeah but is that P100 better than five or six 1080Ti cards?
  - deepnotderp 10y ago
    Because parallelization is so difficult, yes.
    - milesrout 10y ago
      If you are using a GPU, parallelisation is not difficult.
      - jacquesm 10y ago
        I believe you may be missing something. Even on a GPU there are synchronization points in some algorithms and data needs to be passed from one processor to another once you spill over the boundary of what can be held in the RAM. Also, some parts of an algorithm may not be parallelizable at all. GPU parallelization is only 'not difficult' if: - all your data fits on a single GPU - your code is embarassingly parallel - for the total duration of one computation Bonus if you do can use the output of one computation as the input of the next. In all other kinds of computations the usual bottle-necks apply. https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law
      - deepnotderp 10y ago
        I was talking about multi-GPU parallelization. Multi GPU SGD parallelization is not easy. Single GPU parallelization is trivial of course.
- seanmcdirmid 10y ago
  > Most of the major labs I've seen pay six figures easily to their top researchers. Even a normal programmer easily makes six figures, I hope they are doing better than that given the demand in this space.
  - deepnotderp 10y ago
    That was a conservative estimate.
    - posterboy 10y ago
      I thought you might have meant monthly

gwern 10y ago

A better option... if someone else is paying for it. It is literally ~10x the price of a 1080ti ($700 vs a suggested $7000).

BrailleHunting 10y ago
The P100 has much better double-precision performance, but it's single-precision perf doesn't justify the cost in most instances. The DGX1 (w 8 P100's) is another example of an overpriced, turnkey money extraction "solution." Also worth noting it's hard to get a Ti non-founder edition, which has a hefty premium on it for the privilege of being first. The regular 1080 non-FE cards are probably the best bang/buck right now. Folking doing ML/AI/DL, or any HPC really, gotta profile their apps to find bottlenecks and root out resource waste: single prec math? ram? algorithm complexity? storage iops? network latency? etc. because throwing money at a problem is no substitute for having a clue how to use money, time and electricity wisely.
- llukas 10y ago
  Compared to what it can replace DGX-1 is not overpriced in any way. Feel free to invest your time and money into building and debugging issues on your own system. Companies which pay big $$$ to their data scientists have different incentives than regular users - you do not want to waste time of your expensive staff on chasing hardware/software issues.
  - Roritharr 10y ago
    129000$ for the DGX-1, for anyone interested. Saves you debugging your way to a workable 8*1080ti, when you look at it from this angle. Not sure if being able to swap out the cards is what makes building them yourself worth it in addition to the cost saving.
    - modeless 10y ago
      P100 has double rate FP16 which makes it twice as fast, so make that 16x 1080Ti. Also those won't fit into one machine, so now you're building a cluster, and what are you going to use for the interconnect? You won't get linear scaling, so make that more like 24x 1080Tis plus extra development effort for scaling on a cluster, if your problem even scales that way. Now account for the power usage...