3 ms·

Speaking as an AI researcher (although I don't spend all my time in the current trendy bits), the problem is it seems at the moment what you need to do successf

Speaking as an AI researcher (although I don't spend all my time in the current trendy bits), the problem is it seems at the moment what you need to do successful machine learning is: * Lots of data * Lots of CPU/GPU power That's two places where Universities just can't compete with the money which companies can throw at this problem. Also, all AI companies have no interest in working with academia, just taking the staff/students, so there is no useful 2-way communication. EDIT: I just wanted to expand one small point. The data and CPU/GPU power seem to let you research and solve fundamentally different problems. Consider a classic AI technique, like SAT. If you have lots of computers, you can solve bigger SAT problems, but there didn't seem to be an interesting research which required massive clusters of machines, or massive datasets. The big research improvements were all possible on a single reasonable desktop machine, and a selection of commonly available benchmarks.

_delirium 9y ago
The CPU/GPU power isn't really out of reach of academic budgets anymore. There are some kinds of problems that are, but AlphaGoZero ran on a box with 4 GPUs, for example, and there is a lot of cutting-edge research with similar computing budgets. It's more DIY though. You have to spec out and buy your hardware, maintain it, etc. (or a grad student does that in their "spare time"), while if you work in AI at Google all the hardware and networking and such is in most cases left to professionals.
- schaefer 9y ago
  the assertion that alpha go zero ran on a box with four GPUs is not accurate. alpha go zero ran on a box with four TPUs[1]. Those of us on the outside (of google's ai research program) don't have access to that hardware. [1] TPU: tensor processing unit. [1] https://deepmind.com/blog/alphago-zero-learning-scratch/ https://deepmind.com/blog/alphago-zero-learning-scratch/
- netheril96 9y ago
  AlphaGoZero runs on 4 TPUs not GPUs. And that is only for inference, not for training. For training it uses 64 GPUs. More importantly, AlphaGoZero is the final evolution. To reach this stage one needs to experiment a lot, and most of the experiments are failures. To do any meaningful research requires a budget of several times or dozens of times the amount of computing power, and this is only for one project. The computation resources at universities are definitely not sufficient.

_delirium 9y ago

The CPU/GPU power isn't really out of reach of academic budgets anymore. There are some kinds of problems that are, but AlphaGoZero ran on a box with 4 GPUs, for example, and there is a lot of cutting-edge research with similar computing budgets. It's more DIY though. You have to spec out and buy your hardware, maintain it, etc. (or a grad student does that in their "spare time"), while if you work in AI at Google all the hardware and networking and such is in most cases left to professionals.

schaefer 9y ago
the assertion that alpha go zero ran on a box with four GPUs is not accurate. alpha go zero ran on a box with four TPUs[1]. Those of us on the outside (of google's ai research program) don't have access to that hardware. [1] TPU: tensor processing unit. [1] https://deepmind.com/blog/alphago-zero-learning-scratch/ https://deepmind.com/blog/alphago-zero-learning-scratch/
netheril96 9y ago
AlphaGoZero runs on 4 TPUs not GPUs. And that is only for inference, not for training. For training it uses 64 GPUs. More importantly, AlphaGoZero is the final evolution. To reach this stage one needs to experiment a lot, and most of the experiments are failures. To do any meaningful research requires a budget of several times or dozens of times the amount of computing power, and this is only for one project. The computation resources at universities are definitely not sufficient.