5 ms·
To me the most interesting part of the article is the cometary on where basic research is going to happen in the future. The fear is that if it only happens in
by Game_Ender 5y ago
To me the most interesting part of the article is the cometary on where basic research is going to happen in the future. The fear is that if it only happens in large companies, then the unbiased pool of experts society relies on will be smaller and less informed. Along with the issue of nobody being around for the slog of defining a field, setting up databases, competitions and standards. These are what allow well funded corporate labs to apply their skills and compute and blow a problem out of the water. The problem is, would they do the work to define an unknown problem in the first place?
- mrfusion 5y ago> unbiased pool of experts Sounds like an oxymoron these days.
- suetoniusp 5y agoAlways has been
- jonas21 5y ago> DeepMind claimed that they used “128 TPUv3 cores or roughly equivalent to ~100-200 GPUs”. Although this amount of compute seems beyond the wildest dreams of most academic researchers... So, we're talking like what? Maybe $100K to $300K of hardware? Wet biology labs often have multiple pieces of $100K+ equipment at their disposal. Why shouldn't computational labs too?
- Synaesthesia 5y agoYeah but it's also putting it together and properly utlizing it, which takes specialist knowledge.
- tiborsaas 5y agoThis was never really a bottleneck for science. One needs to realize first that something can be done, then it will be done. The cost of computing will also go down in the future and for government funds this sounds like a drop in the sea when they are building multi billion dollar particle accelerators.
- marvin 5y agoImagine building a multi billion dollar computing cluster solely for research. I guess the main reason it hasn't been done is that the deprecation is still huge due to chip advancements.
- ma2rten 5y agoThere is something like this: https://bigscience.huggingface.co/en/#!index.md https://bigscience.huggingface.co/en/#!index.md
- extropy 5y agoYou can rent a 32 TPUv3 pod from Google Cloud at $32 per hour. So 128 pod would be roughly 150 per hour. $1K gives you 8 hours of training time. https://cloud.google.com/tpu/pricing#pod-pricing https://cloud.google.com/tpu/pricing#pod-pricing
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- zmmmmm 5y agoIs that not just for the final training run though? All the experimentation and fine tuning probably meant thousands of trials which may have been significantly bigger scale before they got the model optimised ...