3 ms·
I'm reminded of the following episode of Pinky and the Brain: Brain plans to use a "growing ray" (originally a "shrinking ray") to grow Pinky into super-si
by YeGoblynQueenne 6y ago
I'm reminded of the following episode of Pinky and the Brain:
Brain plans to use a "growing ray" (originally a "shrinking ray") to grow
Pinky into super-size while dressed up as Gollyzilla, while Brain would turn
himself gigantic and stop him, using the name Brainodo, in exchange for
world domination. However, the real Gollyzilla emerges from the ocean and
starts to rampage through the city, making Brain think that the dinosaur is
Pinky. The episode ends with the ray going out of control and making
everything on Earth grow, including the Earth itself, to the point that
Pinky, the Brain, and even Gollyzilla are mouse-sized by comparison again.
https://en.wikipedia.org/wiki/List_of_Pinky_and_the_Brain_episodes https://en.wikipedia.org/wiki/List_of_Pinky_and_the_Brain_ep...
Being able to train deep RL models on commodity hardware is only an advantage
if there isn't anyone that can train on more powerful hardware (or if somehow
training on more powerful hardware fails to improve performance with respect
to your model). Otherwise, you're still just a little mouse and they have all
the compute.
- DrNuke 6y agoYou still need a problem, domain knowledge, a good question and relevant data?
- YeGoblynQueenne 6y agoI don't think FANGs lack any of that, either.
- DrNuke 6y agoNah, they pursue the problems that are relevant to them, which are a very small subset of the problems being tackled globally... so you can still win your particular race at your snail pace, if they are not running?
- YeGoblynQueenne 6y agoThat's not exactly how it works. For example, Go, the board game, is not particularly relevant to Google, owner of DeepMind. Yet they spent considerable resources in creating and training a succession of systems that excel at it. Besides some applications, big technology companies see AI partly as a future investment, partly as very good publicity. My guess in any case. Then again there are the general problems that are relevant to everyone, like natural language understanding, question answering, image recognition and so on, high-level tasks for which solutions have broad applicability. Rather tautologically, such tasks are always relevant to anyone who can perform them well. In any case, if this was not the case there wouldn't be any motivation for academic teams to find ways to train with smaller computing resources, as the article reports.
- currymj 6y agowell, you don't actually have to have your models fight their models. there's lots of things to try that aren't the SOTA rat race. like, if you want to show relative improvement of some new variation of an RL algorithm, this could be a good way to do it. or if you have a new environment that you want to solve for yourself. right now if you try to train anything in a moderately interesting environment on a PC, it takes just a little too long to get results -- makes the whole research process pretty painful.
- YeGoblynQueenne 6y agoI'm afraid that computing resources are not and have never been the limiting factor for innovative work in machine learning in general and in deep learning in particular. I have quoted the following interview with Geoff Hinton a number of times on HN - apologies if this is becoming repetitious: GH: One big challenge the community faces is that if you want to get a paper published in machine learning now it's got to have a table in it, with all these different data sets across the top, and all these different methods along the side, and your method has to look like the best one. If it doesn’t look like that, it’s hard to get published. I don't think that's encouraging people to think about radically new ideas. Now if you send in a paper that has a radically new idea, there's no chance in hell it will get accepted, because it's going to get some junior reviewer who doesn't understand it. Or it’s going to get a senior reviewer who's trying to review too many papers and doesn't understand it first time round and assumes it must be nonsense. Anything that makes the brain hurt is not going to get accepted. And I think that's really bad. What we should be going for, particularly in the basic science conferences, is radically new ideas. Because we know a radically new idea in the long run is going to be much more influential than a tiny improvement. That's I think the main downside of the fact that we've got this inversion now, where you've got a few senior guys and a gazillion young guys. https://www.wired.com/story/googles-ai-guru-computers-think-more-like-brains/ https://www.wired.com/story/googles-ai-guru-computers-think-... In other words, yes, unfortunately, everything is the SOTA rat race. At least anything that is meant for publication, which is the majority of research output.
- currymj 6y ago