10 ms·
Powerful AI Can Now Be Trained on a Single Computer
- RcouF1uZ4gsC 6y ago> His group took advantage of working on a single machine by simply cramming all the data to shared memory where all processes can access it instantaneously. If you can get all your data into RAM on a single computer, you can have a huge speedup, even over a cluster that has in aggregate more resources. Frank McSherry has some more about this, though not directly about ML training. http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html http://www.frankmcsherry.org/graph/scalability/cost/2015/01/...
- ladberg 6y agoSo this basically boils down to keeping your training data in memory? Is there something else I missed?
- dan-robertson 6y agoIt looks obvious when you write it like that but I think many people are surprised by just how much slower distributed computations can be compared to non distributed systems. Eg the COST paper [1] [1] https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf https://www.usenix.org/system/files/conference/hotos15/hotos...
- datameta 6y agoThe machine used is a 36-core + single gpu. So not quite a home computer yet but this is some serious progress! Paper: https://arxiv.org/abs/2006.11751 https://arxiv.org/abs/2006.11751 Source: https://github.com/alex-petrenko/sample-factory https://github.com/alex-petrenko/sample-factory
- cheez 6y agoThat's my machine!
- cbozeman 6y agoI dunno... $8000 builds a 64c/128t 256 GB RAM workstation with the same GPU these researchers used (https://pcpartpicker.com/list/P6WTL2 https://pcpartpicker.com/list/P6WTL2). That's arguably in the realm of home computer for just about anyone making $90,000 and above, I would think; I would also think anyone working in those fields could command at least that salary or greater, unless they're truly entry level positions. Seems it would be a reasonable investment for someone actively working in the area of machine learning / artificial intelligence.
- eanzenberg 6y agoAnyone that can afford a car can afford this.
- 6510 6y agoThe interesting places you could take it!
- deleted 6y ago[deleted]
- nqzero 6y agowhat's the per-hour cost spot price of this machine on AWS ?
- ChuckMcM 6y agowell an m5ad.24xlarge is 96 threads and 384G with your own 2xSSD (saves on EBS bandwidth costs). So fewer threads but a bit more memory. (we'll guess that is a 48 core EPYC 7642 equivalent with 96 threads since there is no 96 core version) That bad boy costs $0.96/hr on the current spot prices (https://aws.amazon.com/ec2/spot/pricing/ https://aws.amazon.com/ec2/spot/pricing/)
- dmurray 6y agoHow does having your own SSD save on bandwidth? You have to send the data to and from the cores where the computation is taking place. There are certainly some workloads where it makes sense to own your own storage and rent computation, but you can't assume that by default for a "powerful AI" workload.
- gameswithgo 6y agothat is a home computer bow thanks to threadripper
- godelski 6y agoThey mention multiple architectures, including down to a 10 core, 128GB ram, and 1080Ti setup. Next setup is improved CPU, double ram, and a 2080Ti. Third setup is 4 2080Tis. See table A.1
- joe_the_user 6y agoSerious question, is there are reason to have a GPU and that many cores? My assumption would that either the GPU or the CPU is the bottleneck, most likely the GPU. Why not spend money for more GPU and fewer cores?
- datameta 6y agoFor shallower models CPUs aren't overshadowed by GPUs as much - but beyond a certain # of parameters the CPU loses out as GPUs do vector math highly efficiently. Some experiment results: https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-deployment-of-deep-learning-models/ https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-depl...
- joe_the_user 6y agoBut that's a fight between either CPUs OR GPU. It doesn't given indication that both could be useful simultaneously.
- chrisbroadfoot 6y ago"The team also tested their approach on a collection of 30 challenges in DeepMind Lab using a more powerful 36-core 4-GPU machine. The resulting AI significantly outperformed the original AI that DeepMind used to tackle the challenge, which was trained on a large computing cluster." Well, they presumably tested the same CPU with 4 GPUs (2080 Ti I think) - maybe they wanted to compare.
- vz8 6y agoHow much RAM did their test workstation have? I can't seem to spot it.
- m463 6y agoI'm guessing most of the perf comes from the gpu memory size.
- Enginerrrd 6y agoWhile that's likely true, I generally find that it's quite rare that enough attention is paid to how information is moved between disk, RAM, CPU and GPU. And paying close attention to that can be extremely helpful. Taking the RAM up to 11 can eliminate a lot of the art to it, which is a good thing.
- rbanffy 6y agoA machine in this class can easily have a terabyte of RAM. Add a couple Optane DC sticks and you have enormous storage at exceedingly high bandwidth.
- genpfault 6y agoPaper lists 3 configs, one with 128GB and two with 256GB.
- deleted 6y ago[deleted]
- dan-robertson 6y agoLots of people are focusing on this being done on a particularly powerful workstation, but the computer described seems to have power at a similar order of magnitude to the many servers which would be clustered together in a more traditional large ML computation. Either those industrial research departments could massively cut costs/increase output by just “magically keeping things in ram,” or these researchers have actually found a way to reduce the computational power that is necessary. I find the efforts of modern academics to do ML research on relatively underpowered hardware by being more clever about it to be reminiscent of soviet researchers who, lacking anything like the access to computation of their American counterparts, were forced to be much more thorough and clever in their analysis of problems in the hope of making them tractable.
- the_optimist 6y agoIt's a good analogy. In particular, GPUs completely broke the trajectory of benefitting from carefully-managed compute granularity. One could say that CS/EE is now back-filling harder problems that didn't demand to be solved over the last decade because of GPUs. Optimizing for cache management and branch prediction is very difficult. And most programming is done a level of abstraction that isn't amenable to staying portable (i.e. staying nimble) after optimization. Plug: Staying algorithmically nimble after optimization is a problem we've "solved for" at our startup (monument.ai)
- Barrin92 6y agoIf anything it seems to me that doing the most work under constraint of resources is precisely what intelligence is about. I've always wondered why the consumption of compute resources is itself not treated as a significant part of the 'reward' in ML tasks. At least if you're taking inspiration from biological systems, it clearly is part of the equation, a really important one even.
- fxtentacle 6y agoI'm surprised that this is IEEE worthy and not just common sense. Of course there'll be huge speedups if, and only if, your dataset fits into main RAM and your model fits into the GPU RAM. But for most state of the art models (think gpt with billions of parameters) that is far from being the case.
- p1esk 6y agoYes. Jukebox model was trained on 512x V100 GPUs for 4 weeks. Try doing that on a $8k workstation. Not saying it wouldn't be a worthwhile goal to improve the algorithms so that it becomes possible. At least on a 8x V100 machine, for Christ's sake. Because that's all I got.
- qayxc 6y ago> At least on a 8x V100 machine, for Christ's sake. Because that's all I got. Well that's still one powerful supercomputer and allows you to pretrain BERT from scratch in just 33 hours [1]. I mean that's $100,000 in hardware you have at your disposal right there, which is still orders of magnitude beyond 8k-level workstation hardware... It speaks to the sad affair that is SOTA in ML/AI - only well funded private institutions (like OpenAI) or multinational tech giants can really afford to achieve it . It's monopolising a technology and papers like this help democratise it again. Edit: [1] https://www.deepspeed.ai/news/2020/05/27/fastest-bert-training.html https://www.deepspeed.ai/news/2020/05/27/fastest-bert-traini...
- fxtentacle 6y agoYes, it would be great to see AI training becoming more democratized again, but with its mere ~2x this paper won't help that much, plus the most expensive part in training a novel AI might well be to hire all those people that you need to create a dataset spanning millions of examples.
- qayxc 6y agoTraining data isn't always an issue. There are plenty of methods that don't require labels or use "weakly labelled" data. Since most contemporary methods only make sense if lots of training data is available in the first place, many companies interested in trying ML do have plenty of manually labelled data available to them. Their issue often is that they don't want to (or can't for regulatory reasons) send their data into the public cloud for processing. Any major speed-up is welcome in these scenarios.
- softwrdethknell 6y agoApple’s in-house SOC is the future. I suspect the cloud has a decade, maybe less, of hype to grift on. Huge data sets on a personal computer and opt-in data sharing with business and healthcare, etc will be the new norm. Further out, software as we know it will cease to exist as entirely custom chips per application are the norm. IN TIME. New hardware wars to capture consumer attention incoming.
- rbanffy 6y agoI don't need to own a fast workstation unless I want to continuously train my models. I can, however, quickly get a cloud instance that's much larger than that and train the model at a fraction of the time and cost of a desktop workstation.
- neatze 6y agoAnd continuously training models is very hard, practicality in RL environment, even then cloud services over long term is possibly more cost effective solution, then hosting your own small cluster (few tightly packed racks).
- rbanffy 6y agoStill, I don't mind the excuse to get a 128 core dual EPYC with a terabyte of RAM and wide PCIe flash storage. But I would rather not have proprietary GPU drivers.
- convivialdingo 6y agoYou aren't wrong - we've vacillated between personal and cloud(mainframe, shared, or centralized) for many years now and it seems that we'll probably continue on this hybrid approach for a while. Personally I prefer to own my own compute, my own storage and my own hardware. Cloud isn't any cheaper in the aggregate long-term, but it does spread risk and costs across a longer term. Considering everything I'd rather own the risk and up-front costs just for my own privacy and self-determination.
- rbanffy 6y agoIt always could be trained on a single computer. It was just a matter of physical size versus time.
- rgovostes 6y agoIs SLIDE being used anywhere, or were flaws discovered? It was supposed to massively accelerate training on CPUs. https://www.hpcwire.com/off-the-wire/rice-researchers-algorithm-that-trains-deep-neural-nets-faster-on-cpus-than-gpus/ https://www.hpcwire.com/off-the-wire/rice-researchers-algori...
- lihan 6y agoDepends on the size of a single computer. Cloud services have something 200 core, 4TB in a single machine too
- DrNuke 6y agoA mid-range gaming laptop can do much more and better in 2020 than in 2018, thanks to novel available batch functions, a number of preliminary hacks and much hw-friendlier frameworks.
- neatze 6y agoMay be my AI tasks are to simplistic, but I never had problem training AI on single machine and as many pointed out there always cloud services, otherwise I find it an impressive work, it takes some courage to say well there so many huge companies with there's frameworks, but we can outperform them all in specific problems. Couple things that stood out from github page: Currently we only support homogenous multi-agent envs (same observation/action space for all agents, same episode duration). For simplicity we actually treat all environments as multi-agent environments with 1 agent. My speculation is that this why they gained such dramatic performance improvement. (but I might be very wrong)
- lostmsu 6y agoTheir results on lasertag are very poor. Looks like their network was incapable of solving it. Which probably means for apple-to-apples comparison to IMPALA they need a better network, which might require them to forego their throughput advantage.
- alex_petrenko 6y agoThe network in our work is exactly the same as in IMPALA paper. The overall score of our agent is slightly higher, it did better on some envs and it did worse on some other envs. These lasertag levels are exploration problems, and with a bit of hyperparameter tuning they are not difficult, it's just the agent that learned to do 30 different things somehow sucks on these levels.
- lostmsu 6y agoWere other problems less about exploration? If not, I'd argue lasertag might be more important. In various benchmarks the geometric mean is usually used to compare total score across different tasks to account for severe issues with specific tasks. If your network is the same as IMPALA, why do you show its results with different hyperparameters? Were some of them necessary for the optimization (e.g. reduced batch size)?
- YeGoblynQueenne 6y agoI'm reminded of the following episode of Pinky and the Brain: Brain plans to use a "growing ray" (originally a "shrinking ray") to grow Pinky into super-size while dressed up as Gollyzilla, while Brain would turn himself gigantic and stop him, using the name Brainodo, in exchange for world domination. However, the real Gollyzilla emerges from the ocean and starts to rampage through the city, making Brain think that the dinosaur is Pinky. The episode ends with the ray going out of control and making everything on Earth grow, including the Earth itself, to the point that Pinky, the Brain, and even Gollyzilla are mouse-sized by comparison again. https://en.wikipedia.org/wiki/List_of_Pinky_and_the_Brain_episodes https://en.wikipedia.org/wiki/List_of_Pinky_and_the_Brain_ep... Being able to train deep RL models on commodity hardware is only an advantage if there isn't anyone that can train on more powerful hardware (or if somehow training on more powerful hardware fails to improve performance with respect to your model). Otherwise, you're still just a little mouse and they have all the compute.
- DrNuke 6y agoYou still need a problem, domain knowledge, a good question and relevant data?
- YeGoblynQueenne 6y agoI don't think FANGs lack any of that, either.
- DrNuke 6y agoNah, they pursue the problems that are relevant to them, which are a very small subset of the problems being tackled globally... so you can still win your particular race at your snail pace, if they are not running?
- YeGoblynQueenne 6y agoThat's not exactly how it works. For example, Go, the board game, is not particularly relevant to Google, owner of DeepMind. Yet they spent considerable resources in creating and training a succession of systems that excel at it. Besides some applications, big technology companies see AI partly as a future investment, partly as very good publicity. My guess in any case. Then again there are the general problems that are relevant to everyone, like natural language understanding, question answering, image recognition and so on, high-level tasks for which solutions have broad applicability. Rather tautologically, such tasks are always relevant to anyone who can perform them well. In any case, if this was not the case there wouldn't be any motivation for academic teams to find ways to train with smaller computing resources, as the article reports.
- godelski 6y agoThere's a lot of research that you can get away with on smaller hardware. Much of my own I can do with a single 2080Ti. BUT at times it is extremely frustrating and I don't have the memory to tackle some problems with just that hardware. If you want to see major improvements in the academic and home side of AI then NVIDIA and AMD need to bring more memory to consumer hardware. But there isn't much incentive because gamers don't need nearly the memory that researchers do.
- gwern 6y agoIt's very nice optimization and systems engineering work which shows what a computer can do if you use it properly. One correction: it's not Doom, but ViZDoom, a simplified version designed for DRL.
- xiaodai 6y agothe most misleading article ever. Twice as fast as best known method ain't gonna cut it for training AlphaZero Go for example. But still 140,000 images per second! That's nuts. Even 70,000 is nuts. I can only play 1000 2048 games in one second. Damn! I am slow
- alex_petrenko 6y agoAuthor of the paper here. Ask me anything!
- YeGoblynQueenne 6y agoHi and congratulations for the article on IEEE. The article makes the point that your technique will give an advantage to academic teams that don't have the resourcers of big corporations. To me it seems that your technique optimises the use of available resources, but the amount of available resources remains the deciding advantage. That is to say, both large corporate teams and smaller academic teams can improve their use of resources using your proposed approach, but large corporate teams have more of those than the smaller academic teams. So the large corporate teams will still come up ahead and the smaller academic teams will still be left "in the dust" as the article puts it. What do you think?
- highfrequency 6y agoThe key is that large corporations/labs achieve scale through many distributed machines. This paper explores optimizations that are particular to a single multi-core machine. These optimizations exploit low-latency shared memory between threads on one machine, and thus cannot be replicated on a distributed cluster.
- highfrequency 6y agoHi Alex, cool stuff! A clarification from Section 3.2 on coordinating between rollout workers and policy workers: you say that we want k/2 > t_inf / t_env for maximal performance. If t_env is defined as the time required for one thread to simulate one step, it seems like we want k/2 > N * t_inf / t_env, where N is the number of cores/threads on the machine. For example, say t_inf = 10 microseconds, t_env = 1 microsecond and we are training on a 30 core machine. When k=600, we batch 300 inference steps in 10 mics, and complete 300 simulation steps on the other half of the rollout workers in 10 mics. Both cohorts of rollout workers finish at the same time, achieving optimal performance. Am I thinking about this correctly? Also, this equation assumes that we can batch up k/2 inference jobs on the GPU without increasing t_inf correct?
- andreyk 6y agoKind of a weird headline, the vision-based RL this article dubs 'powerful AI' could already easily be trained on a single (pretty expensive) computer. They say as much in terms of the speed ups they provide: "Using a single machine equipped with a 36-core CPU and one GPU, the researchers were able to process roughly 140,000 frames per second while training on Atari videogames and Doom, or double the next best approach. On the 3D training environment DeepMind Lab, they clocked 40,000 frames per second—about 15 percent better than second place." So, not massive this-is-now-doable speedup.
- hrgiger 6y agoInteresting article, i will check github at weekend, but assuming atari resolution 40x192 single channel and float size 4 byte and 140000 image you got 4300800000 bytes roughly 4gb data. I wonder if they talk about complete training time
- fongitosous 6y agowith a titan and a threadriper for instance what would be the results?
- del_operator 6y agoHow many RPi 4’s would you need?
- imranq 6y agoThis is incredible, but won’t these techniques just make the gap wider as big tech firms create even larger models?
- bno1 6y agoI wish machine learning would have become mainstream on a language with more competent multithreading capabilities than python. During my machine learning course I knew I could squeeze more performance out of my code by parallelising data preprocessing and training (pytorch), but python cannot do proper multithreading. The multiprocessing module requires you to move data between processes, which is slow.
- cosmodisk 6y agoWould porting certain libraries be a huge undertaking?
- bno1 6y agoWell the core of those libraries is already written in C or C++, so it's a matter of writing bindings to interface with the core. I think tensorflow and pytorch today have some C/C++ bindings, but I don't know how good they are.
- rpedela 6y agoShouldn't be slow if shared memory is used. https://docs.python.org/3/library/multiprocessing.shared_memory.html https://docs.python.org/3/library/multiprocessing.shared_mem...
- bno1 6y agoFirst time I've seen this, looks like its brand new (python 3.8). The problem now is that you have to serialize/deserialize your data and modify your numpy/pytorch code to use the shared memory. It's an improvement in performance, true, but not as fast and easy as just sharing variables between threads.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- blargmaster33 6y agoPattern recognition is NOT GOD DAMN AI!
- 0xbkt 6y agoOut of the question, where should an indie .IO game developer start to build their AI players in their game? For the record, I am a solo developer in progress of developing a to-be-online browser game. I must make intelligent bots and keep players busy until the time it has a lot of online players. I had a look at Reinforcement Learning but I am not sure people are really using for this use case.
- confuseshrink 6y agoI would start with David Silvers (DeepMind) youtube series to get an idea of what's possible or not. Running an already trained reinforcement learning agent is relatively cheap (unless your model is massive). I suspect the reason people aren't using it yet is because it's a) really difficult to get right in training, even basic convergence is not guaranteed without careful tuning b) really difficult to guarantee reasonable behavior outside of the scenarios you're able to reach in QA. edit: Link to lecture series https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTra...
- tpetry 6y agoThere‘s a really old Game AI book by O‘Reilly with a lot of clever simple solutions. You dont need the most complex, innovative and state of the art algorithms for games, most games have a really simple ai concept.
- haffi112 6y agoStart by making agents by using heurists and not AI. Once you have these in your game at least you can release it. If you are not happy with that performance then you can start to look at RL methods.
- rasz 6y agoGame AI has very little, if anything at all other than name, in common with Machine Learning AI. The answer is dont touch ML, start with FSM.
- jchrisa 6y agoIs this an early warning indicator that the cloud-to-edge pendulum has begun swinging back in the direction of personal computing?