12 ms·
I feel like 64 cores is getting rather close to a tipping point: What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the th
by Uehreka 6y ago
I feel like 64 cores is getting rather close to a tipping point: What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the thousands of CUDA cores on a Quadro card? I’m sure right now there are workloads that just need particular x86 instructions, but that feels like a temporary problem.
Am I wrong about that being a temporary problem (that would feel frustrating)? Are these cores just that much faster than CUDA cores? What else am I missing here?
I often hear people talk about getting CPUs like this for deep learning research, but all the deep learning work I’ve done goes straight to CUDA and lands on my GPU.
- lmilcin 6y agoCuda and x86 are different things. I do not personally, but I work with another team that uses tens of thousands of cores for calculations. Just day to day business.
- photon12 6y agoRunning multiple parallel VMs with Xen or KVM is a plausible example.
- staticfloat 6y agoOne big difference is memory availability; the amount of RAM directly accessible from a 64-core CPU is much larger than that of the thousands of CUDA cores on a Quadro card. Perhaps for many workloads intelligent use of the PCIE bus can make up for that, streaming datasets in/out of the card, but for others, having random access to hundreds of GBs of data may be non-negotiable. I'd be interested to know what workloads truly require that; most simulation solvers that I can think of that require large amounts of RAM are working on a discretized spatial grid, and the physical laws they're simulating are fairly "local"; e.g. you can process one area then move on to the next, in a nice stencil operation. This lends itself well to both parallelization and a sort of data-locality, so you can stream the necessary pieces of data in to the cores in an orderly fashion. I don't know what kinds of massively parallelizable workloads require much more random access of memory.
- choeger 6y agoDAE and ODE simulators do not work with that spatial distribution.
- vosper 6y agoI’m not sure about “require”, but we use multi-hundred-gigabytes or RAM instances for our Elasticsearch coordinator nodes, which seems to work well.
- CSDude 6y agoElasticsearch recommends <32GB because of 32-bit per their doc, is it different for coordinator nodes?
- adamtulinius 6y agoThat recommendation is only for the JVM heap to my knowledge.
- Xylakant 6y agoFor coordinator nodes, only the HEAP is relevant, excess RAM for disk buffers is only of interest for data holding nodes.
- Xylakant 6y agoThe underlying reason is that at around 32GB the JVM internally shifts from 32bit pointers for HEAP management to 64bit. The feature is called compressed object pointers (compressed oops) The increased pointer size uses substantial amounts of RAM, a reasonable estimate is that you need at least ~45GB to not have a loss in total available HEAP. Also, GC cycles get longer the more HEAP you have. However, some workloads may require more heap to even be possible and there’s pauseless garbage collector implementations able to handle substantial amounts of memory, so it is possible. It’s just not something you should be doing unless you know exactly what you are doing. See https://wiki.openjdk.java.net/display/HotSpot/CompressedOops https://wiki.openjdk.java.net/display/HotSpot/CompressedOops
- deleted 6y ago[deleted]
- bitL 6y agoReinforcement Learning, in-memory databases, number crunching, raytracing/animation.
- loser777 6y agoIn a single workstation form-factor, once you get enough powerful GPUs, it takes a good amount of CPU power to feed them, even for pure CNNs that GPUs are supposed to be great at. Think data augmentation for training, etc.
- choeger 6y agoOne use case that comes to mind is compilation of languages like C/C++, where you have many heterogeneous, independent, long-running tasks.
- dragontamer 6y ago> What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the thousands of CUDA cores on a Quadro card? Thread divergence. CUDA-cores are linked and take if-statements and for-loops together. This means that if one CUDA core takes a for-loop 1-million times, 31-other CUDA cores will take the 1-million loop with them. (32-cores per NVidia SM). EDIT: CUDA keeps things semantically correct per thread by "throwing away work" (execution mask is disabled, so those 31-other threads effectively execute NOPs), but the cores are wasted. Worst case scenario, your CUDA cores accomplish 1/32th the work they could do, as they spin idle waiting for the last 1 thread to complete a for loop or its unique combination of if-statements. Matrix multiplication doesn't have any thread divergence, because all threads loop the same amount of times on all workitems. This means that Tensors / Deep Learning simply ignores the problem, because they're written as matrix multiplication problems. Avoiding thread-divergence, or mitigating its effects, is possible, but requires advanced programming skill that few have. -------- Examples: Chess -- Traditional multithreaded chess algorithms have every thread check out a different branch of the chess search tree. But because each branch has different positions and moves, its very difficult to code it in such a way that all threads are doing useful work. Web Servers -- If every thread is handling a different request, then every thread probably traverses a different set of if-statements and loops. This is a thread-divergence nightmare. Databases -- SQL Databases traditionally have each thread running a different, independent SQL query. Like web-servers, its difficult to imagine parallelism from a high-level. ------- However, elements of Chess, Web-servers, and Databases can probably be efficiently parallelized onto GPUs. Chess -- This github repo demonstrates that enumerating positions can be done GPU-parallel: https://github.com/ankan-ban/perft_gpu https://github.com/ankan-ban/perft_gpu Web-servers -- Text parsing, Regex, and many other problems common to Web-servers have been parallelized on GPUs. The main question for me, is whether or not the PCIe traversal would be worthwhile (lower bandwidth than RAM, high latency). Web-servers tend to be I/O heavy and less compute heavy... probably not dense enough to benefit from GPU compute. Databases -- Merge-join and Hash-join probably can be parallelized to GPUs. Figuring out the optimal CPU + GPU team is going to be a research problem over the next 10, maybe 20 years.
- asah 6y agohttps://www.google.com/search?q=gpu+postgres https://www.google.com/search?q=gpu+postgres
- imtringued 6y agoGPUs can't run the same code you would have written on a CPU. If you had the choice between an equally performing CPU and GPU you would always use the CPU.
- achempion 6y agoWhy? Isn't the CPU will cost more than GPU because of more general instruction set?
- jacquesm 6y agoOn a per-flops basis yes. But it all depends on your workload, for some workloads GPUs are a better match than CPUs and vice-versa.
- pjc50 6y agoNot really: the CPU costs more and can handle more workloads because it has more complicated control flow. These days, arithmetic is cheap and control flow is expensive. GPUs and SIMD achieve high throughput by tying a set of arithmetic units together with the same control flow. As soon as you get complicated data-dependent workflows - as soon as you put "if" in your code or a virtual method - the GPU starts getting less efficient. It's very much a "train vs car" argument: a train can deliver a large number of execution units , provided they're all going to the same place, whereas a car can re-route far more easily.
- neatze 6y ago"train vs car" argument - really nice analogy.
- syntheticcorp 6y agoFuzzing programs is really a CPU task, although it can usually be horizontally scaled across multiple boxes.
- dragontamer 6y agoSomeone wrote a SIMD-fuzzer across AVX512. Its quite possible that someone figures out an x86 emulator on GPUs that fuzzes register values, similar to AVX512 fuzzing. https://gamozolabs.github.io/fuzzing/2018/10/14/vectorized_emulation.html https://gamozolabs.github.io/fuzzing/2018/10/14/vectorized_e... ------ Fuzzing is unique in that 99% of code probably traverses the same set of if-statements or branches. Whenever code significantly diverges (usually defined by Fuzzers as a significantly different instruction-pointer path of some kind), you've successfully fuzzed a new result. Fuzzers... strangely enough... are programs I'd expect to run pretty well on GPUs or SIMD. If enough R&D research effort were put into it.
- 0xfaded 6y agoPhotogrammetry algorithms that process multiple images independently and then combine the results. Unfortunately they tend to be CPU heavy, and what can be is done with GPU help.
- pengaru 6y agoCompiling large projects like the linux kernel is an obvious one.
- swiley 6y agoYes. Even the cheaper zen processors make this so much more convenient. Gentoo is actually a usable distro again and most of the packages (even things like mono) install decently fast.
- aden1ne 6y agoSuch workloads are typical in bioinformatics, as the majority of tools do not support running on GPUs. Data parallelization (e.g. run the same tool on every chromosome independently) is a common strategy used in that field.
- fock 6y agolike all of the legacy quantum chemical and FEM codes or anything else you run in HPC (which are a lot of codes, which are not ported from one day to the other by an army of self-declared ML-pros). Part of them can be GPU-accelerated, but then you can easily eat TBs of memory with the right system and GPUs still have a way to go there.
- zumachase 6y agoSurprised databases haven’t been mentioned. Anything stateful is still far easier to scale vertically. Otherwise you have to go multi master and start sharding across smaller instances. A massive chip like this can let you keep a pretty simple master/follower architecture for HA without having to share.
- bobjordan 6y agoI'm using a 22 core / 44 thread xeon workstation for development and my primary joy using it is being able to run my pytest test suites in parallel. I went from ~45 minutes to run the test suite to about a minute. It is also pretty sweet to be able replicate our three physical servers for bomquote.com, locally.
- elorant 6y agoHonest question because I'm interested for something similar, why did you go for a Xeon and not an Epyc rig?
- jdboyd 6y agoYou can go to Dell, HP, or Lenovo to get a premade Xeon workstation. Getting Epyc premade requires rather more research to find and qualify a vendor, and that vendor may not make it easy to configure and but from the web site with or having to enter a relationship with a sales rep.
- bobjordan 6y agoI bought the xeon from Ebay at a discount about three years ago, when it was still fairly new. It was some sort of engineering sample and I paid about $3K for it while the list price was at least a multiple higher. We used it for CPU rendering for several years and I've only recently repurposed it for a development workstation, as we moved to software that can utilize GPU rendering. These days I almost certainly buy an AMD Threadripper or Epyc for a development workstation.
- flywheel 6y agoI mainly write nodejs code, and now that it supports threads I've been hammering my CPU with all kinds of parallel tasks. One common thing I do is crunch a few thousand 16MB+ JSON files in parallel. Multiple cores saves quite a bit of time parsing huge JSON payloads. Sure, people may crap on me for using Javascript, whatever... It's what I prefer to use, and 64 cores would only help do all sorts of things in parallel, saving me time. There's all kinds of other things I do that utilize multiple cores. I also run a lot of VMs.
- dTal 6y agoCPUs are much more general! Unscientifically, on my 2950X/1070Ti workstation, Cycles takes approximately the same amount of time to raytrace a scene on the CPU vs the GPU. I think this is a great example of a highly parallel workload that is challenging for GPUs due to lack of coherence - extreme props to the Cycles team for making it work on GPU at all. Also, year-on-year performance increases for nVidia GPUs are relatively boring, while Threadrippers routinely double in performance. I have just built a 3970X/RX5700XT workstation, and while I have not benchmarked it yet I expect the CPU to smash the GPU for Cycles raytracing. Also worth noting that CUDA is a proprietary architecture from a single company - targeting it locks you in, and you need to run proprietary blobs to make it work. All in all, CUDA works well now but I think it's ultimately a dead-end architecture - hugely parallel CPU architectures will eclipse it.
- jacquesm 6y agoThe people that use CUDA don't care about the underlying architecture, they need results. Which ever manufacturer will deliver those results will get the $. Raytracing is not an ideal match workload wise for a GPU, too much divergence, but there are plenty of workloads where GPUs will have an edge for along time to come.
- qayxc 6y agoIt's more than that - NVIDIA invested massively in their software ecosystem and mindshare (e.g. by a huge sponsoring program that saw universities flooded with their hardware). AMD cards offer comparable compute performance and are sometimes even better. What AMD lacks, though, are ready-to-use software packages like CUDA, cuDNN, cuBLAS, and the like. NVIDIA basically bought their way into the research community and solidified their strong foothold by keeping CUDA proprietary. It's fascinating how other companies (Microsoft) get bashed on a regular basis for using the exact same tactics (see Direct3D vs OpenGL on Windows), while people are perfectly fine with NVIDIA doing it...
- jacquesm 6y agoBecause NVIDIA did not do anything illegal - so far. MS is a convicted monopolist that actively went after competitors rather than just to let the market decide. What NVIDIA has done so far is perfectly fine in my book, paying companies not to use competitor hardware, funding third party lawsuits and stealing IP, the browser wars and so on are things that Microsoft has done.
- sradman 6y ago> What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the thousands of CUDA cores on a Quadro card? Column stores like SAP HANA. Compressed bitmap indexes leverage SIMD but not floating point matrix math GPUs. The workstation vs server case is narrower though; benchmarking, dev testing, and local MS PowerPivot like analysis on large in-memory datasets.
- ckluis 6y agoGive me 1 core for the Chrome shell and then 1 core per tab :)
- mschuetz 6y agoTwo reasons why I can't do my stuff with CUDA even though I've got thousands of work packages that are independent of each other: * I'm splitting data sets into batches of around 50-300MB, but each individual batch must be processed sequentially. Each batch loops through the data (multiple times), and every single iteration of a loop depends on state from previous iterations. * Altough each batch essentially does the same thing, it does not operate in lockstep. Lot's of ifs, different execution paths, differently sized sub-problems, etc. I'd benefit from having 200 CPU cores and enough RAM to fit the batches, but I can't let individual GPU threads run through batches of 50MB, each.
- mirekrusin 6y agoChrome?
- secondcoming 6y agoThe impending doom that is WASM will mean that 'dynamic' and 'interactive' adverts will use all these cores.
- throwaway_pdp09 6y agoWhat a miserable hopeless post - "they're going to fuck us and what can we do????". Answer: quite a lot, if you have even minimal technical understanding (like, enough to use a blocklist or an ad blocker). Are you incapable even of that?
- secondcoming 6y agoYes.
- throwaway_pdp09 6y agoWell... upvoted for the honesty.
- kevin_thibedeau 6y agoYour ad blocker is going to filter a private web socket link funneling ads and content together? It's going to patch the WASM?
- CyberDildonics 6y agoWhy would it be difficult to turn off web sockets or turn off multi-threading? Multiple threads already run on single cores all the time, there is no reason a web browser couldn't just put them all on one core.
- throwaway_pdp09 6y agoIt's your broswer. If you have any tech chops you can make it what you want. If.
- CyberDildonics 6y ago
- k_bx 6y agoAny compiler that can properly use multiple cores could potentially benefit. Rust (and Haskell's GHC) uses all 8 cores I have during a sub-period of recompilation, so I'd guess it could as well use more.
- rowanG077 6y agoI know GHC can become slower with more cores so be carefull.
- lattalayta 6y agoAt the moment, offline rendering is still massively parallel, but you run into memory limitations when trying to render on the GPU. High end VFX scenes using Renderman or Arnold can push up to 100 GB+ of memory and run for multiple hours on standard 12-24 core machines. I believe some GPU renderers can do "out-of-core" rendering, but not all of them support that yet. The high end renderers usually split up a large image into "buckets" and then perform path tracing on those regions of an image, and combine the results together. If you have more cores, you can increase the number of simultaneous buckets to evaluate. To put this in perspective, a lot of the recent news around real-time raytracing is when a game engine does 1-4 samples per pixel and then aggressively denoises (in 2D) the result. That's why you often see splotches or errors. On the other hand, high end VFX renders will end up doing hundreds or even thousands of samples per pixel to get a clean, high quality, physically plausible result.
- glogla 6y agoA large SQL database running queries from many users, say a data warehouse or a backend DB for web app like internet banking or something. You can run on multiple nodes, but that brings its challenges, see Jepsen.
- hellofunk 6y agoThere’s such a huge difference in the programming paradigm between writing for multi cores on a GPU versus a CPU. It’s really a very different kind of software architecture. I think it’s unfair to compare the two. And with such a huge CPU core counts, a lot of the very difficult work of organizing an algorithm to run well on a GPU can be avoided because the CPU approaches the same benefits with a more logical architectural design.
- ATsch 6y agoAlthough graphics companies like to talk about "cores" or "processors" to make their GPUs sound powerful, they are really just counting ALUs. If you marketed this chip similarly, you'd end up with something silly like "1024 AVX Core CPU". A closer analogue of x86 cores is what Nvidia calls a "Streaming Multiprocessor" and AMD calls a "Compute Unit". There's generally only up to 64-80 of those on the highest end GPUs too. However the difference is that each of those cores is able to perform 40-64 SIMD operations at once, compared to x86's puny 8 or 16 or so.
- ajross 6y ago> What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the thousands of CUDA cores on a Quadro card? Building software, for one. C compilers and python interpreters don't run on a GPU. Lots of stuff doesn't run on a GPU. In fact in practice the only things that run on a GPU are the tiny handful of known subproblems that the industry has collectively decided are "GPU problems". Like it or not general purpose scalar software is, has always been, and will always remain the standard mechanism by which computing hardware is applied to new problems. Everything else is an optimization around the edges.
- scotty79 6y agoInteresting. Is there a compiler, that runs on GPU?
- the_pwner224 6y agoGPUs are mainly good at doing matrix multiplication and other math with little branching. Compilers have a lot of if/else statements; training neural networks does not. IIRC (and I may be wrong) the architecture is something like many GPU cores share things like the code cache and branch prediction, so if the different cores have to execute different code paths there is a big performance decrease. Edit: another reply to this top level comment: https://news.ycombinator.com/item?id=23788755 https://news.ycombinator.com/item?id=23788755
- Symmetry 6y agoAlso GPUs really like predictable memory access patterns so that they can load the data that'll be needed next ahead of time into registers or dedicated caches explicitly. They don't have cache hierarchy features a CPU has to minimize the average disruption from only partially predictable memory accesses and they don't have out of order windows to paper over the occasional low level cache miss.
- FpUser 6y agoNot to counter the main argument but CPU loves it some predictable patterns as well. pattern like the one below will execute up to 10 times faster if array is sorted due to data locality and branch prediction sum: float a: float array[some_huge_number] fill_array(a, random(1)) for f in a if a[i]>0.5 sum=sum+f
- bob1029 6y agoThis is a point I have been trying to make for a few years now to ever-decreasing amounts of criticism. I think the reality is starting to set in that x86 is actually incredibly powerful, and when you have 64+ parallel copies of that computational facility, you can perform some insane magic. Especially when you start to consider the memory/computation domain you are operating within on CPU when contrasted with a GPU: With a GPU in the loop, there is an extremely painful hop across PCIe before you can experience any degree of cache coherency. There are protocols, buffering and driver stacks involved. Latency is much higher. You have to plan ahead and think about what memory the GPU needs to mutate vs what memory the CPU needs to mutate. You have to split your application, language, and frameworks across 2 computational domains. How we have tolerated this for so long is beyond me. I think most people who get caught up in it are chasing shiny marketing a lot of the time. This criticism aside, I do think GPUs are still very useful for many applications and users. With just a CPU, you are looking at L3 as the typical worst case for cache coherency domain. And L3 is getting to be ridiculously large. If your high-performance application's binary image cannot entirely fit within L3 of a modern x86 CPU, you are probably doing something very wrong. The other big advantage with just a CPU is you have 1 cache-coherent memory domain and a single instruction set to answer to. This means you can write 100% of your software in a single language/framework. Well-architected x86 applications can push instructions many times faster than their core clock speed would seem to indicate possible. Using special frameworks that have sympathy with this memory model mean that just a single x86 core can be made to push tens of millions of logical business transactions per second. And then you have 63 other cores to play with. Extremely deep pipelining and OoO execution are what allow for x86 to chew through general purpose computation so easily. Anything with a lot of recursive depth (e.g. raytracing) benefits massively from this style of processor. I think the next big revolution is developers realizing that they can start porting GPU applications to CPU. Things like raytracing are much better suited for the memory model offered by a pure x86 domain. Sure, a GPU can accelerate some aspects of raytracing, but then you then need to shuttle that information back and forth over a higher latency link, and also fight with 2 completely different technical stacks. Needing to know how to write code for both GPU and CPU is easily the biggest problem with all of this. Especially when you consider how fast GPU APIs move relative to x86. Performance is not the only constraint when dealing with software, especially software as complex as AAA game engines and 3d modeling software. Someone still has to reason with and maintain this stuff over time.
- jarvist 6y agoAlmost all scientific computing. In spite of considerable effort, there are relatively few workloads that use GPUs well, and even for codes which do work on GPU, you tend to end up with partial functionality, and a very restrictive region of parameter space where they work efficiently. (And often you run out of memory very quickly - scientific workloads typically run multi-node and share RAM across the nodes via Global Arrays or equivalent. Using multiple terabytes of memory is absolutely normal.) The computation in deep learning is actually quite simple (matrix-vector operations), and still it has taken considerable effort to develop efficient kernels (such as cuDNN). The next UK supercomputer is based around AMD's 64 CPU chips for this reason. One concern is that we may now be limited by the shared memory bandwidth. https://www.nextplatform.com/2019/10/18/amd-cpus-will-power-uks-next-generation-archer2-supercomputer/ https://www.nextplatform.com/2019/10/18/amd-cpus-will-power-...
- deleted 6y ago[deleted]
- gameswithgo 6y agoa server providing 64 virtual machines in the cloud
- satai 6y agoAlmost anything, that doesn't move lots of data in lockstep (specially - anything with lots of it - then - elses) is much better to run on CPU cores.
- michaelt 6y ago> What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the thousands of CUDA cores on a Quadro card? Servers serving multiple independent requests, VMs or Docker containers. In other words, the same thing people were doing with four 16-core servers, just now it takes up less space :)
- CyberDildonics 6y agoGPUs are good for split join parallelism. They don't even really have real branching and you mostly have to loop a constant amount of times. Meanwhile any data without dependencies can be dealt with on a CPU core. Not only that, huge amounts of software could use a lot more concurrency if it was structured differently. Even a web browser (to my knowledge) doesn't split up DOM layout, parsing individual files, decompressing images, decompressing video etc. into separate threads for each file.
- cptskippy 6y ago> Are these cores just that much faster than CUDA cores? What else am I missing here? Comparing GPU/CUDA/TPU Cores to CPU Cores is like comparing a riding lawnmower to an automobile. A GPU/CUDA/TPU Core is optimized to handle a different type of workload than a CPU. Just because your riding lawnmower can cut your grass faster than your car, doesn't mean your lawnmower is the best thing for highway driving.
- smichel17 6y agoMy understanding of these problems is totally naive, but I wonder if you could mitigate speculative execution vulnerabilities (meltdown, spectre, yes I know the former does not apply to amd) by using the excess cores to speculate enough that a cache hit is no longer meaningful data.
- hyperbovine 6y agoI see this response often and I have to say, "just throw it on the GPU" is phrase only uttered by someone who has never personally attempted to do that. The simplicity of CUDA and `pip install tensorflow-gpu` belie a huge amount of engineering prowess; go look at the source for XLA (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/compiler/xla https://github.com/tensorflow/tensorflow/tree/master/tensorf...) if you don't believe me. To answer your question, tons of scientific computing tasks parallelize nicely and are a horrible fit for a GPU, because they involve branching, or non-deterministic runtimes, or need a lot of memory bandwidth, or rely on highly optimized legacy numerical code that is written in Fortran, or any one of a number of other reasons. Deep learning is a very narrow use fact and, in fact, GPUs and DL symbiotically co-evolved over the last 15 years to the point where GPU hardware is essentially designed to efficiently train neural nets. (And run games.)
- api 6y agoTasks that involve branching, for one. GPU's are fundamentally different kinds of processors. They are for applying similar transforms to vast amounts of data. There are other area where big general purpose X64 or ARM cores excel too, like bit twiddling, memory random access, etc., but branching logic is the fundamental one.
- slewis 6y agoIn reinforcement learning, you typically need to run many copies of an environment (an Atari emulator for example) to collect experience for your agents. Environment code usually makes more sense on a CPU, because it’s a lot more branchy and stateful. Think of two agents that run for awhile and end up in different parts of a virtual world, that require very different code to execute. This might be pretty difficult to parallelize on a GPU. After collecting a bunch of experience, you update whatever function you’re optimizing. This could be something like a neural network that is the brain of your agents. The update step can happen on a GPU. But then we need to go collect a lot more experience again, so we’re back in CPU land. Experience collection often dominates overall compute time in reinforcement learning. Which means you want a lot of CPUs.
- MaxBarraclough 6y ago> What workloads are so massively parallel that they can use 64 cores of x86 but can’t use the thousands of CUDA cores on a Quadro card? I'm a bit behind on GPU architecture, but presumably workloads which are branch-heavy, lack coherent execution within a warp (see dragontamer's comment), and don't make use of floating-point. GPUs are not the equivalent of manycore CPUs.
- RandyRanderson 6y ago"Plain old" (not neural net) machine learning tasks would be sooo much faster.
- zubspace 6y agoWe process, merge and create PDF files. Sometimes large multi-GB PDF files. We try to adjust and optimize them in parallel, but often we also need to merge different files into one for further processing, for example printing or archival. We need a lot of ram and it takes a lot of time. The more CPU, the better.
- pandaman 6y agoWhat they call "cores" in CUDA is different from CPU cores. On GPU "cores" and "threads" refer to the ways in a SIMD ALU. Under this classification a 64 core CPU with AVX-256 like Threadripper is at least 512 cores. So 4K "CUDA cores" translates to 128 "CPU cores" when you are not doing massively parallel computations. 128 cores still is no joke but those are not Zen cores, unfortunately. On top of running off much lower clock (7nm GPUs are at ~2Ghz, 14nm is ~1Ghz) their IPC is nowhere close to that of a CPU, add advanced caches and memory controllers, branch prediction, speculative execution etc and CPU cores are much faster. A GPU will beat any CPU on a SIMD-friendly payload but it won't be any good for serving webpages or compiling code or doing any other task you cannot nicely lay out for a SIMD.
- airocker 6y agoSplit it into 32 4 core VMs and rent it out for 30$ a month?
- MR4D 6y agoChrome. Once chrome is setup for this, I’ll be ready for 128 cores.
- EvgeniyZh 6y agoIn deep learning, preprocessing can be pretty heavy, and then you find yourself CPU-bounded and not making full use of your 4x2080 Ti.