6 ms·
The good new is that Nvidia's high GPU prices motivate everyone (Intel, AMD, ARM, Google, etc.) to try and tackle the problem by making new chips, making more e
by xnx 3y ago
The good new is that Nvidia's high GPU prices motivate everyone (Intel, AMD, ARM, Google, etc.) to try and tackle the problem by making new chips, making more efficient use of current chips, etc. For all the distributed computing efforts that have existed (prime factorization, SETI@Home, Bitcoin, etc.), I'm surprised there isn't some way for gamers to rent out use of their GPU's when idle. It wouldn't be efficient, but at these prices it could still make sense.
- Conscat 3y agoI am certain that several years ago, I was given an ad for exactly such a service and even tried it out, but I cannot for the life of me remember its name. It had some cute salad motif, and its users are named "chefs". EDIT: It was just named Salad. https://salad.com/ https://salad.com/ https://salad.com/download https://salad.com/download
- WanderPanda 3y agoWith interconnect being the biggest limitation these days I don’t think this would work.
- xnx 3y agoI'm not familiar with all the varied uses of GPUs but it seems like image generation could feasibly be distributed: large upfront download of models, then small inputs of text and settings, and small output of resulting images.
- WanderPanda 3y agoFor inference I agree! But training requires centralized gradient steps
- vajrabum 3y agoIf you're in a data center and running large training jobs then RDMA over Nvidia Mellanox Infiniband cards over high speed ethernet (like 100GB) are used to ship coefficients around without having that transfer bottleneck in the CPU.
- xxpor 3y ago100 gig, that's considered cute nowadays. https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances-powered-by-nvidia-h100-tensor-core-gpus-for-accelerating-generative-ai-and-hpc-applications/ https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances... 3.2 terabits.
- Goronmon 3y agoThe good new is that Nvidia's high GPU prices motivate everyone (Intel, AMD, ARM, Google, etc.) to try and tackle the problem by making new chips... Or their dominance leads to competition throwing in the towel and investing resources in a market with less stiff competition. I wouldn't be surprised to see AMD start to pair back ivnestment on high-end GPUs if things continue down this path. I would say Intel likely keeps pushing, but I'm less convinced they can actually make much headway in the near future.
- xnx 3y agoAs was mentioned in another thread on a slightly different topic, it wouldn't be surprising to see all non-Nvidia parties unit around some non-CUDA open standard.
- josemanuel 3y agoDo you mean something like OpenCL?
- xnx 3y agoExactly. More resources might get applied to improving it.
- josemanuel 3y agoI must admit my previous comment was mildly sarcastic. What I was after is: OpenCL is that language/framework that is consortium driven and open. It’s been there since the start of time. Still it cannot dethrone CUDA… Nvidia struck gold with CUDA and its lock-in
- andromeduck 3y agoThe main problem with OpenCL is most people eoue rather work with C++.
- haldujai 3y ago
- myth_drannon 3y agovast.ai allows you to rent out gpu
- Uehreka 3y agoThey’re all pretty motivated, they’ve been motivated for years, and almost nothing is happening. This situation isn’t exactly a poster child for the Efficient Markets Hypothesis. Every year just sounds like “Nvidia’s new consumer GPUs are adding new features, breaking previous performance ceilings, running games at huge resolutions and framerates. Their datacenter cards are completely sold out because they can spin straw into gold, and Nvidia continues to develop new AI and graphics techniques built on their proprietary CUDA framework (that no one else can implement). Meanwhile AMD has finally sorted out raytracing, and their consumer GPUs are… well not as good as Nvidia’s but they’re a better value if you’re looking for a competitor to one of Nvidia’s 60 or 70 line GPUs!”
- ericmay 3y ago> This situation isn’t exactly a poster child for the Efficient Markets Hypothesis. I'm unsure why you're criticizing the Efficient Markets Hypothesis or even using it here, but you need to also analyze this with some time horizon because the market and marketplaces are not static.
- willis936 3y agoTheir description could be used to describe the situation in 2023, 2022, 2021, 2020, 2019, 2018, 2017, and 2016.
- lotsofpulp 3y agoEfficient market hypothesis is unrelated to Nvidia’s competitors being unable to offer a competing product so far. https://www.investopedia.com/terms/e/efficientmarkethypothesis.asp https://www.investopedia.com/terms/e/efficientmarkethypothes... > The efficient market hypothesis (EMH), alternatively known as the efficient market theory, is a hypothesis that states that share prices reflect all information and consistent alpha generation is impossible.
- ikrenji 3y agoyou know what he meant...
- IshKebab 3y agoThere have been various attempts but you need a workload that's basically public and also runs on a single GPU (because you don't have NVLink or similar).
- NavinF 3y agoYou can do that for inference, but most gamers have a single GPU with <24GB VRAM which kinda sucks for training. 3090 or 4090 is the minimum to use reasonable batch sizes
- tzhenghao 3y ago> motivate everyone (Intel, AMD, ARM, Google, etc.) to try and tackle the problem by making new chips Yes, there has been repeated efforts to chip at Nvidia's market share, but there's also a graveyard full of AI accelerator companies that fail to find product market fit due to lack of software toolchain support - and that applies even for older Nvidia GPUs and their compatible toolchains, let alone other players like AMD. This isn't a hit on Nvidia, I'm just saying things move so quickly in the space that even the only-game-in-town is trying to catch up. Nvidia is also leading by being one or two hardware cycles ahead of their competition. I'm pretty confident AI workloads in enterprise is their next major focus [1]. I think this more than anything else will accelerate AI adoption in enterprise if well executed. To your point, I think the industry needs to focus more on the toolchains that sit right between the deep learning frameworks (PyTorch, Tensorflow etc.) and hardware vendors (Nvidia, AMD, Intel, ARM, Google TPU etc.) Deep learning compilers will dictate if we allow all AI workloads run on just Nvidia or several other chips. [1] - https://www.nvidia.com/en-us/data-center/solutions/confidential-computing https://www.nvidia.com/en-us/data-center/solutions/confident...
- tric 3y ago> I'm surprised there isn't some way for gamers to rent out use of their GPU's when idle. https://rendernetwork.com/ https://rendernetwork.com/ "The Render Network® Provides Near Unlimited Decentralized GPU Computing Power For Next Generation 3D Content Creation." "Render Network's system can be broken down into 2 main roles: Creators and Node Operators. Here's a handy guide to figure out where you might fit in on the Render Network: Maybe you're a hardware enthusiast with GPUs to spare, or maybe you're a cryptocurrency guru with a passing interest in VFX. If you've got GPUs that are sitting idle at any time, you're a potential Node Operator who can use that GPU downtime to earn RNDR."
- euazOn 3y agoAlso the Horde for Stable Diffusion, pretty good concept: https://github.com/Haidra-Org/AI-Horde/blob/main/FAQ.md https://github.com/Haidra-Org/AI-Horde/blob/main/FAQ.md
- peter303 3y agoThe larger language models now employ a trillion parameters. This is faster when memory and computing is tighter, not distributed. Cerebus's million core super-wafer addresses this.
- ElectricalUnion 3y ago> I'm surprised there isn't some way for gamers to rent out use of their GPU's when idle. The main reason why you need massive ammounts of fast VRAM in the first place is that the main limitation of AI is memory bandwidth. Can't simply distribute an algorithm that is already throughput limited by memory bandwidth and distribute it with awful latency and bandwidth and hope for any improvement.
- EVa5I7bHFq9mnYK 3y agoIn bitcoin mining, GPU phase lasted only two years, before been outcompeted first by specialized FPGAs and then by ASICs. Nobody used GPUs for bitcoin mining since 2013. Maybe ML will follow similar path. But the computation is much different from ML, doesnt need memory at all.
- PeterStuer 3y agoAren't tensorcores basically asics for ml?
- EVa5I7bHFq9mnYK 3y agoif they have transistors for gaming, this is extra unneeded cost from the matrix multiplication point of view. If they have not, those are not GPUs anymore.
- magic_hamster 3y agoThe only way to compete with Nvidia is to supply a drop in replacement for CUDA with the same (or better) performance for price. Good luck with that. In the current state of things, Nvidia is like a car manufacturer that exclusively owns the concept of tires.