6 ms·
12GB max is a non-starter for ML work now. Why not come out with a reasonably priced 24gb card even if it isn't the fastest and target it at the ML dev world? A
by jmward01 2y ago
12GB max is a non-starter for ML work now. Why not come out with a reasonably priced 24gb card even if it isn't the fastest and target it at the ML dev world? Am I missing something here?
- Implicated 2y agoI was wondering the same thing. Seems crazy to keep pumping out 12gb cards in 2025.
- shrewduser 2y agothese are the entry level cards, i imagine the coming higher end variants will have the option of much more ram.
- deleted 2y ago[deleted]
- tofuziggy 2y agoYes exactly!!
- enragedcacti 2y ago> Am I missing something here? Video games
- rs_rs_rs_rs_rs 2y agoIt's insane how out of touch people can be here, lol
- heraldgeezer 2y agoI have been trying to hold my slurs in reading this thread. These ML AI Macbook people are legit insane. Desktops and gaming is ugly and complex to them (because lego is hard and macbook look nice unga bunga), yet it is a mass market Intel wants to move in on. People here complain because Intel is not making a cheap GPU to "make AI" on when that's a market of maybe 1000 people. This Intel card is perfect for an esports gaming machine running CS2, Valorant, Rocket Leauge and casual or older games like The Sims, GoG games etc. Market of 1 million + right there, CS2 alone is 1mil people playing everyday. Not people grinding leetcode on their macs. Every real developer has a desktop, epyc cpu, giga ram and a nice GPU for downtime and run a real OS like Linux or even Windows (yes majority of devs run Windows)
- throwaway48476 2y agoIntel GPUs don't sell well to gamers. They've been on the market for years now. >market of maybe 1000 people The market of people interested in local ai inference is in the millions. If it's cheap enough the data center market is at least 10 million.
- heraldgeezer 2y agoYes, Intel cards have sucked. But they are trying again!
- kimixa 2y agoYes, everyone knows the best way of making a product not suck is to give up and never try again. /s
- ac29 2y ago> Intel GPUs don't sell well to gamers. They've been on the market for years now. Intel has only had discrete GPUs on the market for 2 years. I guess that is a plural number of years, but only barely.
- terhechte 2y agoMost devs use windows (https://www.statista.com/statistics/869211/worldwide-software-development-operating-system/?utm_source=chatgpt.com https://www.statista.com/statistics/869211/worldwide-softwar...). Reddit llocallama alone has 250k users. Clearly the market is bigger than 1000 people. Why are gamers and Linux people always so aggressive diminutive of other people’s interests?
- heraldgeezer 2y ago>Why are gamers and Linux people always so aggressive diminutive of other people’s interests? Both groups have a high autism % We love to be "technically correct" and we often are. So we get frustrated when people claim things that are wrong.
- jmward01 2y agoHow big is NVIDIA now? You don't think breaking into that market is a good strategy? And, yes, I understand that this is targeted at gamers and not ML. That was the point of the comment I made. Maybe if they did target ML they would make money and open a path to the massive server market out there.
- bjoli 2y agoA video card that beats the 4060 for under $250 is very much going to be a problem for AMD and is going to eat the "low end" market if it is reasonably stable.
- bryanlarsen 2y agoThese are $200 low end cards, the B5X0 cards. Presumably they have B7X0 and perhaps even B9X0 cards in the pipeline as well.
- deleted 2y ago[deleted]
- zamadatix 2y agoThere has been no hint or evidence (beyond hope) Intel will add a 900 class this generation. B770 was rumoured to match the 16 GB of the A770 (and to be the top end offering for Battlemage) but it is said to not have even been taped out yet with rumour it may end up having been cancelled completely. I.e. don't hold your breath for anything consumer from Intel this generation better for AI than tha A770 you could have bought 2 years ago. Even if something slightly better is coming at all there is no hint it will be soon.
- hulitu 2y ago> These are $200 low end cards Hm, i wouldn't consider 200$ low end.
- Narishma 2y agoIt is when the high end is 1000$ or more.
- amaranth 2y agoThere isn't a cheaper card that's worth buying over using the iGPU you already have so yeah, that's the low end.
- wtallis 2y agoNVIDIA and AMD don't even make new GPU silicon this low-end. Their smallest current-gen GPUs all debuted at higher price points, though the Radeon RX 7600 is now available at the same price that the B580 is launching at.
- 2y ago
- dgfitz 2y agoML is about hit another winter. Maybe intel is ahead of industry. Or we can keep asking high computers questions about programming.
- PittleyDunkin 2y ago> ML is about hit another winter. I agree ML is about to hit (or has likely already hit) some serious constraints compared to breathless predictions of two years ago. I don't think there's anything equivalent to the AI winter on the horizon, though—LLMs even operated by people who have no clue how the underlying mechanism functions are still far more empowered than anything like the primitives of the 80s enabled.
- klodolph 2y agoYeah… I want to think of it like mining, where you’ve found an ore vein. You have to switch from prospecting to mining. There’s a lot of work to be done by integrating our LLMs and other tools with other systems, and I think the cost/benefit of making models bigger, Bigger, BIGGER is reaching a plateau.
- kimixa 2y agoI think there'll be a "financial" winter - or another way a bubble burst - the investment right now is simply unsustainable, how are these products going to be monetized? Nvidia had a revenue of $27billion in 2023 - that's about $160 per person per year [0] for every working age person in the USA. And it's predicted to more than double in 2024. If you reduce that to office workers (you know, the people who might actually get some benefit, as no AI is going to milk a cow or serve you starbucks) that's more like $1450/year. Or again more than double that for 2024. How much value add is the current set of AI products going to give us? It's still mostly promise too. Sure, like most bubbles there'll probably still be some winners, but there's no way the current market as a whole is sustainable. The only way the "maximal AI" dream income is actually going to happen is if they functionally replace a significant proportion of the working population completely. And that probably would have large enough impacts to society that things like "Dollars In A Bank" or similar may not be so important. [0] Using the stat of "169.8 million people worked at some point in 2022" https://www.bls.gov/news.release/pdf/work.pdf https://www.bls.gov/news.release/pdf/work.pdf [1] 18.5 million office workers according to https://www.bls.gov/news.release/ocwage.nr0.htm https://www.bls.gov/news.release/ocwage.nr0.htm
- layer8 2y agoThe ML dev world isn’t a consumer mass market like PC gaming is.
- hajile 2y agoLaunching a new SKU for $500-1000 with 48gb of RAM seems like a profitable idea. The GPU isn't top-of-the-line, but the RAM would be unmatched for running a lot of models locally.
- layer8 2y agoYou can’t just throw in more RAM without having the rest of the GPU architected for it. So there’s an R&D cost involved for such a design, and there may even be trade-offs on performance for the mass-market lower-tier models. I’m doubtful that the LLM enthusiast/tinkerer market is large enough for that to be obviously profitable.
- hajile 2y agoThat would depend on how they designed the memory controllers. GDDR6 only supporting 1-2gb modules at present (I believe GDDR6W supports 4gb modules). If they were using 12 1gb modules, then increasing to 24gb shouldn't be a very large change. Honestly, Apple seems to be on the right track here. DDR5 is slower than GDDR6, but you can scale the amount of RAM far higher simply by swapping out the density.
- KeplerBoy 2y agoIt's a 192 bit interface, so 6 16gbit chips.
- KeplerBoy 2y agoOf course you can just add more RAM. Double the capacity of every chip and you get twice the RAM without ever asking an engineer. People did it with the RTX3070. https://www.tomshardware.com/news/3070-16gb-mod https://www.tomshardware.com/news/3070-16gb-mod
- ggregoire 2y ago> 12GB max is a non-starter for ML work now. Can you even do ML work with a GPU not compatible with CUDA? (genuine question) A quick search showed me the equivalence to CUDA in the Intel world is oneAPI, but in practice, are the major Python libraries used for ML compatible with oneAPI? (Was also gonna ask if oneAPI can run inside Docker but apparently it does [1]) [1] https://hub.docker.com/r/intel/oneapi https://hub.docker.com/r/intel/oneapi
- suprjami 2y agoThere is ROCm and Vulkan compute. Vulkan is especially appealing because you don't need any special GPGPU drivers and it runs on any card which supports Vulkan.
- pineapple_sauce 2y agohttps://github.com/intel/intel-extension-for-pytorch https://github.com/intel/intel-extension-for-pytorch
- PhasmaFelis 2y ago> Am I missing something here? This is a graphics card.
- davrosthedalek 2y agoSir, this is a Wendy's
- heraldgeezer 2y agoThis is not an ML card... this is a gaming card... Why are you people like this?
- whalesalad 2y agoI still don't understand why graphics cards haven't evolved to include sodimm slots so that the vram can be upgraded by the end user. At this point memory requirements vary so much from gamer to scientist so it would make more sense to offer compute packages with user-supplied memory. tl;dr GPU's need to transition from being add-in cards to being a sibling motherboard. A sisterboard? Not a daughter board.
- heraldgeezer 2y agoGDDR does not exist in sodimm form factor. Intel and AMD internal GPUs can use normal computer RAM. But they are slower for that reason and many others.
- kimixa 2y agoOne of the reasons GPUs can have multiples of CPU bandwidth is they avoid the difficulties of pluggable dimms - direct soldered can have much higher frequencies at lower power. It's one of the reasons why ARM Macbooks get great performance/watt, memory being even "closer" than mainboard soldered RAM so getting more of those benefits, though naturally less flexibility.
- MindSpunk 2y agoEven DDR5 has this problem. Go look at what soldered DDR5 can do frequency wise compared to DIMMs. It's one of the problems the new CAMM form factor aims to help solve, making it tractable to push the memory frequency up beyond what DIMMs can get yout currently.
- fngjdflmdflg 2y agoI think a lot of replies to this post are missing that Intel's last graphics card wasn't received well by gamers due to poor drivers. The GT 730 from 2014 has more users than all Arc cards combined according to the latest Steam survey.[0] It's entirely possible that making a 24gb local inference card would do better since they can contribute patches for inference libraries directly like they did for llama.cpp, as opposed to a gaming card where the support surface is much larger. I wish Intel well in any case and hope their drivers (or driver emulators) improve enough to be considered broadly usable. [0] https://store.steampowered.com/hwsurvey/videocard/ https://store.steampowered.com/hwsurvey/videocard/ - 0.19% share
- mort96 2y agoWho cares?