3 ms·

600 GB/s of memory bandwidth isn't anything to sneeze at. ~$1000 for the Pro B70, if Microcenter is to be believed: https://www.microcenter.com/product/709007

600 GB/s of memory bandwidth isn't anything to sneeze at. ~$1000 for the Pro B70, if Microcenter is to be believed: https://www.microcenter.com/product/709007/intel-arc-pro-b70-single-fan-32gb-gddr6-pcie-50-graphics-card https://www.microcenter.com/product/709007/intel-arc-pro-b70... https://www.microcenter.com/product/708790/asrock-intel-arc-pro-b70-creator-32gb-gddr6-pcie-50-graphics-card https://www.microcenter.com/product/708790/asrock-intel-arc-...

qingcharles 6mo ago
I think the B65 is priced at $650. Both supported by llamacpp I believe. With that power draw you could run two of them.
giancarlostoro 6mo ago
Intel GPU prices have stayed fine, but I do wonder if they are viable for Inference if they will wind up like Nvidia GPUs, severely overpriced.
hedgehog 6mo ago
Recent kernels have SR-IOV support for these chips too. B&H has them listed for $950. https://www.bhphotovideo.com/c/product/1959142-REG/intel_33p01ib0bb_arc_pro_b70_32gb.html https://www.bhphotovideo.com/c/product/1959142-REG/intel_33p... When 32GB NVIDIA cards seem to start at around $4000 that's a big enough gap to be motivating for a bunch of applications.
- robotnikman 6mo ago
  I'm probably going to snag one of the Intel cards just for the SR-IOV and use with VM's
  - hedgehog 6mo ago
    Depending what you're doing AMD's support for VirtIO Native Context might be a useful alternative (I think it gives less isolation which could be good or bad depending on use).
  - scrubs 6mo ago
    I tried to use SRIOV to virtualize mellanox nics with vlans on redhat Linux. Long story short it did not work. Per Nvidia the os has to also run open switch. This work was on an already complex setup in finance ... so adding open switch was considered too much additionally complexity. This requirement is not something I run across in the docs. Anybody know better?
    - hedgehog 6mo ago
      The situation in networking is a lot different than graphics. I don't know much other than that it depends on what specific protocol, card, firmware, and network topology you're using and there's not really generic advice. If the question is setting up Ethernet switching inside the card so VFs can talk to the network, then I think the Linux switchdev tools can configure that on their own without Open vSwitch but you probably need to find someone who understands your specific type of deployment for better advice.
jauntywundrkind 6mo ago
I tend to agree that the vram size and bandwidth is the core thing, but this B70 Pro allegedly has 387 int8 tops vs a 5090 having 3400 int8 tops. 600 compares vs 1792GB/s. I'm delighted so see an option with quarter the price! But man, a tenth the performance? https://www.techpowerup.com/347721/sparkle-announces-intel-arc-pro-b70-and-intel-arc-pro-b65-series-graphics-cards https://www.techpowerup.com/347721/sparkle-announces-intel-a... https://www.tomshardware.com/pc-components/gpus/nvidia-announces-rtx-50-series-at-up-to-usd1-999 https://www.tomshardware.com/pc-components/gpus/nvidia-annou...
- wmf 6mo ago
  It's more like a 70 class card with extra VRAM.
- adgjlsfhk1 6mo ago
  especially for exploratory work 1/10th the perf is fine. Intel isn't able to compete head to head with Nvidia (yet), but vram is capability while speed is capacity. There will be plenty of use cases where the value prop here makes sense.
- ColonelPhantom 6mo ago
  838 seems to be the real INT8 TOPS number for the 5090; going from 800 to 3400 takes an x2 speedup for sparsity (so skipping ops) and another x2 speedup for FP4 over INT8. So it's closer to half the speed than a tenth. Intel also seems to be positioning this card against the RTX PRO 4000 Blackwell, not the 5090, and that one gets more like 300 INT8 TOPS. It also has less memory but at a slightly higher bandwidth. The 5090 is much faster and IIRC priced similarly to the PRO 4000, but is also decidedly a consumer product which, especially for Nvidia, comes with limitations (e.g. no server-friendly form factor cards available, and there are or used to be driver license restrictions that prevented using a consumer card in a data center setup).
  - jauntywundrkind 6mo ago
    Thank you for the correction. That seemed way too lopsided to be believed. This assessment balances the memory to tops ratio much much more evenly, which is to be expected! I was low key hoping someone would help me make sense of how wildly disparate figures were, but I wasn't seeing. AMD R9700 is 378/766 tops int8 dense/sparse. 644GB/s of 32GB memory. ~$1400. To throw one more card into the mix. Intel undercutting that nicely here. You're right that for companies, the pro grade matters. For us mere mortals, much less so. Features like sr-iov however are just fantastic so see! Good job Intel. AMD has been trickling out such capabilities for a decade (cards fused for "MxGPU" capability) & it makes it such an easier buy to just offer it straight up across the models.
cmovq 6mo ago
I mean it kind of is considering that's comparable to a 5070 which has 672 GB/s? Benefit of NVIDIA being the only one using GDDR7 for now I guess.
- daemonologist 6mo ago
  7800 XT has 624 GB/s as well, and can be found for $400 used. 16 GB of course.
  - BizarroLand 6mo ago
    I've heard ROCm is still a crapshoot though. Is that true?
    - daemonologist 6mo ago
      If you stick with your OS/package manager-distributed version, installation isn't painful anymore (provided that version approximately overlaps with your generation of GPU). It's okay for inference, and okay for training if you don't stray too far beyond plain torch. If you want to run code from a paper or other more esoteric stuff you're still going to have a bad time. I don't have an Intel dGPU, but I suspect the situation there is even worse. I mean you go to the torch homepage: https://pytorch.org/get-started/locally/ https://pytorch.org/get-started/locally/ and Intel isn't even mentioned. (It's here though: https://docs.pytorch.org/docs/stable/notes/get_start_xpu.html https://docs.pytorch.org/docs/stable/notes/get_start_xpu.htm...)
varispeed 6mo ago
The product would be excellent in 2024, but now it's a landfill filler. You can run some small models at pedestrian speed, novelty wears off and that's it. Intel is not looking in the future. If they released Arc Pro B70 with 512GB base RAM, now that could be interesting. 32GB? Meh.
- throwaway85825 6mo ago
  It's true that it's severely late and missed it's market window but 512gb just isn't possible.

qingcharles 6mo ago

I think the B65 is priced at $650. Both supported by llamacpp I believe. With that power draw you could run two of them.

giancarlostoro 6mo ago

Intel GPU prices have stayed fine, but I do wonder if they are viable for Inference if they will wind up like Nvidia GPUs, severely overpriced.

hedgehog 6mo ago

Recent kernels have SR-IOV support for these chips too. B&H has them listed for $950. https://www.bhphotovideo.com/c/product/1959142-REG/intel_33p01ib0bb_arc_pro_b70_32gb.html https://www.bhphotovideo.com/c/product/1959142-REG/intel_33p... When 32GB NVIDIA cards seem to start at around $4000 that's a big enough gap to be motivating for a bunch of applications.

robotnikman 6mo ago
I'm probably going to snag one of the Intel cards just for the SR-IOV and use with VM's
- hedgehog 6mo ago
  Depending what you're doing AMD's support for VirtIO Native Context might be a useful alternative (I think it gives less isolation which could be good or bad depending on use).
- scrubs 6mo ago
  I tried to use SRIOV to virtualize mellanox nics with vlans on redhat Linux. Long story short it did not work. Per Nvidia the os has to also run open switch. This work was on an already complex setup in finance ... so adding open switch was considered too much additionally complexity. This requirement is not something I run across in the docs. Anybody know better?
  - hedgehog 6mo ago
    The situation in networking is a lot different than graphics. I don't know much other than that it depends on what specific protocol, card, firmware, and network topology you're using and there's not really generic advice. If the question is setting up Ethernet switching inside the card so VFs can talk to the network, then I think the Linux switchdev tools can configure that on their own without Open vSwitch but you probably need to find someone who understands your specific type of deployment for better advice.

jauntywundrkind 6mo ago

I tend to agree that the vram size and bandwidth is the core thing, but this B70 Pro allegedly has 387 int8 tops vs a 5090 having 3400 int8 tops. 600 compares vs 1792GB/s. I'm delighted so see an option with quarter the price! But man, a tenth the performance? https://www.techpowerup.com/347721/sparkle-announces-intel-arc-pro-b70-and-intel-arc-pro-b65-series-graphics-cards https://www.techpowerup.com/347721/sparkle-announces-intel-a... https://www.tomshardware.com/pc-components/gpus/nvidia-announces-rtx-50-series-at-up-to-usd1-999 https://www.tomshardware.com/pc-components/gpus/nvidia-annou...

wmf 6mo ago
It's more like a 70 class card with extra VRAM.
adgjlsfhk1 6mo ago
especially for exploratory work 1/10th the perf is fine. Intel isn't able to compete head to head with Nvidia (yet), but vram is capability while speed is capacity. There will be plenty of use cases where the value prop here makes sense.
ColonelPhantom 6mo ago
838 seems to be the real INT8 TOPS number for the 5090; going from 800 to 3400 takes an x2 speedup for sparsity (so skipping ops) and another x2 speedup for FP4 over INT8. So it's closer to half the speed than a tenth. Intel also seems to be positioning this card against the RTX PRO 4000 Blackwell, not the 5090, and that one gets more like 300 INT8 TOPS. It also has less memory but at a slightly higher bandwidth. The 5090 is much faster and IIRC priced similarly to the PRO 4000, but is also decidedly a consumer product which, especially for Nvidia, comes with limitations (e.g. no server-friendly form factor cards available, and there are or used to be driver license restrictions that prevented using a consumer card in a data center setup).
- jauntywundrkind 6mo ago
  Thank you for the correction. That seemed way too lopsided to be believed. This assessment balances the memory to tops ratio much much more evenly, which is to be expected! I was low key hoping someone would help me make sense of how wildly disparate figures were, but I wasn't seeing. AMD R9700 is 378/766 tops int8 dense/sparse. 644GB/s of 32GB memory. ~$1400. To throw one more card into the mix. Intel undercutting that nicely here. You're right that for companies, the pro grade matters. For us mere mortals, much less so. Features like sr-iov however are just fantastic so see! Good job Intel. AMD has been trickling out such capabilities for a decade (cards fused for "MxGPU" capability) & it makes it such an easier buy to just offer it straight up across the models.

cmovq 6mo ago

I mean it kind of is considering that's comparable to a 5070 which has 672 GB/s? Benefit of NVIDIA being the only one using GDDR7 for now I guess.

daemonologist 6mo ago
7800 XT has 624 GB/s as well, and can be found for $400 used. 16 GB of course.
- BizarroLand 6mo ago
  I've heard ROCm is still a crapshoot though. Is that true?
  - daemonologist 6mo ago
    If you stick with your OS/package manager-distributed version, installation isn't painful anymore (provided that version approximately overlaps with your generation of GPU). It's okay for inference, and okay for training if you don't stray too far beyond plain torch. If you want to run code from a paper or other more esoteric stuff you're still going to have a bad time. I don't have an Intel dGPU, but I suspect the situation there is even worse. I mean you go to the torch homepage: https://pytorch.org/get-started/locally/ https://pytorch.org/get-started/locally/ and Intel isn't even mentioned. (It's here though: https://docs.pytorch.org/docs/stable/notes/get_start_xpu.html https://docs.pytorch.org/docs/stable/notes/get_start_xpu.htm...)

varispeed 6mo ago

The product would be excellent in 2024, but now it's a landfill filler. You can run some small models at pedestrian speed, novelty wears off and that's it. Intel is not looking in the future. If they released Arc Pro B70 with 512GB base RAM, now that could be interesting. 32GB? Meh.

throwaway85825 6mo ago
It's true that it's severely late and missed it's market window but 512gb just isn't possible.