3 ms·
Since the Xeon alone could run the model in this set up it'd be more interesting if they compared the performance uplift with using 0/1/2..8 Arc A770 GPUs. Als
by zamadatix 2y ago
Since the Xeon alone could run the model in this set up it'd be more interesting if they compared the performance uplift with using 0/1/2..8 Arc A770 GPUs.
Also, it's probably better to link straight to the relevant section https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md#flashmoe-for-deepseek-v3r1 https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quic...
- colorant 2y agoYes, you are right. Unfortunately HN somehow truncated my original URL link.
- zamadatix 2y agoSounds like submission "helper" tools are working about as well as normal :). Did you have the chance to try this out yourself or did you just run across it recently?
- hmottestad 2y agoIf you’re running just one GPU your context is limited to 1024 tokens, as far as I could tell. I couldn’t see what the context size is for more cards though.