5 ms·

DeepSeek-VL2: MoE Vision-Language Models for Advanced Multimodal Understanding

36 points by selvan 2y ago

erulabs 2y ago
> Note: You may need 80GB GPU memory to run this script with deepseek-vl2-small and even larger for deepseek-vl2. Is this a typo? 80GB VRAM? I make great money, but there's no way I'm talking myself (or my wife) into spending 15k+ on an AI rig. I feel like I'm 16 again, pining after video cards I cannot afford or justify.
- oarsinsync 2y ago
  Definitely not a typo. Also: > # vl2-tiny, 3.37B-MoE in total, activated 1B, can be run on a single GPU < 40GB
- tucnak 2y ago
  How did you arrive at 15k? For personal use, it's common to get aftermarket 3090's, link them up four-way (96 GB is not half-bad) and put it in some server motherboard with as many DDR5 channels as possible... imho in 2025 this is a total waste of time, but it's possible, and at a fraction of your estimated cost... Seriously though, H100's go for $2/hr these days. I personally spot g6e.48xlarge (384 GB) which goes for under $1. Why bother hoarding many years out-of-date hardware which is like running three washing machines all day...
  - oarsinsync 2y ago
    > I personally spot g6e.48xlarge (384 GB) which goes for under $1. How? I see this around $30?!
    - tucnak 2y ago
      My mistake! I used to get 48xlarge in eu-central-1 for $2/hr but it could be luck for all I know, seems Spot pricing for these instances are fluctuating considerably: it's at $8/hr now. https://aws.amazon.com/ec2/spot/pricing/ https://aws.amazon.com/ec2/spot/pricing/
- runeblaze 2y ago
  That’s your H100 and A100’s GPU memory, not too sure but I imagine many AI practitioners would not find that a weird number
- smcleod 2y ago
  Absolutely not $15k, you can go and buy a 2 year old MacBook with 96GB of unified memory (around 85GB usable for the GPU) for a few thousand USD, or a brand new 128GB version for $4500 USD~, or grab 4x 24GB GPUs second hand etc... I personally have a 96GB MacBook Pro from 2023, and my home server has 80GB of vRAM (2x 3090, 2x A4000). It's not as uncommon as you might think if you work in tech.

36 points by selvan 2y ago

> Note: You may need 80GB GPU memory to run this script with deepseek-vl2-small and even larger for deepseek-vl2. Is this a typo? 80GB VRAM? I make great money, but there's no way I'm talking myself (or my wife) into spending 15k+ on an AI rig. I feel like I'm 16 again, pining after video cards I cannot afford or justify.

oarsinsync 2y ago
Definitely not a typo. Also: > # vl2-tiny, 3.37B-MoE in total, activated 1B, can be run on a single GPU < 40GB
tucnak 2y ago
How did you arrive at 15k? For personal use, it's common to get aftermarket 3090's, link them up four-way (96 GB is not half-bad) and put it in some server motherboard with as many DDR5 channels as possible... imho in 2025 this is a total waste of time, but it's possible, and at a fraction of your estimated cost... Seriously though, H100's go for $2/hr these days. I personally spot g6e.48xlarge (384 GB) which goes for under $1. Why bother hoarding many years out-of-date hardware which is like running three washing machines all day...
- oarsinsync 2y ago
  > I personally spot g6e.48xlarge (384 GB) which goes for under $1. How? I see this around $30?!
  - tucnak 2y ago
    My mistake! I used to get 48xlarge in eu-central-1 for $2/hr but it could be luck for all I know, seems Spot pricing for these instances are fluctuating considerably: it's at $8/hr now. https://aws.amazon.com/ec2/spot/pricing/ https://aws.amazon.com/ec2/spot/pricing/
runeblaze 2y ago
That’s your H100 and A100’s GPU memory, not too sure but I imagine many AI practitioners would not find that a weird number
smcleod 2y ago
Absolutely not $15k, you can go and buy a 2 year old MacBook with 96GB of unified memory (around 85GB usable for the GPU) for a few thousand USD, or a brand new 128GB version for $4500 USD~, or grab 4x 24GB GPUs second hand etc... I personally have a 96GB MacBook Pro from 2023, and my home server has 80GB of vRAM (2x 3090, 2x A4000). It's not as uncommon as you might think if you work in tech.