4 ms·

You're expecting too much from a cheap card. AMD offers many 16GB cards, and used RTX 3090s (24GB) are very affordable and are the go to for AI.

tigeroil 3y ago
What do you consider to be affordable? In the UK used RTX 3090s still go for well over 1k.
- LaurensBER 3y ago
  For a company that's very affordable, for an individual we're not there yet but I've seen some impressive demos running on a M1/M2 laptop so no doubt time will bring down the hardware requirements even more (and hopefully the price of used 3090s as well).
  - LoganDark 3y ago
    RWKV is also fairly performant on CPU. disclaimer: recently did a bunch of work on rwkv.cpp.
- skippyboxedhero 3y ago
  You're getting ripped off kid. I can get you one for £600.
  - theaussiestew 3y ago
    How?
    - topspin 3y ago
      Screw around on ebay, bidding for used cards from individual sellers, as opposed to 'top' sellers that get substantially more for their listings. They may or may not have been burning up the GPU for the last couple years overclocking. Eventually you can land a $600-ish 3090 [1]. Good luck. [1] https://www.ebay.com/itm/266275573529 https://www.ebay.com/itm/266275573529 (read the listing...)
      - latchkey 3y ago
        Overclocking doesn't burn up a GPU. https://www.reddit.com/r/pcmasterrace/comments/1vpljk/a_basic_crash_course_in_overclocking_facts/ https://www.reddit.com/r/pcmasterrace/comments/1vpljk/a_basi...
      - justinclift 3y ago
        That one's not a good example: > Condition: For parts or not working
        deleted 3y ago
        [deleted]
      - pipes 3y ago
        I got burnt on a second hand card a few years ago, reckon it had been used for crypto mining, weird checker board pattern would appear every so often. That said I still bought a 3070 for around 300 off FB market place recently. Was in the guys house, what I saw reassured me he used it for games.
    - skippyboxedhero 3y ago
      Just need to know where to look. These other replies are comical. You can get them for £600 with a two year warranty. Don't buy them on EBay.
- cypress66 3y ago
  That's insane. In Argentina I bought one for 500 dollars.
  - tomcam 3y ago
    Now $550… No wait, $582… Hang on, $622 (Gentle joke. Argentina has even worse inflation than the USA)
    - smcl 3y ago
      I think even there the value of things denominated in USD is a bit more stable, it's the local currency that is plummeting in value
    - cypress66 3y ago
      Nah, in USD it'll remain the same (I bought it with USD) Of course in ARS the price will change almost every week.
smcleod 3y ago
Used RTX cards (of any spec) are super expensive here in Australia still.
epolanski 3y ago
Cards in the same bracket offered the same amount of VRAM 8 years ago to be fair.
mstipetic 3y ago
I just got a 3090 for that purpose (used, 600 EUR) but have no idea where to start? Do you have a recommendation which LLM should I try running on it?

tigeroil 3y ago

What do you consider to be affordable? In the UK used RTX 3090s still go for well over 1k.

LaurensBER 3y ago
For a company that's very affordable, for an individual we're not there yet but I've seen some impressive demos running on a M1/M2 laptop so no doubt time will bring down the hardware requirements even more (and hopefully the price of used 3090s as well).
- LoganDark 3y ago
  RWKV is also fairly performant on CPU. disclaimer: recently did a bunch of work on rwkv.cpp.
skippyboxedhero 3y ago
You're getting ripped off kid. I can get you one for £600.
- theaussiestew 3y ago
  How?
  - topspin 3y ago
    Screw around on ebay, bidding for used cards from individual sellers, as opposed to 'top' sellers that get substantially more for their listings. They may or may not have been burning up the GPU for the last couple years overclocking. Eventually you can land a $600-ish 3090 [1]. Good luck. [1] https://www.ebay.com/itm/266275573529 https://www.ebay.com/itm/266275573529 (read the listing...)
    - latchkey 3y ago
      Overclocking doesn't burn up a GPU. https://www.reddit.com/r/pcmasterrace/comments/1vpljk/a_basic_crash_course_in_overclocking_facts/ https://www.reddit.com/r/pcmasterrace/comments/1vpljk/a_basi...
    - justinclift 3y ago
      That one's not a good example: > Condition: For parts or not working
      - deleted 3y ago
        [deleted]
    - pipes 3y ago
      I got burnt on a second hand card a few years ago, reckon it had been used for crypto mining, weird checker board pattern would appear every so often. That said I still bought a 3070 for around 300 off FB market place recently. Was in the guys house, what I saw reassured me he used it for games.
  - skippyboxedhero 3y ago
    Just need to know where to look. These other replies are comical. You can get them for £600 with a two year warranty. Don't buy them on EBay.
cypress66 3y ago
That's insane. In Argentina I bought one for 500 dollars.
- tomcam 3y ago
  Now $550… No wait, $582… Hang on, $622 (Gentle joke. Argentina has even worse inflation than the USA)
  - smcl 3y ago
    I think even there the value of things denominated in USD is a bit more stable, it's the local currency that is plummeting in value
  - cypress66 3y ago
    Nah, in USD it'll remain the same (I bought it with USD) Of course in ARS the price will change almost every week.

smcleod 3y ago

Used RTX cards (of any spec) are super expensive here in Australia still.

epolanski 3y ago

Cards in the same bracket offered the same amount of VRAM 8 years ago to be fair.

mstipetic 3y ago

I just got a 3090 for that purpose (used, 600 EUR) but have no idea where to start? Do you have a recommendation which LLM should I try running on it?