3 ms·
man I finally got a 3080 and. ow you're telling me I need a second?! haha seriously though, I'm kinda sad the model size is 2GB then straight to 13GB.
by LesZedCB 4y ago
man I finally got a 3080 and. ow you're telling me I need a second?!
haha seriously though, I'm kinda sad the model size is 2GB then straight to 13GB.
- desindol 4y agoyou can get it running with resizable bar
- Namidairo 4y agoIn theory the 2B model should be somewhere in between, whenever it gets fixed.
- moyix 4y agoAssuming that it's just that hardcoded check I should be able to add it today. But if that check is load-bearing (i.e. if it relies on those assumptions elsewhere in the code) it could be a bit more painful. Edit: sadly, it's not that easy. Removing that check lets the 2B model load, but it produces gibberish. I've opened an issue with FasterTransformer here, and will also try to debug it further myself, but unfortunately it's not obvious how they're using that assumption. https://github.com/NVIDIA/FasterTransformer/issues/268 https://github.com/NVIDIA/FasterTransformer/issues/268
- moyix 4y agoGot it working :) You can now use the 2B models in FauxPilot as well.
- amelius 4y agoYou probably don't need physical GPUs but you can rent them in some cloud for a few days.