3 ms·
> Due to the large size of the model (314B parameters), a machine with enough GPU memory is required to test the model with the example code What type of machi
by gardenhedge 3y ago
> Due to the large size of the model (314B parameters), a machine with enough GPU memory is required to test the model with the example code
What type of machine do you need to play around with this?
- a_wild_dandan 3y agoA single 192GB M2 Mac using a 4-bit quant would work.
- 317070 3y agoProbably a machine with about 628 GB of GPU memory. (2 bytes per parameter) So 8xH100 (80Gb each) should do it.
- Marlinski 3y agoI suppose it can be quantizised
- anigbrowl 3y ago'Chunky beast, needs 320 Gb VRAM likely 4 bit, likely is being run 8 bit on 8 x 80 Gb GPUs.' -Emad