Feorn

8 ms·

1. 1.▲
   by Feorn 3y ago

   A completely unquantized fp16 model weight 7B LLM is about 15GB on a disk. You need closer to 24GB of memory for inference with a decently sized context. Quantization is black magic of the software variety that seems to be able to significa
2. 2.▲
   by Feorn 3y ago

   Most brains don't treat themselves to an abundance of training data, the ones who do seem pretty knowledgeable to me. Witty comment aside, the human brain is pretty efficient in terms of energy use considering it's taking in a ton
3. 3.▲
   by Feorn 3y ago

   I think ROM sites were an artifact of the internet speeds we had access to early on. Where it wasn't practical to just download an entire library for an older console. Downloading an N64 ROM over dialup still took a little longer than
4. 4.▲
   by Feorn 3y ago

   A reference 3090 is longer by 69mm, wider by 29mm, and thicker by a slot than a PCIe A100. Though I think the comment you're replying to was talking about them both using the same Nvidia GPU architecture, Ampere.