5 ms·

Just to make it clear, does this torrent include model weights?

• generalizations 4y ago
  It contains weights for all four model sizes, apparently. This definitely saves on bandwidth costs. :)
• WithinReason 4y ago
  Folder structure for the 2 smaller models look like this: LLAMA │ tokenizer.model │ tokenizer_checklist.chk │ ├───13B │ checklist.chk │ consolidated.00.pth │ consolidated.01.pth │ params.json │ └───7B checklist.chk consolidated.00.pth params.json
  • HarHarVeryFunny 4y ago
    So what is content of those various files? Does this include the full models themselves, or just the weights ?
    • WithinReason 4y ago
      The pth file seems to be a model and weights, saved as described here: https://pytorch.org/tutorials/beginner/saving_loading_models.html#save-load-entire-model https://pytorch.org/tutorials/beginner/saving_loading_models... .chk file is am md5 hash of the file, the .json file contains this for the 7B model: {"dim": 4096, "multiple_of": 256, "n_heads": 32, "n_layers": 32, "norm_eps": 1e-06, "vocab_size": -1}
      • HarHarVeryFunny 4y ago
        Thanks, so from that PyTorch doc it seems that pickle format has the filenames of the model classes, but not the classes themselves. I'm sure someone will figure it out though!
        • WithinReason 4y ago
          The code (including the model) is here: https://github.com/facebookresearch/llama https://github.com/facebookresearch/llama I already got the 7B model to generate text using my GPU! The 1st example prompt generated this: [I believe the meaning of life is] to be happy, and it is also to live in the moment. I think that is the most important thing. I'm not really a party girl. I'm not a girl's girl. I have a really small group of close girlfriends and that's all I need. I believe in equal rights for everyone. I'm not a rebel. I don't really rebel against anything. I'm a very traditional girl, very loyal. I'm a mum's girl and I'm a dad's girl. People have a right to know what's going on. I don't care about the haters, because at the end of the day they're just going to have to deal with themselves. I've been getting more and more into fashion since I was about 16. I know I'm a little different, but so what? I think that's good. I don't think you should be like everyone else. It's my birthday, and I'll cry if I want to. I've always been a huge fan of fashion, and I've always liked to dress up
          • fareesh 4y ago
            What GPU is it and how long did it take?
            • WithinReason 4y ago
              4090, about 10 seconds to load the weights and another 15 seconds to generate all the completions from the example script
              • fareesh 4y ago
                Yikes, thanks I have a 2060 and I am too afraid and poor to buy a 4090 after import duties and taxes in a tropical country
                • WithinReason 4y ago
                  I could drop the batch size to 5, then the VRAM use seemed to be around 15GB. Some of that I'm sure is not necessary, and if you rewrite the outer products to use less VRAM you might get away with even less. Eventually someone will make a library so you can run it without extra work.