5 ms·

I have not found alpaca to be comparable to chatgpt, but it could be because of bugs in the version I installed through dalai. I might try reinstalling it becau

I have not found alpaca to be comparable to chatgpt, but it could be because of bugs in the version I installed through dalai. I might try reinstalling it because I suspect there might be some sort of file corruption issue or whatever. I gave it the prompt “cats aren’t always fuzzy” and it wrote a lengthy livejournal-esque rambling journal entry about a woman and her husband having money issues. It was funny, but lightyears away from chatgpt. It does sometimes create some really funny hallucinations though, like inventing prefectures in Japan that don’t exist etc.

DustinBrett 4y ago
I also got that text about the married couple and their money issues. Alpaca didn't impress me at all so far.
- disgruntledphd2 4y ago
  Alpaca wasn't great. The 13b and 30b models are much better, but just for sentence completion. Personally, I think that the RLHF does make a big difference but maybe it's a bug in the quantization code as suggested up thread.
  - braingenious 4y ago
    I’m also a bit confused by the quantization thing. Why exactly is everybody running the same program on the same file? Why not just include the quantized weights? It seems like if somebody figured out the “correct” way to quantize the 7b weights it would make way more sense to just torrent the output rather than distribute a fixed program.
    - homarp 4y ago
      quantization takes lots of RAM: https://github.com/qwopqwop200/GPTQ-for-LLaMa/blob/main/README.md https://github.com/qwopqwop200/GPTQ-for-LLaMa/blob/main/READ... says llama-13B takes 42GB and 33B takes more than 64GB...
      - disgruntledphd2 4y ago
        I quantised all of the models(though the 65Bn one came out weird) and I only have 64Gb of RAM, so dunno.
    - sebzim4500 4y ago
      There is no reason not to do that, except i) Distributing large files through torrents is slightly annoying if you don't already happen to have a seedbox ii) People are still messing around with quantization settings, they might think that they are a few days away from a much better version iii) No one wants to be sued by Meta. I think the risk is pretty small but not zero.

DustinBrett 4y ago

I also got that text about the married couple and their money issues. Alpaca didn't impress me at all so far.

disgruntledphd2 4y ago
Alpaca wasn't great. The 13b and 30b models are much better, but just for sentence completion. Personally, I think that the RLHF does make a big difference but maybe it's a bug in the quantization code as suggested up thread.
- braingenious 4y ago
  I’m also a bit confused by the quantization thing. Why exactly is everybody running the same program on the same file? Why not just include the quantized weights? It seems like if somebody figured out the “correct” way to quantize the 7b weights it would make way more sense to just torrent the output rather than distribute a fixed program.
  - homarp 4y ago
    quantization takes lots of RAM: https://github.com/qwopqwop200/GPTQ-for-LLaMa/blob/main/README.md https://github.com/qwopqwop200/GPTQ-for-LLaMa/blob/main/READ... says llama-13B takes 42GB and 33B takes more than 64GB...
    - disgruntledphd2 4y ago
      I quantised all of the models(though the 65Bn one came out weird) and I only have 64Gb of RAM, so dunno.
  - sebzim4500 4y ago
    There is no reason not to do that, except i) Distributing large files through torrents is slightly annoying if you don't already happen to have a seedbox ii) People are still messing around with quantization settings, they might think that they are a few days away from a much better version iii) No one wants to be sued by Meta. I think the risk is pretty small but not zero.