6 ms·

I am a Vicuna developer. We plan to release the weights once we have addressed all concerns and have a low-resource version of the inference code ready. We rele

I am a Vicuna developer. We plan to release the weights once we have addressed all concerns and have a low-resource version of the inference code ready. We released the demo first to get some early feedback on the model.

techdragon 4y ago
We hear a lot about "concerns" and many of us don't share the same ones... It would be good for clarity to know what are the concerns you feel are important enough to hold back releasing the weights?
- number6 4y ago
  Financial and Political would be my guess. But maybe I just want to tease out an answer...
- zhwu 4y ago
  It is mainly because of the legal issues caused by the license of llama model weights. We need to figure it out with Meta's llama team before releasing.
  - jart 4y ago
    Hi. I wrote the weights file format that llama.cpp uses, as of yesterday https://github.com/ggerganov/llama.cpp/pull/613 https://github.com/ggerganov/llama.cpp/pull/613 What can I do to assist you getting these deltas ready?
imjonse 4y ago
Has LoRA been considered as possible alternative for finetuning on your dataset? In that case releasing the 'diff' against the LLaMA weights would be simpler to work with.
- zhisbug 4y ago
  Yeah that might work but this model wasn’t tuned with lora
nenkoru 4y ago
It would be great if you can help me with this PR as well as adding a support for exporting a model that was quantized using GPTQ, bitsandbytes, plain torch. This would bring a lot of benefit from both worlds: - Low memory footprint(thanks quantization) - Fast inference(thanks io binding) Particularly in case of alpaca I have seen a 5x decrease in latency on A100 and 10x on AMD EPYC. I believe this is the way for users to have an AI that could genereate a response as fast as it can on their hardware. I have also added a link to my profile on hf with small alpacas turned into ONNX format. Take a look into them. [1] https://github.com/huggingface/optimum/pull/922 https://github.com/huggingface/optimum/pull/922 [2] https://huggingface.co/nenkoru https://huggingface.co/nenkoru
luckystarr 4y ago
If it's based on LLaMA, aren't these weights just some sort of "patch" for the initial model, which is licensed under a restrictive license? Or is this work "transferable" to other LLMs, once they become available?
shallichange 4y ago
Why is it not called Vicuña as it should? Vicuna does not sound the same way

techdragon 4y ago

We hear a lot about "concerns" and many of us don't share the same ones... It would be good for clarity to know what are the concerns you feel are important enough to hold back releasing the weights?

number6 4y ago
Financial and Political would be my guess. But maybe I just want to tease out an answer...
zhwu 4y ago
It is mainly because of the legal issues caused by the license of llama model weights. We need to figure it out with Meta's llama team before releasing.
- jart 4y ago
  Hi. I wrote the weights file format that llama.cpp uses, as of yesterday https://github.com/ggerganov/llama.cpp/pull/613 https://github.com/ggerganov/llama.cpp/pull/613 What can I do to assist you getting these deltas ready?

imjonse 4y ago

Has LoRA been considered as possible alternative for finetuning on your dataset? In that case releasing the 'diff' against the LLaMA weights would be simpler to work with.

zhisbug 4y ago
Yeah that might work but this model wasn’t tuned with lora

nenkoru 4y ago

It would be great if you can help me with this PR as well as adding a support for exporting a model that was quantized using GPTQ, bitsandbytes, plain torch. This would bring a lot of benefit from both worlds: - Low memory footprint(thanks quantization) - Fast inference(thanks io binding) Particularly in case of alpaca I have seen a 5x decrease in latency on A100 and 10x on AMD EPYC. I believe this is the way for users to have an AI that could genereate a response as fast as it can on their hardware. I have also added a link to my profile on hf with small alpacas turned into ONNX format. Take a look into them. [1] https://github.com/huggingface/optimum/pull/922 https://github.com/huggingface/optimum/pull/922 [2] https://huggingface.co/nenkoru https://huggingface.co/nenkoru

luckystarr 4y ago

If it's based on LLaMA, aren't these weights just some sort of "patch" for the initial model, which is licensed under a restrictive license? Or is this work "transferable" to other LLMs, once they become available?

shallichange 4y ago

Why is it not called Vicuña as it should? Vicuna does not sound the same way