4 ms·

Wow sorry y'all, we didn't expect this to take off so quickly. Working on getting this scaled up now so everyone can play!

aaronrelph 4y ago
And we're back! Might be little slow while traffic is high but responses are coming through.
- hmate9 4y ago
  Heads up: The UI is not usable on iOS safari if the URL bar is set to be on the bottom. The bar covers up the text area and send button and if you scroll down it jumps back up to be hidden again.
  - deleted 4y ago
    [deleted]
- deleted 4y ago
  [deleted]
numlocked 4y ago
Did you run the fine-tuning on LLaMA yourselves based on the 52k examples from Alpaca? Or is there a 7B pre-trained alpaca model out there that you grabbed?
- deleted 4y ago
  [deleted]
- aqader 4y ago
  For this demo, we're using the 8bit version here: https://huggingface.co/tloen/alpaca-lora-7b https://huggingface.co/tloen/alpaca-lora-7b We also fine-tuned and OSS'd a 30b version here that you can checkout (on the cleaned 52k Alpaca dataset) https://huggingface.co/baseten/alpaca-30b https://huggingface.co/baseten/alpaca-30b
  - underlines 4y ago
    did you use the cleaned and improved alpaca dataset from https://github.com/tloen/alpaca-lora/issues/28 https://github.com/tloen/alpaca-lora/issues/28 ?
    - aqader 4y ago
      Yes, we did! The dataset has since been cleaned even more so we're due to update the model.
  - UncleOxidant 4y ago
    Can you comment on the '8bit version' from above? Does that mean these parameters are uint8's (converted from the original float16 params)? Looking in your pytorch code I see some float16 declarations. I've been running alpaca.cpp 13b locally and your 7b model performs much better than it does. I had assumed this was because alpaca.cpp was converting weights to 4bits from float16, but is there some other fine tuning you're doing that might also account for the better performance of chatLLaMA over alpaca.cpp?
aaronrelph 4y ago
Alright, after a few hiccups, you should be seeing a noticeable improvement in response times now!

aaronrelph 4y ago

And we're back! Might be little slow while traffic is high but responses are coming through.

hmate9 4y ago
Heads up: The UI is not usable on iOS safari if the URL bar is set to be on the bottom. The bar covers up the text area and send button and if you scroll down it jumps back up to be hidden again.
- deleted 4y ago
  [deleted]
deleted 4y ago
[deleted]

numlocked 4y ago

Did you run the fine-tuning on LLaMA yourselves based on the 52k examples from Alpaca? Or is there a 7B pre-trained alpaca model out there that you grabbed?

deleted 4y ago
[deleted]
aqader 4y ago
For this demo, we're using the 8bit version here: https://huggingface.co/tloen/alpaca-lora-7b https://huggingface.co/tloen/alpaca-lora-7b We also fine-tuned and OSS'd a 30b version here that you can checkout (on the cleaned 52k Alpaca dataset) https://huggingface.co/baseten/alpaca-30b https://huggingface.co/baseten/alpaca-30b
- underlines 4y ago
  did you use the cleaned and improved alpaca dataset from https://github.com/tloen/alpaca-lora/issues/28 https://github.com/tloen/alpaca-lora/issues/28 ?
  - aqader 4y ago
    Yes, we did! The dataset has since been cleaned even more so we're due to update the model.
- UncleOxidant 4y ago
  Can you comment on the '8bit version' from above? Does that mean these parameters are uint8's (converted from the original float16 params)? Looking in your pytorch code I see some float16 declarations. I've been running alpaca.cpp 13b locally and your 7b model performs much better than it does. I had assumed this was because alpaca.cpp was converting weights to 4bits from float16, but is there some other fine tuning you're doing that might also account for the better performance of chatLLaMA over alpaca.cpp?

aaronrelph 4y ago

Alright, after a few hiccups, you should be seeing a noticeable improvement in response times now!