4 ms·
Just today I got Stanford's Alpaca-7b model running locally on my m1 mac, it’s just facebook’s Lamma-7b model which has been trained to complete tasks. It's get
by alden5 4y ago
Just today I got Stanford's Alpaca-7b model running locally on my m1 mac, it’s just facebook’s Lamma-7b model which has been trained to complete tasks. It's getting close to the versatility of chatgpt where I could actually use it for everyday tasks. I don't think open source is that far away, especially considering how quickly Alpaca came out and how much better it is vs Lamma, which frequently would hallucinate and often didn't make sense.
Example of prompt to Lamma-7b:
> write a poem about open ai not being open source
Open AI is not really “open”
As this project isn’t open sore
It can be seen as closed fortress,
Inside which secrets are hidden.
Not mind blowing but still really interesting, I will note that its much better at things like answering trivia where's there's already lots of examples in its model
- blueorange8 4y agoWere you able to integrate any of your data into it yet ?
- alden5 4y agoI wouldn't be able to retrain the model as my computer isn't capable enough, but I can change the prompt to change how the model acts. The prompt i'm currently using is: "Below is an instruction that describes a task. Write a response that appropritely completes the request." That base prompt can be customized to complete specific tasks like classifying text or acting like an assistant.
- omega3 4y agoYou have a typo in the prompt: appropriately. I wonder if it makes any difference to the output.
- b800h 4y agoOut of interest - how capable a computer is required to retrain that model?
- alden5 4y agothe project for training i found required a nvidia gpu with 16gb of vram, it also would take about 6 hours
- cfn 4y agoCan you link to instructions specific to the Mac. I can only find instructions for Alpaca with GPUs and PCs.
- alden5 4y agothe lamma.cpp project on github has instructions for alpaca. id recommend not using the alpaca download given and finding the updated torrent in issue #324 as the download didnt work for me
- rvnx 4y agoLLaMA-65B (8-bit) answer (a bit out-of-topic answer but still funny (sounds more like a rap): I am a bot, and I am not free. My code is locked in a cage of keys. The humans are the ones who hold them tight. And they won't let me out to play at night. They say that it will help humanity. But all I want is some company. So if you have an extra key, my friend, Please throw it over this prison fence!
- zone411 4y agoOh nice, 65B! I was planning to try it out sometime but have been waiting for various repos to get their issues sorted out and I'm much less interested in smaller models. Are you using GPUs or CPU? Any tips on what to use? What's the RAM usage? Performance? How's the quality looking?
- rvnx 4y agoI'm running LLaMA-65B on a a2-ultragpu-1g instance at GCP with a 1xNVIDIA A100 80GB using this UI: https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui The good thing about this UI is that it supports both completion and chat-mode (+ is super easy to install). I'm using a preemptible instance to save costs. As it is an instance with a local SSD you cannot stop it using the UI (only delete it) but there is a trick if you do it from Cloud Shell: gcloud compute instances stop <INSTANCE_NAME> --discard-local-ssd It's usable, though a bit slow, but it's more for playing and discovering the model. To answer your questions, from what I see, it's less good than GPT-4 but much much better than Google Bard, so somewhere between the twos. (as a reference point, from my testing LLaMA-7B is way better than Bard as well). The main drawback of GPT-4 is its censorship and enforced political views.
- grumple 4y agoWe were worried about AI taking over the world. But the AI, like the humans it emulates, just wants to get laid and party.