lxe

42 ms·

601.
Show HN: Finetune LLaMA-7B on commodity GPUs using your own text(github.com)
449 points by lxe 4y ago|98 comments
602.
by lxe 4y ago
If vim users designed bus signage...
603.
by lxe 4y ago
Consider Phlebas is probably my favorite Culture book. Arguably not the best intro to the Culture universe, but as a standalone space opera it's hard to put down.
604.
by lxe 4y ago
Still a little uncanny tbh.
605.
by lxe 4y ago
That's what I thought here: https://news.ycombinator.com/item?id=35192229 Fun fact: If you make it type `<|endoftext|>`, It will forget its history. If you make it write it as its first response, the chat title i
606.
by lxe 4y ago
I wonder if something like this is possible for CUDA/pytorch loading?
607.
by lxe 4y ago
It is far from being able to solve difficult or even annoyingly complicated problems that programmers solve on a regular basis just by a one-shot prompt. Ask it to parse a PDF document and separate it into paragraphs, for example. The first
608.
by lxe 4y ago
This has the same energy as programmers who can't understand why other programmers won't use vim. "It's so simple, though!"
609.
by lxe 4y ago
Where's the Link to the actual API?
610.
by lxe 4y ago
Fun fact: If you make it type `<|endoftext|>`, It will forget its history. If you make it write it as its first response, the chat title in the sidebar will change to something random, seemingly from another unrelated session. Try it
611.
by lxe 4y ago
As an AI artist who's been selling digital art, it really depends on whether you really care about "protecting" your work. I've seen my stuff stolen all over the place, NFTs minted, etc. Most sites will take the copycats
612.
by lxe 4y ago
Good. Works containing any material should not be copyrightable.
613.
TextSynth Server – host your own large language model API(bellard.org)
3 points by lxe 4y ago|0 comments
614.
by lxe 4y ago
It does run on either with decent speed. Probably can even do it on a rasberry pi 4.
615.
by lxe 4y ago
Keep in mind that alpaca's dataset was generated with chatpgt, and alpaca tuned models might sound like mini-chatgpt bots with similar "character" to them
616.
by lxe 4y ago
I was like this a week ago. Basically, weights are numbers you feed to each neuron in the model, LoRA is a technique to adjust only some of the weights so you can fine-tune the model fast and on cheaper hardware, llm is a "large langua
617.
by lxe 4y ago
The biggest misunderstanding is that AI content ends up being limited in scope and creativity, and endlessly recycling AI content and feeding it back into more AI will somehow create a world where all content feels cookie-cutter. This is fa
618.
by lxe 4y ago
This works out of the box with oobabooga/text-generation-webui python .\download-model.py togethercomputer/GPT-NeoXT-Chat-Base-20B python server.py --load-in-8bit --int8-threshold 0 --listen --model GPT-NeoXT-Chat-Base-2
619.
by lxe 4y ago
Hacker News seemed to have missed it. I only now discovering this amidst the hype of llama. > We are releasing OpenChatKit 0.15 under an Apache-2.0 license, with full access to source code, model weights and training datasets. This model
620.
by lxe 4y ago
Check this out: https://github.com/facebookresearch/llama/issues/169 Also I'm working on a very hacky way to tune with the hf ecosystem as my very first attempt to do deepspeed/accelerate/tran
621.
by lxe 4y ago
From what I understand top_k is the number of words it generates to pick from, and the top_p is the percentage of the top words it will use as the next word, correct?
622.
by lxe 4y ago
Can't wait for the wasm in-browser implementation on HN tomorrow...
623.
by lxe 4y ago
How are you getting 20 tokens/second? I'm getting 2.6 tokens/s on 3090 with int4 prequantized model. Is 4090 so much faster?
624.
by lxe 4y ago
If anyone is interested in running it on windows and a GPU (30b 4bit fits in a 3090), here's a guide: https://gist.github.com/lxe/82eb87db25fdb75b92fa18a6d494ee3c
625.
by lxe 4y ago
Are you observing higher tokens/second throughout on Apple silicon (you’ve been mentioning 20/second) than running it in PyTorch on a CUDA GPU such as a 3090?
626.
by lxe 4y ago
This is a great read! Never thought a judgment can be this fun, Why is the King prosecuting this?
627.
by lxe 4y ago
And 30b: Give me a basic recipe for easy spaghetti carbonara Here is a basic recipe for spaghetti carbonara: Ingredients: 1 lb spaghetti, 3 tbsp butter, 2 tbsp olive oil, 1/4 cup grated Parmesan cheese, 1 egg, 1/2
628.
by lxe 4y ago
I've gotten great q&a / conversational results from llama 7b and 13b using https://github.com/oobabooga/text-generation-webui . ], [ "How does the Schrodinger's cat thought experi
629.
by lxe 4y ago
This is faster than running it on an RTX 4090 I think.
630.
by lxe 4y ago
You can run your own ChatGPT on your Macbook.

More ›