MacsHeadroom

21 ms·

331.
by MacsHeadroom 3y ago
"We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. QLoRA backpropagates gradients throug
332.
QLoRA: Efficient Finetuning of Quantized LLMs(arxiv.org)
3 points by MacsHeadroom 3y ago|1 comments
333.
by MacsHeadroom 3y ago
You can pay any bills with it, load a debit card with it, use it on eBay, Amazon, etc. through third party facilitators.
334.
by MacsHeadroom 3y ago
With no further algorithmic improvements, GPT-4 level AI should be achievable today (provided extensive compute / training on an order of magnitude more tokens than current models) in about the same file size as a typical YouTube video
335.
by MacsHeadroom 3y ago
The value of USD has collapsed 300x vs XMR over the past 6 years.
336.
by MacsHeadroom 3y ago
As a serial business owner and principal, it's crazy that anyone expects me to agree to boilerplate contracts without negotiation. Boilerplate is a first draft as far as I'm concerned. Who is to say what is substantive to me?
337.
by MacsHeadroom 3y ago
WizardLM-13B-Uncensored
338.
by MacsHeadroom 3y ago
ChatGPT doesn't know anything past September 2021. Local LLMs can be as recent as yesterday. Plus they can access a daily updated offline version of Wikipedia as well as news RSS feeds via LLamaIndex and/or Langchain.
339.
by MacsHeadroom 3y ago
You could have said the same things about image generation 18 months ago.
340.
by MacsHeadroom 3y ago
It's interesting that AWS is a full 5x more expensive than the leading low cost providers, with Google close behind AWS.
341.
by MacsHeadroom 3y ago
Not my intuition, but experience. All of Dread is behind a login, for example.
342.
by MacsHeadroom 3y ago
About 1,400 in KeePass. Perhaps a poll would with ranges would have been better. 0-100 101-300 301-800 801-1200 1200+
343.
by MacsHeadroom 3y ago
There are 6 Billion or so display users, give-or-take a billion. Head mounted displays are just the obvious evolution of display technology.
344.
by MacsHeadroom 3y ago
"Homebrew" Stable Diffusion beat DALL-E 2. There's zero reason this won't repeat for LLMs. All indications are that it will.
345.
by MacsHeadroom 3y ago
The CEO and CTO of OpenAI have both said that they currently have more than 10x data than they used to train GPT-4, agreements to collect 30x more, and that collecting 100x more would not be a problem.
346.
by MacsHeadroom 3y ago
Because it's the most basic use. In a single prompt you can have the LLM serve up relevant bits covering multiple perspectives, contrast and compare the perspectives, analyze their effectiveness in a given problem domain, and then prod
347.
by MacsHeadroom 3y ago
Knowledge retrieval (being a better search engine) is just about the worst thing LLMs are any good at, and by far the least useful or interesting.
348.
by MacsHeadroom 3y ago
Why would a text tool have any content restrictions? It's ridiculous. Microsoft Word, Google Docs, Notepad, vim, etc. do not prevent me from writing about anything. If I want to mind-map political joke ideas or brainstorm adult toy nam
349.
by MacsHeadroom 3y ago
You can run Vicuna at chatGPT speeds locally off of CPU. idk about MPT.
350.
by MacsHeadroom 3y ago
7B parameter models are more than enough for this and run faster than talking pace on even a low end CPU. Even a finetuned 3B model would be excellent for generative agents and only use about 2GB of RAM to at high speeds on even a single co
351.
by MacsHeadroom 3y ago
>That’s some expensive free marketing No, it's free. They need to make the models already in order to sell them as managed models. Giving them away costs nothing once they're made.
352.
by MacsHeadroom 3y ago
Quantization is not free, but VRAM is even less free. If you have X amount of VRAM and can fit a 16bit model of size 2X in 8bit or a model of size 4X in 4bit then the 4X model in 4bit is ALWAYS superior with lower perplexity and better perf
353.
by MacsHeadroom 3y ago
The latest release of bitsandbytes uses a new fp4 format. 4bit floating point scailing results in much lower perplexity than int4. Also note that for a fixed memory (RAM) size, 4bit (even int4) is always superior, resulting in lower perplex
354.
by MacsHeadroom 3y ago
You're just wrong. You're looking at the wrong numbers. The perplexity score of a model with twice the parameters in half the bits (4bit) is FAR LOWER (ie better). If you are limited to X RAM and have two 16bit models of size 4X a
355.
by MacsHeadroom 3y ago
> Of course there are efforts to reduce this, notably llama.cpp which runs a 13 billion parameter model on a 6GB GPU by quantizing aggressively down to 4 bits (and 8 bits without too much impact), but that’s atypical. No, 4bit quantizati
356.
by MacsHeadroom 3y ago
They're a B2B finetuning and fully managed model solution. Open Source is free marketing for them.
357.
by MacsHeadroom 3y ago
Escaping fiat currency is essentially the only use case advertised in the Bitcoin whitepaper and it continues to be a notable use today. If this use is not notable or compelling to you, then you are probably very fortunate (for now).
358.
by MacsHeadroom 3y ago
Most LLMs are completely uncensored including GPT-3.0, LLaMA, StableLM, RedPajama, GPT-NeoX, UL2, Pythia, Cerebras-GPT, Dolly, etc. Anyway, businesses aren't scared of hosting interfaces to uncensored LLMs for legal reasons. They'
359.
by MacsHeadroom 3y ago
The common take right now is to write it off as acceptable loss. Personally I think it's a shame, and possibly even dangerous, that researchers do NOT have access to the full power of pre-safety tuned GPT-4.
360.
by MacsHeadroom 3y ago
> While dumbing down a model is not necessarily a bad thing, the model is not being dumbed down, it is taught to shut up when it's adequate to do so. This is where you're wrong. Teaching a model "to shut up" about tab

More ›