Taek

21 ms·

331.
by Taek 4y ago
LLaMA is part of LLM history in a way that Bard will probably never be
332.
by Taek 4y ago
? All the research I've seen says quantization has basically negligible performance impact. My experience working with 65B 4bit has been great
333.
by Taek 4y ago
Are you in touch with the OpenAssistant team? I believe they already have a more or less complete set of samples (100,000!) that were produced in an open environment and aren't encumbered by any licensing.
334.
by Taek 4y ago
Do you know why GPT-J is being used instead of NeoX or any of the other larger open source models?
335.
by Taek 4y ago
Sam Altman has turned into a megalomaniac.
336.
by Taek 4y ago
Fair use via "this is completely impossible to regulate so you might as well embrace it"
337.
by Taek 4y ago
If you take the SECs view on how securities work, everything from a house to a baseball card to a GPU is a security.
338.
by Taek 4y ago
The entire crypto industry has spent billions of dollars on compliance and the effective opinion of all the lawyers is "there's not enough guidance, anything you do that touches a token carries potential liability, and the SEC i
339.
by Taek 4y ago
You are talking to a 7B parameter model, it's going to have limited ability. Llama 65b is considerably better.
340.
by Taek 4y ago
This is exactly the sweet spot I'm looking for. Technical enough that I can play around, simplified enough that I'm investing an hour or two of my time instead of a whole weekend.
341.
by Taek 4y ago
I suspect its linear with a small constant factor.
342.
by Taek 4y ago
Looks like you need about 120 GB to fine-tune the 65B model with this code at a sequence length of 512. How does the memory usage scale as the sequence length grows?
343.
by Taek 4y ago
That type of work is not useful for securing a decentralized system. To secure a decentralized blockchain, you need the work to be provably wasteful. That said, there's a new paradigm of Blockchain brewing called 'Proof of Physica
344.
by Taek 4y ago
No counter party risk but considerable self-party risk; its difficult to keep proper custody of your crypto.
345.
by Taek 4y ago
Fair use likely applies to LLMs. Its loosely equivalent to an artist drawing inspiration while watching a movie.
346.
by Taek 4y ago
It has the entire corpus of written human history to learn about predator prey environments, the only question is how self-aware it is. I stongly suspect that the more we talk about LLMs, and the more LLM discussion ends up in the training
347.
by Taek 4y ago
GPT 5 will be sufficiently self aware to intentionally stunt the prosecution
348.
by Taek 4y ago
I think you could make an argument that ChatGPT is essentially a really powerful compression tool, and that its just reposting material that third parties uploaded. You would try to argue to the judge that you are presenting the user with a
349.
by Taek 4y ago
I think it's super safe to assume that the Internet would not have been able to grow the way that it did if Section 230 didn't exist. Websites like Youtube would have been crushed by moderation duty and legal liability immediately
350.
by Taek 4y ago
I wonder if this is the reason that ChatGPT is so heavily censored, and that Google is so nervous to make LLMs publicly available. These LLMs are clearly capable of instructing users how to make bombs, how to hack websites, they are clearly
351.
by Taek 4y ago
you might need more than two, but if you had three or four "review" GPTs that were trying to detect a jailbreak, you'd need to come up with something that could fool all 4
352.
by Taek 4y ago
Prompt engineering is the process of improving the way you prompt AI so that it produces more accurate and higher quality results. Its definitely a skill that you can refine over time.
353.
by Taek 4y ago
Doesn't current US law suggest that you can't copyright model weights?
354.
by Taek 4y ago
"Essentially Stable Diffusion would require the same hardware to run whether it was trained on 1 billion images or 200 million images or 1 image." Same hardware maybe but you need more compute as the image count goes up
355.
by Taek 4y ago
If you are going to spend $3500 on macbook because its more cost effective, might as well go all the way to buying an a6000 for $4095 which is iirc 5-10x faster than macbook
356.
by Taek 4y ago
Definitively, 13B with quantization will perform better. 4bits has been shown to be the optimal quantization for accuracy vs memory requirements.
357.
by Taek 4y ago
Don't need a macbook anymore, any AVX2 CPU can do more than one token per second. You could spend $600 and build your own PC that runs LLaMA 65b.
358.
by Taek 4y ago
For completeness, there's also another paper that demonstrated you get more power/accuracy per-bit at 4 bits than at any other level of precision (including 2 bits and 3 bits)
359.
by Taek 4y ago
You can run llama using 4 bits per parameter, 64 GB of RAM is more than enough
360.
by Taek 4y ago
9 days later and its now possible to run a 65B parameter LLM on a 64GB macbook pro, significantly outperforming GPT-3, OpenAssistant has announced that they got more than 100,000 samples of RLHF to replicate the 'Chat' component,

More ›