Taek

23 ms·

271.
by Taek 3y ago
Before stable diffusion, nobody released weights at all. Meta et al only started sharing their models with the world when they realized how fast a developer ecosystem was building around the best models. Without stability, all of AI would s
272.
by Taek 3y ago
Rust made a critical safety mistake when it chose its async paradigm. It gave the code the option to decide when to yield. What that means is that when I'm writing async code, I have to audit every library I import to make sure that li
273.
by Taek 3y ago
Website says 320 GB for GPTQ / int4. I'm not sure why, my guess would be multi-head attention
274.
by Taek 3y ago
The brain is literally liquid cooled (blood), it can totally dispense of 100w of heat if it has to. (that said, other commenters are correct, the brain used 20w)
275.
by Taek 3y ago
XL more likely refers to the parameter count, which is 3 billion instead of <1 billion
276.
by Taek 3y ago
The problem is that the stock market is moving in an increasingly greater-fool direction. Companies that don't ever pay dividends, companies that don't yield control rights, companies that have extensive protections for the founde
277.
by Taek 3y ago
We educate our children to participate in greater fool games. Go to college, regardless of tuition. Put your money in the stock market, regardless of its health and structure. Buy bonds, regardless of the interest rate. Own a house, regardl
278.
by Taek 3y ago
This character was added to Unicode in 2001, which notably is after Elon acquired x.com
279.
by Taek 3y ago
I didn't realize that the llama license forbids you from using its outputs to train other models. That's essentially a dealbreaker, synthetic data is going to be the most important type of training data from here on out. Any model
280.
by Taek 3y ago
That's not necessarily reliable data, sometimes a company that is in a losing position will bring in a well paid CEO to reduce the damage. -5% can be an excellent result if you were expecting -20% as the status quo.
281.
by Taek 3y ago
Its not quite the same, the highres fix in Auto1111 used the same model twice, the upscaling for SDXL uses two different models for each stage.
282.
by Taek 3y ago
An LLM is a million times better than a YouTube video because it can stop and re-explain anything specific that is confusing you, or have discussions about tangential but off-curriculum topics, ask you questions based on its own appraisal o
283.
by Taek 3y ago
If an LLM is being used to make decisions for the government, the people have a right to know what logic is driving the decisions. The government should demand open source code, open weights, and open research from any AI company it purchas
284.
by Taek 3y ago
2005-2010 for me
285.
by Taek 3y ago
I disagree. When I was a kid I always bought Dell for my desktop computers because you could get the same specs for $300-$400 less than other providers (HP, Compaq, etc). My Mom would give me a $600 budget and I would do hours of research f
286.
by Taek 3y ago
That's why Open Source LLMs are critical. Any commercial LLM you use is going to have advertisements baked in, preferences for particular products, perhaps political biases. Corporations will pay through the nose to have a trusted assi
287.
by Taek 3y ago
Why not enforce that decompression is a necessary part of the data cleaning? There's no reason to operate on complex formats like mp4 directly.
288.
by Taek 3y ago
This seems like a problem you can only solve with an invite or credential system. If you are an invited writer (or have some sort of literary degree) you can submit content, otherwise you gotta let people invite you. AI content is still all
289.
by Taek 3y ago
*the api wasn't nerfed. ChatGPT4 was most certainly changed
290.
by Taek 3y ago
Another vote from me for guanaco 65b. Here's a link to the model and weights: https://huggingface.co/TheBloke/guanaco-65B-GPTQ I use it all the time at home. It's decent at things like summaries and writing c
291.
by Taek 3y ago
I don't know what the breakdown is, but I know there was code for training the llama models on a DGX (640 GB of VRAM, repo is now gone), and it could only train the 7b model without using deepspeed 3 (offloading). The ML engineers in m
292.
by Taek 3y ago
The general rule of thumb that I'm familiar with is that you need about 80 bytes of VRAM per parameter when you are doing training. Inference is different and a lot more efficient, and LoRA is also different and more efficient, but tra
293.
by Taek 3y ago
Often times you can increase the humor by calling attention to the joke, even if the joke is already obvious. Puns in particular seem (at least to me) to get funnier the longer you think about them so prompting the reader to think about the
294.
by Taek 3y ago
I don't think that works if your attacker has millions of IPs and is only using 200 per second.
295.
by Taek 3y ago
Fun to think how the rules could change if you made the maze 3D. Now jumping is allowed. Flying as well? Touching walls? Cameras can't necessarily see the whole maze anymore either.
296.
by Taek 3y ago
Have there been thoughts to adjust the rules or make a parallel format that increases the amount if opportunity that competitors have to get creative?
297.
by Taek 3y ago
What are some barriers to going faster?
298.
by Taek 3y ago
Going to mention this as a top level comment because a lot of people here seem to believe that Monero is an anonymous cryptocurrency. Unfortunately, techniques to break Monero's anonymity have been known since 2018, and these technique
299.
by Taek 3y ago
I hate to burst the bubble but strategies to break Monero's anonymity have been known since 2019. It turns out that the general strategy of mixing a subset of outputs together is insufficient to provide robust privacy, even if the mix
300.
by Taek 3y ago
For "full fine tuning", mathematically there's no difference. Fine tuning is just extending the training on new data. What you are suggesting is called "curriculum learning", and though it hasn't been applied t

More ›