Taek

25 ms·

361.
by Taek 4y ago
I typically see people claim 2-3 tokens per word.
362.
by Taek 4y ago
And they've already collected over 100,000 samples, iirc ChatGPT was trained on something like 30,000 samples, so the open models should already be positioned to succeed.
363.
by Taek 4y ago
I don't think the quality of the OpenAssistant data represents that idea at all.
364.
by Taek 4y ago
Its worth buying a 3060
365.
by Taek 4y ago
OpenAssistant has already collected 100,000 human feedback examples, estimated 5,000+ hours of human work via crowd sourced volunteers. Enough programmers want this badly enough that its going to happen. Inference at 8 GB and fine tuning at
366.
by Taek 4y ago
Because the cards are so expensive, you really do want them running 24/7. The electricity is not a big deal for these really expensive chips
367.
by Taek 4y ago
I believe in terms of climate impact the chips (made in Taiwan) overwhelm everything else.
368.
by Taek 4y ago
Electricity costs are basically irrelevant because the cards are so expensive. A100 cards consume 250w each, with datacenter overheads we will call it 1000 kilowatts for all 2048 cards. 23 days is 552 hours, or 552,000 kilowatt hours total.
369.
by Taek 4y ago
Not access to GPUs for training? What would it take to get a 65B model out of eleuther?
370.
by Taek 4y ago
This is not the case with Stable Diffusion. The open source community is moving faster and innovating faster, and the open models are by far the best. Midjourney is a contender, but it doesn't hold a candle to the best open models
371.
by Taek 4y ago
I don't want to pile on, but security vulnerabilities are often composed of a series of seemingly insignificant weaknesses that get chained together in highly clever ways. Anything that is accessed over http gives a potential attacker
372.
by Taek 4y ago
Counter examples: + Stable Diffusion + Linux + Firefox + ffmpeg + bittorrent + plenty more
373.
by Taek 4y ago
They did open source all of their battery patents. Tesla isn't Mozilla, but they do have somewhat of a moral high ground to stand on.
374.
by Taek 4y ago
I hadn't thought about targeting a constructed user persona, that's a neat idea. But otherwise yes, its a recursive model that runs inference a bunch of times to improve the quality of its output.
375.
by Taek 4y ago
I've had success using a multistage inference approach. Establish a set of rules, then have the LLM ask itself whether its current suggested world building complies with all the rules. Then have it tweak its own story. Finally, after a
376.
by Taek 4y ago
These models are only capable of coherent conversation in the first place because they are so large. As soon as you shrink it down to be 'domain specific' its ability to form coherent sentences even within its own domain greatly r
377.
by Taek 4y ago
its not open source, only approved researchers are allowed to access the weights. More ivory towering from the AI industry.
378.
by Taek 4y ago
Can't have a stable diffusion moment if you refuse to release the weights to the general public. Stable diffusion only got to where it is because 10,000 people with otherwise zero reputation were able to play around with the code and m
379.
by Taek 4y ago
Its not open source until the weights are available. I have the hardware I need to run it but the required files are not available unless you receive special access. You can't use what has been released unless you want to spend $500,00
380.
by Taek 4y ago
For Stable Diffusion inference I recommend an nvidia 3060. If you want to train your own stable diffusion models I recommend a 3090. If you want to work with LLMs I recommend 2x3090. I also recommend at least 64 GB of RAM, and at least 2 TB
381.
by Taek 4y ago
It doesn't have any tensor cores, which means it will run a lot slower for many machine learning tasks, especially the deep learning stuff.
382.
by Taek 4y ago
I like this idea, and given today's technology it feels like something that could be accomplished and rolled out in the next 30 years. If the compiler (like rust) can prove that OOB memory is never accessed, the hardware/kernel&#x
383.
by Taek 4y ago
For reference, GPT-NeoX is a 20B parameter model, and it runs on 45 GB of VRAM. On an 80 GB A100 you could probably run a 35B parameter model. Maybe 8 A100 cards to do inference on ChatGPT? Or 32 3090 cards, which would run you under $40k t
384.
by Taek 4y ago
One thing I'm not seeing mentioned is the security implications of running a filesystem in VRAM. Unless things have changed, GPU VRAM is basically free-for-all and any application can do read/write operations anywhere in the memor
385.
by Taek 4y ago
I don't like the framing of 'learn something new' because you are very likely to have falsehoods presented to you as reliable facts. Its fun to ask Marie Curie what she thinks about blockchains, but anything you ask about her
386.
by Taek 4y ago
My laptop has had 8 GB of RAM since 2009. My $600 HP that I got in 2022 has 32 GB.
387.
by Taek 4y ago
Anything that relies on the ChatGPT API is subject to being insta-killed the moment OpenAI decides its not good to have them around. Productivity tools shouldn't depend on external parties like that.
388.
by Taek 4y ago
No more that as you get more AIs, all AIs get shittier together.
389.
by Taek 4y ago
Plain js on the frontend. Complex logic handled in rust-wasm. Backed handled in Rust using the Actix Web http server, using websockets, proxied through a very basic nginx instance with TLS. My own servers for the entire cloud, maybe with Cl
390.
by Taek 4y ago
If you are just looking for related vocabulary words, correctness is not a concern.

More ›