152334H

9 ms·

61.
Non-determinism in GPT-4 is caused by Sparse MoE(152334h.github.io)
397 points by 152334H 3y ago|181 comments
62.
by 152334H 3y ago
Strange. There seems to be no mention of vast.ai in the GPU Cloud Providers list, but they're definitely cheaper than all of the options listed. Perhaps the writer kept them unmentioned in an attempt to reduce demand?
63.
by 152334H 3y ago
> I think it's more plausible that teaching focuses on this surface knowledge because it's much easier and more legible, and looks and feels very much like "programming education" to someone who does not have actual d
64.
by 152334H 3y ago
I agree with that, but it's hard for me to get excited with the knowledge that it'll almost certainly be discarded and forgotten. I've seen too many papers that looked interesting from a theoretical perspective, but were simp
65.
by 152334H 3y ago
No model, no code. About as exciting as Imagen-video, which was released eons ago.
66.
LLaVA: Large Language and Vision Assistant(llava-vl.github.io)
3 points by 152334H 3y ago|0 comments
67.
by 152334H 4y ago
Regardless of what you want to learn, "small daily activities" is a bit hard. You can learn some stuff by osmosis, following the feeds of AI devs && AI channels, but the bulk of what I learn comes from starting projects &a
68.
by 152334H 4y ago
I don't have much experience myself. I only started ~10 months ago -- just a month or two before Stable Diffusion. You just have to do it every day. It's fun!
69.
by 152334H 4y ago
It'll work! I just haven't touched any of the 4bit stuff myself, so I don't personally know how to add it. Great low-hanging fruit for anyone else to take on.
70.
by 152334H 4y ago
It's better because 1. it's using vicuna as a base. 2. It has a pretty high quality fine-tuning dataset. I initially missed this, and it's a very important advantage. 3. (speculatively) it doesn't collapse to extremely s
71.
by 152334H 4y ago
On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are p
72.
by 152334H 4y ago
Majority of deployed services right now are TorToiSe derivatives. https://git.ecker.tech/mrq/ai-voice-cloning
73.
by 152334H 4y ago
just fyi, this will change soon
74.
by 152334H 4y ago
Where A,B,C are empirical constants, Loss = A/parameters + B/dataset_size + C Just read the chinchilla paper.
75.
by 152334H 4y ago
I spoke with the authors of the paper; the leftmost points in Figure 1 were generated with batch-size 1, indicating ~1.2x and ~2x improvements in speed over DeepSpeed for 30B and 175B models respectively. For reference, this is speeding up
76.
Why can't TorToiSe be fine-tuned?(152334h.github.io)
1 points by 152334H 4y ago|0 comments
77.
by 152334H 4y ago
HN is just incredibly bad at figuring out what kind of ML projects are worth getting excited about and what aren't. MJ v4 doesn't even use Stable Diffusion as a base [0]; a fine-tune of the latter will never come close to achievin
78.
by 152334H 4y ago
With all of these outages, you have to wonder if it's a lack of skilled engineers on their part, or if they simply don't have enough GPUs to keep the lights on all the time.
79.
by 152334H 4y ago
No mention of any other competitors that've been doing this stuff for several years? Uberduck? Fakeyou? Coqui? 15?
80.
by 152334H 4y ago
Another example of "ChatGPT for open-ended law": https://broodingomnipresence.substack.com/p/brooding-omnipre... Tangentially related: DoNotPay's (probably illegal) plans to use LLMs in actual court sess
81.
by 152334H 4y ago
More samples available on their github page: https://valle-demo.github.io/ Personally, I find that their samples aren't near anything I'd call "dangerous". I cross-compared the baseline examples to the V
82.
by 152334H 4y ago
Forgive the anthropomorphising, but I find it very similar to how I'd handle someone forcing me to explain two contradictory claims coherently.
83.
by 152334H 4y ago
I see no one has mentioned Byte-Pair Encoding (BPE) yet. https://www.gwern.net/GPT-3#bpes The reason ChatGPT struggles with tasks involving individual character counting is that it can't see individual characters. Th
84.
by 152334H 4y ago
Interesting, thanks.
85.
by 152334H 4y ago
Any word on what the neovim plugin does? Considering some features like, > Then, most recently accessed 20 files of the same language are queried from VSCode. are probably not available by default in nvim? --- Also, do you think there&#x
86.
by 152334H 4y ago
There would be a lot less mundane questions on the internet if people would google/chatGPT first. I think it is valuable to try to direct people to using said tools when they are sufficient for an answer.
87.
by 152334H 4y ago
"Whisper V2" is a bit of an exaggeration. It's * only an update to the `large` model (and therefore irrelevant for hobbyist, low CPU use cases) * architecturally identical to the original whisper model * just trained on more
88.
by 152334H 4y ago
Not exactly a novel idea. https://twitter.com/AdeptAILabs/status/1590396065072951296
89.
by 152334H 4y ago
I'm interested in finding out which day it'll start to break. GPT-3/ChatGPT are not perfectly accurate programmers, and some of the longer problems with more fluff text might distract it from the problem. I would be really su