D-Machine

35 ms·

571.
by D-Machine 1y ago
This is too metaphorical, but, still, basically correct. Nice to see that. Essentially, in the latent / embedding / semantic space, "seahorse emoji" is something that is highly probable. Actually, more accurately, since
572.
by D-Machine 1y ago
ChatGPT Extended Thinking nails it very quickly: https://chatgpt.com/s/t_68e3476bf2dc8191bd54b2f5200aceb1 > No. There is no official seahorse emoji in Unicode. Closest: [gives list of closest emoji]. > source: h
573.
by D-Machine 1y ago
Basically, yes. However, if we make a distinction between respirators (e.g. N95 mask) and masks (including "surgical" masks, which don't really have a meaningfully better FFE than cloth masks), then at least respirators offer
574.
by D-Machine 1y ago
>> I don't mean "sort of", I mean literally. Well, then we disagree, or are talking past each other, and I think writing out the equations and code show that, as I said, these are really not exactly the same thing. From
575.
by D-Machine 1y ago
He literally said "with the MAGA gang desperately trying to characterize this kid who murdered Charlie Kirk as anything other than one of them". Interpretation hinges on the "desperately" here, but your reading of this a
576.
by D-Machine 1y ago
Yeah, I am aware that there are sort of long residuals even in classic ViTs, and that, as you say, you can sort of even skip the whole transformer. Like you said, though, this seems very unlikely in practice, and at least, this is a dif
577.
by D-Machine 1y ago
Yup, normalization doesn't fix vanishing gradients. It just sort of generally helps with conditioning (as you likely know, given the PhD, probably in part because it helps keep certain Lipschitz constants from being too large), just li
578.
by D-Machine 1y ago
Thing is, even if the "trying to portray [him] as anything other than..." reading was intended and the correct reading, the statement is still closer to the opposite of the truth. Granted, it is not reasonable to expect everyone
579.
by D-Machine 1y ago
Well, you can build deep networks without residual connections, as long as you keep normalizing and initialize very carefully. I'm sure there are other clever ways to keep things well conditioned, like fancy reparameterizations or e.
580.
by D-Machine 1y ago
Yup, I am pretty up to date with the literature (mostly) and read about DenseNet a long time ago (and have trained trained and tuned quite a few of them too). I've also done enough experimentation with tuning custom CNN architectures (
581.
by D-Machine 1y ago
I'm not sure your claim is correct, nor particularly precise. Skip connections do propagate feature maps from earlier to later/deeper in the network, and one can model this mathematically as using an identity function to do this,