3 ms·
We can name these hypothetical objects Recursive Neural Networks.
by optimalsolver 2y ago
We can name these hypothetical objects Recursive Neural Networks.
- whimsicalism 2y agoi know you're jesting but RNNs are recursive along the sequence length where I am describing recursion along the depth.
- refulgentis 2y agoLike decode the next token, then adjust what you're paying attention to, then decode it again?
- deleted 2y ago[deleted]
- nine_k 2y agoIsn't it the only way to, say,understand a pun?
- refulgentis 2y agoThat is exactly how LLM inference is performed, so I'm being cheeky (I'm 99% sure anyone proposing anything in this thread is someone handwaving based on limited understanding)
- whimsicalism 2y agoYou would be wrong, but that is fine. Been working with attention since 2018. Why assume I know little and leave snarky comments (and basically a repetition of the prior joke at that, subbing RNN for transformer)?
- refulgentis 2y agoTo playfully invite for you to participate in conversation further, so that I may humbly learn from you. "I don't know what you're talking about" seemed too spartan and austere and aggressive, and you reciprocated politely, if again sparsely, when the other person playfully invited you to elaborate.
- webmaven 2y agoWell, you've now made your original intent specific, but in case you didn't draw the requisite lesson I'll make that explicit. Because text has less bandwidth than almost any other medium, certain forms of humor are much more likely to be understood (in this case, your "gentle playfulness" was taken to be snark, sarcasm, and point scoring). If you insist on using this and similar forms of humor that, ordinarily, depend quite strongly on intonation to convey intent, you'll have to be much more explicit to avoid being misunderstood. You are going to have actually state your intent explicitly as part of your communication. This need not entirely destroy the humor, for example, you might try something like this: And so I say to you (playfully, sir, playfully): etc. Or this: Yadda yadda yadda. (I kid, I kid!) The Internet-native forms of this are the humble ;-) or the newer j/k, but I find that it is all too easy to overlook a 3-character sequence, particularly if the passage being so marked is even as long as a single paragraph, but they can serve their purpose when used for the commonplace one-liner.
- Ycombigatorz 2y ago[dead]
- anamax 2y ago"blah, blah, blah" can be an expression of scornful boredom or the utterance of a vampire.
- refulgentis 2y agoYou are painfully boring
- benreesman 2y agoDepthwise RNN?
- YeGoblynQueenne 2y agoRecursive NNs are not the same as Recurrent NNs: https://en.wikipedia.org/wiki/Recursive_neural_network https://en.wikipedia.org/wiki/Recursive_neural_network Well ish. The article above explains that Recursive-NNs are hierarchical whereas RNNs are linear. I guess the distinction is a little on the fine side. Anyway carry on. Pedantic moment over.
- whimsicalism 2y agoThe recursive neural networks described there are a failed academic project from more than a decade ago, predating modern deep learning. Basically everyone using the phrase recursive nn nowadays is probably just mispeaking for RNN. RNNs also are not linear
- ska 2y agoNo opinion on the specifics of this distinction, but it's worth noting that in research, an awful lot of successful projects have their origins in failed projects of decades ago...
- whimsicalism 2y agoMy experience working in machine learning academia is an overfocus on failed projects from the early 00s to 90s that really only stopped in 2020+. We can often trace back successful projects to failed precursors, but often the people behind the successful project are not even familiar with the failed precursor and the 'connection to the past' only really occurs in retrospect. See the 'adjoint state method' and connections with backprop.
- ska 2y agoThis is sometimes true, sure. And often the older work has more entered the general consciousness than being chased down by searching specific cites. On the other hand, very little is truly new, and recency bias can lead you into all sorts of back-eddy's. Once the dust has settled, there are often much clearer through lines than in looked like at the time. It's hard to see when you are on the moving front though.
- conradev 2y agoYep: https://arxiv.org/abs/2305.13048 https://arxiv.org/abs/2305.13048
- p1esk 2y agoWe did: https://en.m.wikipedia.org/wiki/Recursive_neural_network https://en.m.wikipedia.org/wiki/Recursive_neural_network