lucidrains

7 ms·

1.
by lucidrains 16d ago
trying to bring verification to bio? it is going to be tough, there's just an unbelievable amount of noise in wet lab work
2.
by lucidrains 1mo ago
there is a 2018 documentary on her life titled Kusama: Infinity. it appears you can watch it online for free now.
3.
by lucidrains 2mo ago
Reminiscent of his other short stories from the Martian Chronicles, also rife with apocalyptic themes
4.
by lucidrains 6mo ago
quite surprised to see SAC, considering the deepmind ping pong paper resorted to evolutionary strategies, iirc
5.
by lucidrains 9mo ago
They need to have a service that allows for pets. It is the only reason I still use Ubers!
6.
by lucidrains 1y ago
there has been some movement on that front, in the form of adding expand / delete tokens! https://hkunlp.github.io/blog/2025/dreamon/
7.
by lucidrains 1y ago
It is a reference to the beatles song, mainly because Noam Shazeer is a music lover
8.
by lucidrains 1y ago
very nice, will have to try it out! this is the same research group from which Robin Rombach (of stable diffusion fame) originated from
9.
by lucidrains 1y ago
https://theopenexo.nau.edu/
10.
by lucidrains 1y ago
I am sorry to hear that.
11.
by lucidrains 1y ago
indeed, it just becomes less likely
12.
by lucidrains 1y ago
if you are a man and make it past age of 29 without starting to hear voices, you can breathe a sigh of relief (I did)
13.
by lucidrains 1y ago
Love to know as well!
14.
by lucidrains 1y ago
oh wow, I was looking for use of islands or map-elites that I missed this.. thought it was the blandest mimetic evolution possible
15.
by lucidrains 2y ago
does this not mean we should explore usage of talking heads (Shazeer et al) a bit more? https://arxiv.org/abs/2003.02436
16.
by lucidrains 2y ago
Indeed, the best setting is in nature on a beautiful day
17.
by lucidrains 2y ago
amazing work! thank you!
18.
by lucidrains 2y ago
would be interested to see thunderkittens (great name!) tackle the flash attention backwards pass, which is an order of magnitude harder than the forward
19.
by lucidrains 2y ago
https://arxiv.org/abs/2404.08819
20.
by lucidrains 2y ago
wow, so prophet net does work! i spent so much time experimenting with it back in the day, but just lacked the scale to see a positive result.
21.
by lucidrains 2y ago
indeed, simsiam is a great example of the effectiveness of using stop gradient
22.
by lucidrains 2y ago
could even try it with a fraction of the attention heads, instead of introducing new tokens
23.
by lucidrains 2y ago
yes, it is a stop gradient mask on the attention matrix, iiuc. worth trying
24.
by lucidrains 3y ago
I can't think of anyone better to teach attention mechanism to the masses. This is a dream come true
25.
by lucidrains 3y ago
I've never worked with a project with more than 7 dimensions, yet
26.
by lucidrains 3y ago
oh interesting, Jeff Hawkin's HTM?
27.
by lucidrains 3y ago
"There are only two tragedies in life: one is not getting what one wants, and the other is getting it." - Oscar Wilde
28.
by lucidrains 3y ago
actually got some government shill who can't even spell to respond. touch a nerve?
29.
by lucidrains 3y ago
exactly, this guy understands how business in China works.
30.
by lucidrains 3y ago
would be nice if it were just more attention. there could be something else though

More ›