lucidrains

7 ms·

31.
by lucidrains 3y ago
previous systems could not compose objects within the scene correctly, not to this degree. what changed to allow for this? could this be a heavily cherrypicked example? guess we will have to wait for the paper and model to find out
32.
by lucidrains 3y ago
Yeah I know I've been to weddings from players who met on the site. It was magical
33.
by lucidrains 3y ago
No, you aren't wrong. For ML people, it is quite simple and hopefully the final code reflects that The aim is really to give a good base for follow up research / modifications, which I think there will be many for this paper
34.
by lucidrains 3y ago
Lol, hey Danny. Indeed it was great times I may bring it back, rebuilt in rails and svelte
35.
by lucidrains 3y ago
just to better match the math equations in the SPIN paper
36.
by lucidrains 3y ago
hey, appreciate the interest! repo is not done yet, but probably will be around month's end
37.
by lucidrains 3y ago
been using the self checkout at Uniqlo recently. works beautifully
38.
by lucidrains 3y ago
nice! will make reading papers on the phone so much more pleasant!
39.
by lucidrains 3y ago
slow and steady wins the race
40.
by lucidrains 3y ago
gradtree is cool! thanks for the share!
41.
by lucidrains 3y ago
Yannic did a great interview with the authors some time ago https://youtu.be/_NMQyOu2HTo
42.
by lucidrains 3y ago
https://arxiv.org/abs/2307.06440
43.
by lucidrains 3y ago
thank you for sharing your thoughts! is there anyway I can get in touch with you, through email or some other means?
44.
by lucidrains 3y ago
you have no idea what you are talking about
45.
by lucidrains 3y ago
this is fixing a different issue, not the one you are measuring.
46.
by lucidrains 3y ago
huge! thank you Tri!
47.
by lucidrains 3y ago
got a response here https://github.com/lucidrains/soundstorm-pytorch/discussions...
48.
by lucidrains 3y ago
can an expert in the field comment on whether the results are more or less impressive than Soundstorm?
49.
by lucidrains 3y ago
same, I am too old to really care if others cannot see this as nearly the perfect language
50.
by lucidrains 3y ago
Same, I love it! Having done ruby, javascript, python, I'm allowed one niche language (although I really hope it grows in popularity!)
51.
by lucidrains 3y ago
tldr: bigger text encoder is better. SD will catch up quickly, as conditioning on a new set of precomputed text embeddings is a trivial change
52.
by lucidrains 3y ago
besides everything that was mentioned here, what made it finally click for me early in my journey was running through this excellent tutorial by Peter Bloem multiple times https://peterbloem.nl/blog/transformers highly
53.
by lucidrains 3y ago
the code is here https://github.com/hazyresearch/safari you should try it and let us know your verdict.
54.
by lucidrains 4y ago
or love, understand, help each of those human beings.
55.
by lucidrains 4y ago
that line of research is still going. https://github.com/lucidrains/block-recurrent-transformer-py... i think it is worth continuing research on both fronts.
56.
by lucidrains 4y ago
It is only linear in terms of memory, not compute. Flash attention is a big advance, but not enough for 1 million tokens
57.
by lucidrains 4y ago
Yikes, didn't know this was an issue at Google as well. That is scary
58.
by lucidrains 4y ago
When I was working at Uber, we had a case of a guy stalking his ex-girlfriend's every movement through the backend. They eventually found out, fired him, and stepped up auditing.
59.
by lucidrains 4y ago
while this topic has visibility, there is also another relatively unexplored research direction, fine-tuning pretrained transformers into RNNs https://arxiv.org/abs/2210.04243 https://aclanthology.org/2
60.
by lucidrains 4y ago
that's the one I bought!

More ›