lucidrains

9 ms·

61.
by lucidrains 4y ago
bought "Mastering Nim" just yesterday!
62.
by lucidrains 4y ago
you are probably thinking of https://arxiv.org/abs/2207.03481 for inference, there is https://github.com/bigscience-workshop/petals however, both are only in the research phase. start tinkering!
63.
by lucidrains 4y ago
Yannic and the community he has built is such an educational force of good. His youtube videos explaining papers have helped me and so many others as well. Thank you Yannic for all that you do!
64.
by lucidrains 4y ago
they are missing a cute mascot like gophers for go, crabs for rust, dinosaurs for zig
65.
by lucidrains 4y ago
well, you are assuming these efficient matrices cannot become superhuman teachers that show us the inner workings of the universe in a more optimal way.
66.
by lucidrains 4y ago
personalized RL agents that finds aesthetic trajectories through the music latent space... soon, i hope :D
67.
by lucidrains 4y ago
it is neither as simple as the person you are responding to, nor as complicated as you make it seem. it will only get simpler with time.
68.
by lucidrains 4y ago
in some medical circles, alzheimers is referred to as "type 3 diabetes"
69.
by lucidrains 4y ago
Recommend reading "Man's Search for Meaning" by Viktor Frankl. Kurosawa film, "Ikiru" is worth a watch too Otherwise I believe finding purpose is a deeply personal journey for every single person on the planet. Good
70.
by lucidrains 4y ago
https://arxiv.org/abs/2008.01540
71.
by lucidrains 4y ago
True, it goes against the spirit of deep learning Still the Transframer results are interesting and has encouraged me to look into it
72.
Transformers are sample-efficient world models(arxiv.org)
67 points by lucidrains 4y ago|10 comments
73.
by lucidrains 4y ago
yup, training on DCT representation in general is an untapped research direction. can also help with larger images, longer audio, etc
74.
by lucidrains 4y ago
yea of course https://video-diffusion.github.io/ we are going all the way to the holodeck (my guess is latent diffusion on weights, aka hypernetwork, of a 4d NERF)
75.
by lucidrains 4y ago
out of curiosity, which hospital is this?
76.
by lucidrains 4y ago
so is dreambooth worth open-sourcing then given textual inversion?
77.
by lucidrains 4y ago
already done https://twitter.com/nousr_/status/1563912133120647168
78.
by lucidrains 4y ago
Ikr! Hang on, this ride is about to get crazy :)
79.
by lucidrains 4y ago
It is from the Heidelberg giants, Patrick Esser and Robin Rombach They continued building off their latent diffusion direction (encode with vqgan-vae and then diffusion in latent space) All roads lead to rome https://github.com&#
80.
by lucidrains 4y ago
To add to this, the deep learning field has already moved on towards MSA-less structure prediction. None of this would be possible without building on top of the work open sourced by Deepmind. https://www.biorxiv.org/content
81.
by lucidrains 4y ago
Imagen actually shows some of the components in DALLE2 is unnecessary, so Imagen will end up being easier to build. I'll definitely add the dynamic thresholding trick from Imagen to DALLE2 repository though; that is a finding that shou
82.
by lucidrains 4y ago
"Loneliness is not cured by human company. Loneliness is cured by contact with reality." - Anthony de Mello
83.
by lucidrains 4y ago
hey thanks! :^) hope someone makes the next big discovery with them
84.
by lucidrains 4y ago
have an implementation of this over at https://github.com/lucidrains/memorizing-transformers-pytorc... , for any researcher exploring retrieval and memory with attention networks
85.
by lucidrains 4y ago
Attention is all we need
86.
by lucidrains 5y ago
incorrect, they modified the license so it can be used for commercial use - https://github.com/deepmind/alphafold/commit/8173117130e6df8...
87.
by lucidrains 5y ago
revised scaling laws https://arxiv.org/abs/2203.15556 https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scalin...
88.
by lucidrains 5y ago
yes, they are most likely finetuning with this type of pretraining https://arxiv.org/abs/2103.10360 quite easy to build
89.
by lucidrains 5y ago
https://alphacode.deepmind.com/ gives you a glimpse inside of what emerged from a similar attention net trained on code. however, whether the attention net has been forced upon pixels, language, amino acid sequences, the re
90.
by lucidrains 5y ago
I would recommend Routing Transformer https://github.com/lucidrains/routing-transformer but the real truth is nothing beats full attention. Luckily, someone recently figured out how to get past the memory bottleneck.

More ›