schopra909

8 ms·

31.
by schopra909 8mo ago
honestly, it's really hard to shorten the feedback loop in this space. For this, we really just did run one experiment at a time and visually inspect the results everywhere. when you're going 0 -> 1, you're looking for &qu
32.
by schopra909 8mo ago
Hadn’t seen that before! Seems very in line with what with the broader points about regularization. In table 4 they show faster convergence in 200 epochs when used alongside REPA. I’d be curious to see if it ended up beating REPA by itself
33.
by schopra909 8mo ago
yep, Apache 2.0! so anyone's welcome to download and hack away
34.
by schopra909 8mo ago
Hi HN, I’m one of the two authors of the post and the Linum v2 text-to-video model ( https://news.ycombinator.com/item?id=46721488 ). We're releasing our Image-Video VAE (open weights) and a deep dive on how we built it.
35.
Learnings from 4 months of Image-Video VAE experiments(linum.ai)
129 points by schopra909 8mo ago|16 comments
36.
by schopra909 8mo ago
This is very cool. Side note, I really dig the JavaScript animations on the causal block diffusion blog post. Made the concept immediately clear
37.
by schopra909 9mo ago
That all being said, you can just delete the T5 from memory after encoding the text so save on memory. The 2B parameters will take up 4 Gb of memory but activations will be a lot more given size of context windows for video. A 720p 5 second
38.
by schopra909 9mo ago
Great idea! We haven’t tried it but def interested to see if that works as well. When we started down this path, T5 was the standard (back in 2024). Likely won’t be the text encoder for subsequent models, given its size (per your point) and
39.
by schopra909 9mo ago
I think YC just release video on the basics of diffusion, but honestly I don’t have a good end to end guide. We’re going to write up going 0->1 on a video model (all the steps) over the coming months. But it likely won’t be a class or an
40.
by schopra909 9mo ago
https://www.linum.ai/field-notes
41.
by schopra909 9mo ago
Not public yet — we’re going to clean it up so it’s readable and release it as blog posts. First one will be everything you need to know on building a VAE for image and video. Should be out in a few weeks. We’re figuring out the write balan
42.
by schopra909 9mo ago
T5 Encoder is ~5B parameters so back of the envelope would be ~10GB of VRAM (it's in bfloat16). So, for 360p should take ~15 GB RAM (+/- a few GB based on the duration of video generated). We can update the code over the next day
43.
by schopra909 9mo ago
Per the RAM comment, you may able to get it run locally with two tweaks: https://github.com/Linum-AI/linum-v2/blob/298b1bb9186b5b9ff6... 1) Free up the t5 as soon as the text is encoded, so you reclaim GPU RA
44.
by schopra909 9mo ago
Should be fixed now! Thanks again for the heads up
45.
by schopra909 9mo ago
Oh damn! Thanks for catching that -- going to ping the HF folks to see what they can do to fix the collection link. In the meantime here's the individual links to the models: https://huggingface.co/Linum-AI/linum-v
46.
Show HN: Text-to-video model from scratch (2 brothers, 2 years, 2B params)(huggingface.co)
158 points by schopra909 9mo ago|24 comments
47.
2025 in Movies: Political Dissidence and Familial Claustrophobia(thesahilchopra.com)
2 points by schopra909 10mo ago|0 comments
48.
by schopra909 11mo ago
I think you nailed it. For us it’s classifiers that we train for very specific domains. You’d think it’d be better to just finetune a smaller non-LLM model, but empirically we find the LLM finetunes (like 7B) perform better.
49.
by schopra909 1y ago
It’s not clear to me what the bottleneck for OCR to “100%” work with LLMS is. In my work we do a lot of stuff with image understanding and captioning (not OCR). There object identification and description works great, since all the models a
50.
by schopra909 1y ago
Very cool parallel. Never thought about it this way — but makes complete sense
51.
by schopra909 2y ago
Quite the opposite — if you have a long sequence only a smattering of the words will influence the meaning of the current word. Everything else is “noise”. Attention is really good at finding this smattering of words (ie assign most weight
52.
by schopra909 2y ago
The prompting feature looks great — excited to try this out.
53.
Linum (YC W23) is a hiring a founding AI engineer to train text-to-video models(workatastartup.com)
1 points by schopra909 2y ago
54.
Linum (YC W23) is hiring a founding AI engineer to train text-to-video models(workatastartup.com)
1 points by schopra909 2y ago
55.
by schopra909 3y ago
This is really impressive — excited to play around with it. Congrats on the launch!
56.
by schopra909 3y ago
This looks really promising! Really stoked to see where you guys take this.
57.
by schopra909 3y ago
This is really cool! As a next step, it would be even more useful if you could auto-inpaint on regions of the detected issues (e.g., malformed hands). That way you could maintain the subject / most of the image -- if you like everythin