sabareesh

8 ms·

61.
by sabareesh 2y ago
Start with FineWebEdu
62.
by sabareesh 2y ago
I ran several nccl test had no issue with bandwidth. https://github.com/NVIDIA/nccl-tests?tab=readme-ov-file
63.
by sabareesh 2y ago
At this point it is still not worth considering AMD but may me this will change soon. I would look into semianalysis report
64.
by sabareesh 2y ago
I was eyeing 4060 before going with 4090. But it boils down to cuda cores and memory bandwidth
65.
by sabareesh 2y ago
Roughly it takes 7 days to train on 100B tokens on 500M model
66.
by sabareesh 2y ago
You might find more information here helpful https://sabareesh.com/posts/llm-intro/ But i am still in process of evaluating post training process with RL. RLHF is almost a mirage that shows what is possible but n
67.
by sabareesh 2y ago
It is co-authored by AI but I left it because it made some indirect sense. I clarified on the parent comment
68.
by sabareesh 2y ago
I clarified bit more on the article regarding this. But basically "Well this may not directly provide benefit but because this is a consumer grade card these features enabled having support for more advanced features such as bfloat16 a
69.
by sabareesh 2y ago
It is previous architecture and it doesnt support newer version of Flash Attention , fp8 training etc
70.
by sabareesh 2y ago
You get more vram but not enough cores
71.
by sabareesh 2y ago
That is real concern especially 4090 is not power efficient , as a100 and h100, h200. I live in Reno so it was ok
72.
by sabareesh 2y ago
Yup my initial reason behind is to learn all the quirks
73.
by sabareesh 2y ago
Definitely agree but part of the reason why i built this to learn about all the overhead and gotchas
74.
by sabareesh 2y ago
Well during training all GPUs were consuming max ~ 450W
75.
by sabareesh 2y ago
Here is some LLM models not diffusion though. https://huggingface.co/sabareesh88/fw14k this post has additional details https://sabareesh.com/posts/llm-intro/
76.
by sabareesh 2y ago
One of the motivation is to do several distillation, experimentation, research. But as you mentioned there are better ways to do this
77.
by sabareesh 2y ago
I was contemplating between building rig vs using the cloud but for some reason I want to get hands on. So you can always rent them for a fraction of cost
78.
by sabareesh 2y ago
Here is some additional journey apart from the rig. https://sabareesh.com/posts/llm-intro/
79.
by sabareesh 2y ago
Yeah depends on the price, definitely 24GB is limiting
80.
by sabareesh 2y ago
Yes it is , thanks for the feedback. I will soon add it to github
81.
All You Need Is 4x 4090 GPUs to Train Your Own Model(sabareesh.com)
125 points by sabareesh 2y ago|120 comments
82.
by sabareesh 2y ago
Hey HN I am sharing my experience on how i pretrained my own LLM by building a ML rig at home
83.
by sabareesh 2y ago
This is the key, cold applying for jobs no longer work
84.
by sabareesh 2y ago
Yeah with cloudflare you need to do it every 30 seconds as the timeout is is 60 seconds
85.
by sabareesh 2y ago
That's intriguing; one might assume that a key feature of eFuse would be the ability to reset easily. But I guess it could be implemented without it
86.
by sabareesh 2y ago
Just flip it upside down
87.
by sabareesh 2y ago
I know everyone is complaining about the power button why not just flip it upside down ?
88.
by sabareesh 2y ago
Well good old days of ioio which you can connect with android phone through usb or bluetooth and play with lower level pins.
89.
by sabareesh 2y ago
Recently came across this video which goes into history of Japanese computers by Asianometry https://www.youtube.com/watch?v=CEtgzO-Im8w
90.
by sabareesh 2y ago
If so good why are you selling it as a service ?

More ›