4 ms·
Ask HN: What computer to buy if you want to do small fine tuning and inference?
I am not looking at anything expensive, but want to buy something for fine tuning small models or run small models for experimenting and code assistance.
- deleted 2y ago[deleted]
- tocs3 2y agoDo you have a computer? What does it not do that you need? I am not an expert of any sort but I have seen lots of people (and myself) buy equipment (computers, tools, bicycles, etc) and not use but a fraction of the capabilities or have to go out and but something else because it was the wrong thing.
- talldayo 2y agoYou can buy a 3060ti with modern CUDA support and 8gb of VRAM for ~$220 right now. If your only goal is inference/finetuning that should be the best option at it's price point.
- gtirloni 2y agoCheck these out: - https://www.hyperstack.cloud/technical-resources/tutorials/how-to-choose-the-right-gpu-for-llm-a-practical-guide https://www.hyperstack.cloud/technical-resources/tutorials/h... - https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/ https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni... - https://preview.redd.it/the-llm-gpu-buying-guide-august-2023-v0-4nve5pq5oaib1.png?width=1248&format=png&auto=webp&s=d6c59b7fdc75f671d933299b581c800d5adbb6ef https://preview.redd.it/the-llm-gpu-buying-guide-august-2023...