4 ms·
I was surprised recently when I fired up ollama on my refurbished Thinkpad -- a laptop that doesn't even have a GPU. All of the hype had me convinced that I cou
by evbogue 2y ago
I was surprised recently when I fired up ollama on my refurbished Thinkpad -- a laptop that doesn't even have a GPU. All of the hype had me convinced that I couldn't run any of this LLM stuff at home!
It's a little bit slower, but while I wait for the text to generate I have another cup of coffee.
Sometimes I even prompt myself to generate some text while I'm waiting.
- heiploy 2y agotraining is the phase that needs all that compute
- evbogue 2y agoThis is good to know. I had read somewhere (that was probably on the Internet) that every time I submitted a prompt at the Meta AI web site that I was vaporizing an entire bottle of water, so imagine how thrilled I was to be saving so much water by prompting AI at home! But alas, the water was already vaporized. The climate? Already changed.
- gloflo 2y agoNope, climate is changing to even worse. It's not a "oops, OK now we live with this new reality" but "oh fuck, the rollercoaster is getting steeper AND is accelerating more and more, the breaks are lose and we already lost half of the wagons".
- evbogue 2y agoMaybe with enough H100s we can next word predict a solution to this global issue.
- m3kw9 2y agoCurrent 1b model will do you no good, just rotate through all the free stuff and it would cover most of you usecases
- evbogue 2y agoI will admit that Llama3.1 70B does make my old Thinkpad pretty cranky. But winter is coming, so if I can change the climate of my bedroom while I'm waiting that's always a bonus this time of year.
- Sohcahtoa82 2y agoHeh, back in 2014, I heated my room with an AMD R9 290 by mining crypto. My cat loved it, too. She'd lay on my desk right behind my computer and get blasted by the heat. I was in an apartment that used resistive heat, so the crypto I mined was effectively free since energy consumed by my GPU meant using the heater less.
- evbogue 2y agoThis is begging for a distributed work algorithm that favors GPUs in cold bedrooms to render your next greatest hallucination.