3 ms·
Ye ... with "just like this" I actually mean "proficient with compiling C projects on an Unixy system" which is like years of dev, "power user" or admin experie
by rightbyte 3y ago
Ye ... with "just like this" I actually mean "proficient with compiling C projects on an Unixy system" which is like years of dev, "power user" or admin experience (not being sarcastic here). For the audience here I would say that "just" is about right though.
I have no clue about anything LLM related. I just made it run after reading some comment on HN pointing in its direction.
My point is that these locally run LLMs seems way "underreported". I even tried to make a "Show HN" post about it but it got zero interest.
But maybe I am missing something?
- ilaksh 3y agoThe Makefile for llama.cpp is really good and doesn't require hacking to make it work.
- mitthrowaway2 3y agoEven on Windows?
- marcosdumay 3y agoIt's probably a hell to run it on Windows. But again, for the audience here, that's not expected to be a large roadblock.
- derac 3y agoIt's actually really easy to run this stuff on Windows. I've run oogabooga, koboldcpp, tabbyapi, and more with no issues.
- benlivengood 3y agoIt builds on Windows with a couple tweaks to the includes with mingw64, I think some of the Windows-related includes need to be explicitly included. This was a few months ago and things may be better now.
- ruune 3y agoWith almost every AI story I read on here (granted, I don't read that many, I find them quite boring), the top comment seems to be something like "you can do the same on your machine, see ollama". Case in point, your comment was the first one listed for me. So the interest for running LLMs on here seems quite high. Don't know why your post didn't catch anyone.