5 ms·
Not that it is necessarily of value, but has anyone got a LLM to run on bare metal?
by lachlan_gray 3y ago
Not that it is necessarily of value, but has anyone got a LLM to run on bare metal?
- tomrod 3y agoSome of the smaller ones, yes, the huggingface.co libraries make it pretty simple.
- kgwgk 3y ago"In computer science, bare machine (or bare metal) refers to a computer executing instructions directly on logic hardware without an intervening operating system." https://en.wikipedia.org/wiki/Bare_metal https://en.wikipedia.org/wiki/Bare_metal
- kgwgk 3y agoI know I shouldn’t question the wisdom of downvoters but… come on!
- immibis 3y agoWhat is stopping you from running llama2.c on bare metal?
- kgwgk 3y agoWould you say that running on bare metal the huggingface.co libraries - as the comment that I replied to suggested - is pretty simple?
- immibis 3y agoIt doesn't use the huggingface.co libraries?
- kgwgk 3y agolachlan_gray asked whether anyone has got a LLM to run on bare metal. tomrod replied to lachlan_gray that the huggingface.co libraries make it pretty simple. I pointed out to tomrod what is the meaning of the expression “bare metal”. I don’t understand what’s the point of your reply to me in that context.
- tomrod 3y agoI can't speak for the p___g contest others want to engage in, but why not, lets make bare metal LLMs happen! https://github.com/rreilink/PiPyOS https://github.com/rreilink/PiPyOS <-- seems like a good starting point, bare metal python. I appreciate the clarification earlier in the comment chain for what you meant by bare metal -- I had interpreted it as on-prem.
- immibis 3y agoIsn't that what this is?
- SomeRndName11 3y agoYo can try it on FreeDOS, it is very close to bare metal.