3 ms·
This is nice, and prompts a couple suggestions for all of us... regarding open source, open standards, and open platforms general. Especially when we realize t
by neilv 3y ago
This is nice, and prompts a couple suggestions for all of us... regarding open source, open standards, and open platforms general.
Especially when we realize the HN appeal of titles about LLM things that mention "open source", "run locally", "uncensored", etc.
1. It'd be good to have the top of the `README.md` and other documentation to acknowledge at least the major components that go into the black box. For example, I see no mention in this GitHub README.md of `llama.cpp`.
2. Please strongly consider making open platforms the go-to default, when you build some open source thing. For example, for some "open source" work lately, I see Apple products being made the default go-to (such as for llama.ccp, which is understandable based on a motivation of seeing what Apple Silicon could do, but then it spread). During this latest industry disruption, that's practically force-feeding network effects to a proprietary platform. For example, if we make the easy path be to use Linux, that's going get more people using Linux, and reduce people already on Linux gravitating away because all the latest black box repos people found seemed to be pushing them to used GluedClosedBook 2000. And similarly with other open/closed facets of this space.
When we're making a consumer product that necessarily has to run on a closed platform due to the nature of the product (e.g., inferencing running on a smartphone), that's a different situation.
But by default -- including for stuff run in the cloud or data center, on development workstations, and for hobby purposes -- it'd really help the sustainable/surviable openness battle to do that in established open ways.
- kelvie 3y agoYeah, this needs to be made somewhat clear that GPU support doesn't work on Linux, which makes this slow as molasses.