4 ms·
I see this statement a lot and have no idea how people come to this conclusion. I have a beefy 16k$ workstation with 2 4090s and I could barely run the LLAMA 65
by sashank_1509 3y ago
I see this statement a lot and have no idea how people come to this conclusion. I have a beefy 16k$ workstation with 2 4090s and I could barely run the LLAMA 65B model at a very slow pace. Let us say we do have the model weights to GPT-4 and GPT3.5, me as the average consumer I don't know how this helps me in any way. I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even inference, and even then it will be a slow, unpolished experience.
On the other hand OpenAI’s API makes things blazingly fast and dirt cheap to the average consumer. It honestly does feel like they have enabled the power of AI to be accessible to anyone with a laptop. If that requires fending off competition from Behemoths like Google, Meta by not releasing model weights then so be it. This critique would be more apt to Nvidia who are artificially increasing datacenter GPU prices thus pricing out the average consumer. OpenAI is doing the opposite.
- hackernewds 3y agoAn "average" person is not someone who knows how to call an API. Perhaps only on HN
- IMTDb 3y agoThen an "average" person is certainly not someone who is able to download and run an LLM on their device.
- rolisz 3y agoIf they don't know how to call an API, they won't know how to run local models (at the moment it's quite a pain to set-up all the dependencies)
- bioemerl 3y agoIt's actually not bad, the hard part is getting the hardware. Kobold will install itself most of the time with a double click.
- theshrike79 3y ago"AI" as we know it is hardly 6 months old now, just wait a while and it'll be grandma accessible.
- esafak 3y agoYou exaggerate a bit! Machine learning and language models have been around for decades. OpenAI itself has been around since 2015.
- wellthisisgreat 3y ago> LLAMA 65B model at a very slow pace How does it compare to GPT 3.5, or 4? I mean if you ask the same questions. Is it usable at all? I tried the models that work with 4090 and they were completely useless for anything practical (code questions, etc.). Curiosities sure, but on Eliza level.
- kgwgk 3y agoIs there a simple question / answer that you would find illuminating?
- wellthisisgreat 3y agothe one that I used for GPT-4 and the local ones was a bit obscure: "how to configure internal pull-up resistor on PCA9557 from NXP in firmware" the GPT4 would give a paragraph of > The PCA9557 from NXP is an 8-bit I/O expander with an I2C interface. This device does not have an internal pull-up resistor on its I/O pins, but it does have software programmable input and output states. and then write a somewhat meaningful code. the local LLMs failed even at the paragraph stage could you try that?
- deleted 3y ago[deleted]
- bioemerl 3y ago> . I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even inference Give it a decade and you might be able to, but without the model you'll never have the option.
- ilaksh 3y agoI have been thinking about trying to do LoRA style fine tuning of Flacon-40b or Falcon-7b on RunPod. The new OpenAI 16k context and functions thinking made me lose the urge to get into that. Was questionable whether it could really write code consistently anyway even if very well fine tuned. But at least that is something that can be attempted without $25k.