3 ms·
Has anyone actually used GPT4? It's not "cheap". It was roughly $150 for me to build a small dataset with a few thousand quarter-page chunks of text for a data
by oceanplexian 3y ago
Has anyone actually used GPT4? It's not "cheap".
It was roughly $150 for me to build a small dataset with a few thousand quarter-page chunks of text for a data project using GPT4. GPT3 is substantially cheaper but it would hallucinate 30% of the time; honestly a nice fine-tune of LlaMA is on-par with GPT3 and after the sunk cost all it costs is a few $0.01 in electricity to generate the same sized dataset.
- slowhadoken 3y agoIt's insanely expensive to run and operate "AI". Meredith Whittaker's talk on AI is very insightful https://www.youtube.com/watch?v=amNriUZNP8w https://www.youtube.com/watch?v=amNriUZNP8w
- redox99 3y agoEven GPT3.5 can be much more expensive. In some specific tasks, a finetuned 7B llama can work as well as GPT3.5. You can rent a 3090 at $0.20/h on vast.ai, or $0.40/h on runpod. Using VLLM at 400t/s that's 1440000 generated tokens. Generating that amount of tokens with GPT3.5 would be $2.88.
- agnokapathetic 3y ago> "In some specific tasks, a finetuned 7B llama can work as well as GPT3.5." "some" is doing a lot of heavy lifting here. Also: don't discount the labor cost of curating a fine tuning dataset, running a FT training run, even if the hardware is cheap.
- moffkalast 3y agoUnless your use case isn't in English in which case LLama is as useful as a one-legged man in an ass-kicking contest. LLama models only really shine for things that GPTs would refuse to even consider because of corporate RLHF, and if you need to keep your data local I suppose. For the rest they're second rate at best.
- deleted 3y ago[deleted]