5 ms·
> Cheaper and faster GPT-4 — This is their top priority. In general, OpenAI’s aim is to drive “the cost of intelligence” down as far as possible and so they wil
by purplecats 3y ago
> Cheaper and faster GPT-4 — This is their top priority. In general, OpenAI’s aim is to drive “the cost of intelligence” down as far as possible and so they will work hard to continue to reduce the cost of the APIs over time.
this certainly aligns with the massive (albeit subjective and anecdotal) degradation in quality i've experienced with ChatGPT GPT-4 over the past few weeks.
hopefully a superior (higher quality) alternative surfaces before its unusable. i'm not considering continuing my subscription at this rate.
- nonethewiser 3y agoI wonder of it actually is because they’re tuning it to make it less offensive (by their standards). Thats the only explanation I keep seeing repeated.
- reaperman 3y agoI would be very surprised. Things that are very, very far from that are also much worse. I'm having difficulty finding the difference between GPT-3.5 and GPT-4 for a lot of my programming tasks lately. It's noticeably degraded.
- littlestymaar 3y agoThat's a convenient explanation that's been repeated over and over by certain people, but cost is a much more likely explanation: inference for large models is extraordinarily expensive when you have millions of users and their pricing model always seemed way too low to pay for that. They have likely been subsidizing their users since the launch of their commercial offering (and this is pretty common strategy for SV startups) but they've been so successful that they now need to scale the cost down in order not to burn all their cash too fast.
- brucethemoose2 3y agoAnthropic's Claude is said to be very good. Instruction tuned LLaMA 65B/Falcon 40B are good, especially with an embeddings database. ...But OpenAI has all the name recognition and ease of use now, so it might not even matter if others ambiguously surpass OpenAI models.
- timeserious 3y agoCan you ELI5 why an embeddings database helps here? Can pinecone/milvus be used to 'extend memory' of OSS and vendor LLMs without retraining?
- brucethemoose2 3y agoFirst some context: llm "prompts" are actually the whole conversation + initial context. They learn nothing, hence the whole conversation gets fed into them every time, but the instruction following ones are trained to answer your most recent chat response. In a nutshell, part of your llm prompt (usually your most recent question?) gets fed as a query for the embedding/vector database. It retrieves the most "similar" entries to your question (which is what an embedding database does), and that information is pasted into the context of the llm. Its kinda like pasting the first entry from a local Google search into the beginning of your question as "background." Some implementations insert your old conversations (that are too big to fit into the llm's context window) into the database as they are pushed out. This is what I have seen, anyway. Maybe some other implementations do things better.
- thomasahle 3y ago> part of your llm prompt (usually your most recent question?) gets fed as a query for the embedding/vector database How is it embedded? Using a separere embedding model, like Bert or something? Or do you use the LLM itself somehow? Also, how do you create content for the vector database keys themselves? Also just some arbitrary off the shelf embedding? Or do you train it as part of training the LLM?
- brucethemoose2 3y agoYeah its completely seperate. The LLM just gets some extra text in the prompt, that is all. The text you want to insert is "encoded" into the database which is not particularly compute expensive. You can read about one such implementation here: https://github.com/chroma-core/chroma https://github.com/chroma-core/chroma
- rafark 3y agoExactly. It’s become ‘cheap’. This is why we need more good competition.
- ChatGTP 3y agoShould "intelligence" have ever costed anything? It's like saying "air should cost money".
- theonlybutlet 3y agoHas anyone compared the ChatGPT GPT-4 performance in the plus subscription to that of the API? Has the API performance deteriorated just as much? It would be strange if it did as I'd assume the models costs are priced in there.
- laratiede 3y agoPeople state this as if it is fact when there is no good way to measure this. I have had random runs of good days and bad days since starting to use chatGPT.