6 ms·
> unlimited higher-speed GPT-4 access aka the nerfed version. high speed means the weights were relaxed leading to faster output but worse reasoning and memory
by Racing0461 3y ago
> unlimited higher-speed GPT-4 access
aka the nerfed version. high speed means the weights were relaxed leading to faster output but worse reasoning and memory.
- nostrebored 3y agoOr it means that the compute on the inference nodes is more efficient? Or that it’s tenanted in a way that decreases oversaturation? Or you’re getting programmatic improvements in the inference layer that are being funded by the enterprise spend?
- sebzim4500 3y agoIf they had a code improvement that made inference faster without damaging capability they would roll it out everywhere. Compute is money, after all. Worst case just add a `sleep()` to the non-enterprise version.
- bagels 3y agoDo you have any references on this? I have only seen a lot of speculation.
- Racing0461 3y agoIt's been discussed on twitter and /r/chatgpt but i've noticed it myself. I always find it funny when people say chatgpt hasn't changed since launch when i see it with my own eyes. > The party told you to reject the evidence of your eyes and ears. It was their final, most essential command
- bagels 3y agoI'm most curious about this: "weights were relaxed" Is that something you've seen with your own eyes? What does it even mean, how did you observe it? Seems hard to verify without proprietary information, if it even means something to begin with.
- slsii 3y agoWhat does it mean to “relax” weights and how does that speed up output?
- sigotirandolas 3y agoI assume he means quantization (e.g. scaling the weights from 16-bit to 4-bit) and it speeds up the output by reducing the amount of work done.
- GaggiX 3y agoOr they have the priority on high-end hardware or even dedicated one.