4 ms·

OpenAI APIs for GPT and Dalle have issues like non determnism, and their special prompt injection where they add stuff or modify your prompt (with no option to

OpenAI APIs for GPT and Dalle have issues like non determnism, and their special prompt injection where they add stuff or modify your prompt (with no option to turn that off. Makes it impossible to do research or to debug as a developer variations of things.

throwaway314155 2y ago
While that's true for their ChatGPT SaaS, the API they provide doesn't impose as many restrictions.
- simion314 2y ago
  >While that's true for their ChatGPT SaaS, the API they provide doesn't impose as many restrictions. There are same issues with GPT API, 1. non reproducible is there in the API 2. even after we ensure we do a moderation check on the input prompt, soemtimes GPT will produce "unsafe" output and accuse itself of "unsafe" stuff and we get an error but we pay for GPT "un-safeness" IMO if the GPT is producing unsafe stuff then I should not pay for it's problems. 3. dalle gives no seed so no reproducible, and no option to opt out on their GPT modifying the prompt , so images are sometimes absurdly enhanced with extreme amount of details or extreme diversity, so you need to fight against their GPT enhancing. What extra option we have with the APIs that is useful ?
  - SoothingSorbet 2y ago
    > 1. non reproducible is there in the API It should be reproducible if you set the temperature to 1, have you tried that?
  - throwaway314155 2y ago
    Respectfully, this just seems like a few reasons LLM's are frustrating at the moment. Having said that, there is indeed a seed and temperature parameter in the chat/assistant API that will enable (much stronger) determinism. The reason it's not 100% guaranteed to be deterministic is because they may run their model across different hardware, and hardware-level mistakes may accumulate. With regard to DALLE - that's a fair complaint I didn't realize they don't have a seed for their API. You should really try switching to an open model if you can. You'll have complete control. I recommend flux-schnell or flux-dev.

throwaway314155 2y ago

While that's true for their ChatGPT SaaS, the API they provide doesn't impose as many restrictions.

simion314 2y ago
>While that's true for their ChatGPT SaaS, the API they provide doesn't impose as many restrictions. There are same issues with GPT API, 1. non reproducible is there in the API 2. even after we ensure we do a moderation check on the input prompt, soemtimes GPT will produce "unsafe" output and accuse itself of "unsafe" stuff and we get an error but we pay for GPT "un-safeness" IMO if the GPT is producing unsafe stuff then I should not pay for it's problems. 3. dalle gives no seed so no reproducible, and no option to opt out on their GPT modifying the prompt , so images are sometimes absurdly enhanced with extreme amount of details or extreme diversity, so you need to fight against their GPT enhancing. What extra option we have with the APIs that is useful ?
- SoothingSorbet 2y ago
  > 1. non reproducible is there in the API It should be reproducible if you set the temperature to 1, have you tried that?
- throwaway314155 2y ago
  Respectfully, this just seems like a few reasons LLM's are frustrating at the moment. Having said that, there is indeed a seed and temperature parameter in the chat/assistant API that will enable (much stronger) determinism. The reason it's not 100% guaranteed to be deterministic is because they may run their model across different hardware, and hardware-level mistakes may accumulate. With regard to DALLE - that's a fair complaint I didn't realize they don't have a seed for their API. You should really try switching to an open model if you can. You'll have complete control. I recommend flux-schnell or flux-dev.