5 ms·

Agree with this except for one data point. OpenAI does enhance/tweak/do something with the models at different levels. This can be determined by: 1. A change i

Agree with this except for one data point. OpenAI does enhance/tweak/do something with the models at different levels. This can be determined by: 1. A change in the current model number (eg. gpt-3.5-turbo-0613) 2. On ChatGPT UI, the date at the bottom (eg. August 2023) So it isn’t correct to say “it is incredibly obvious that nothing has happened”. Not that obvious to me. A bit like how you can never tell for sure if Coca Cola has tweaked their formula, or McDonalds has changed the recipe for its signature sauce. Only in this case, the model number going up or the date becoming more recent leads credence to something having changed.

sigmoid10 3y ago
The ChatGPT UI is indeed a wildcard. But it is irrelevant here because according to the github repo this page queries the API and OpenAI guarantees it doesn't change models with version number information (like gpt-4-0613, which is mentioned in the latest images). So this "experiment" would make a lot more sense if it was only run once every few months when the API actually offers new model versions and then generate a bunch of images for every model, instead of generating one single image every day (which is meaningless due to non-deterministic noise, even if the model had somehow changed since yesterday). That is also how it was done in the original study during the development of GPT4. I don't know how this experiment came up with its logic, unless they had a gross misunderstanding about how these models actually work (which admittedly seems common among tech interested folks here).
- imdsm 3y ago
  When I last spoken to Logan, he confirmed that there are no changes between the API models, so 0314 and 0613 are it. That's two models so far that I've collected SVGs for. With regard to batch vs daily -- it makes no different in terms of output, but by going daily, I don't need to track model changes and generate a new batch. Also it's fun to see each daily unicorn.
- sheepscreek 3y ago
  That’s a good summary and it makes a lot of sense.

sigmoid10 3y ago

The ChatGPT UI is indeed a wildcard. But it is irrelevant here because according to the github repo this page queries the API and OpenAI guarantees it doesn't change models with version number information (like gpt-4-0613, which is mentioned in the latest images). So this "experiment" would make a lot more sense if it was only run once every few months when the API actually offers new model versions and then generate a bunch of images for every model, instead of generating one single image every day (which is meaningless due to non-deterministic noise, even if the model had somehow changed since yesterday). That is also how it was done in the original study during the development of GPT4. I don't know how this experiment came up with its logic, unless they had a gross misunderstanding about how these models actually work (which admittedly seems common among tech interested folks here).

imdsm 3y ago
When I last spoken to Logan, he confirmed that there are no changes between the API models, so 0314 and 0613 are it. That's two models so far that I've collected SVGs for. With regard to batch vs daily -- it makes no different in terms of output, but by going daily, I don't need to track model changes and generate a new batch. Also it's fun to see each daily unicorn.
sheepscreek 3y ago
That’s a good summary and it makes a lot of sense.