3 ms·

In my experience, these updates don't matter that much. For example, GPT-4 still uses Pydantic v1 syntax (which for the most part is deprecated) even though Pyd

In my experience, these updates don't matter that much. For example, GPT-4 still uses Pydantic v1 syntax (which for the most part is deprecated) even though Pydantic v2 came in June 2023. the information regarding v1 is so much more in GPT-4's training data that "on average" it picks that instead of the new v2 syntax data. This has been a major pain point for me, because GPT-4 constantly uses @validator, "const", "always", etc. features that don't work on v2.

Der_Einzige 3y ago
This is why with future models and their effectively unlimited context size you'll simply concatenate the entire API codebase you're using into the context of your prompt.
- influx 3y ago
  I've done similar already, but ChatGPT still tends to rewrite the code to the old API, even if it's correctly using the new API. Ironically, I just had this happen to me while using ChatGPT 4 to use the OpenAI API.
- vitorsr 3y ago
  As far as I can tell, pricing per token is not going away anytime soon.
  - joshspankit 3y ago
    At 100k and above price will naturally come down to be equivalent
- blharr 3y ago
  Although the attention is really good at focusing on specific information, I've had problems with this approach. It seems like if I ask a task and provide a large enough API it gets distracted or ignores the chunk of text and does its own thing anyway. Also it's still more expensive to increase the context length
SushiHippie 3y ago
GPT-4 knowledge cutoff is september 2021. GPT-4-Turbo knowledge cutoff was April 2023. Both models were not able to have any pydantic v2 training data.
swyx 3y ago
have you tried adding a clause that all output must come in pydantic v2 in your prompt? additionally you can add a few common fewshot examples to get it in the pydantic v2 state
- behnamoh 3y ago
  I tried exporting the Pydantic documentation page on syntax changes as PDF, feeding that to GPT-4, and then telling it to be careful about Pydantic syntax changes. That helped it use the new syntax, but we can't feed in the entire Pydantic doc so it still didn't get the details right.
fatkam 3y ago
In my experience, updates decrease quality because of ever increasing censorship. Improvements are negated really quickly.

Der_Einzige 3y ago

This is why with future models and their effectively unlimited context size you'll simply concatenate the entire API codebase you're using into the context of your prompt.

influx 3y ago
I've done similar already, but ChatGPT still tends to rewrite the code to the old API, even if it's correctly using the new API. Ironically, I just had this happen to me while using ChatGPT 4 to use the OpenAI API.
vitorsr 3y ago
As far as I can tell, pricing per token is not going away anytime soon.
- joshspankit 3y ago
  At 100k and above price will naturally come down to be equivalent
blharr 3y ago
Although the attention is really good at focusing on specific information, I've had problems with this approach. It seems like if I ask a task and provide a large enough API it gets distracted or ignores the chunk of text and does its own thing anyway. Also it's still more expensive to increase the context length

SushiHippie 3y ago

GPT-4 knowledge cutoff is september 2021. GPT-4-Turbo knowledge cutoff was April 2023. Both models were not able to have any pydantic v2 training data.

swyx 3y ago

have you tried adding a clause that all output must come in pydantic v2 in your prompt? additionally you can add a few common fewshot examples to get it in the pydantic v2 state

behnamoh 3y ago
I tried exporting the Pydantic documentation page on syntax changes as PDF, feeding that to GPT-4, and then telling it to be careful about Pydantic syntax changes. That helped it use the new syntax, but we can't feed in the entire Pydantic doc so it still didn't get the details right.

fatkam 3y ago

In my experience, updates decrease quality because of ever increasing censorship. Improvements are negated really quickly.