4 ms·

He, imo correctly, puts them both in the category of extra degrees of freedom that can allow the user to overfit and get results that appear more impressive tha

He, imo correctly, puts them both in the category of extra degrees of freedom that can allow the user to overfit and get results that appear more impressive than the underlying reality about how the model has generalized.

jameshart 3y ago
But generating new prompts for GPT is incredibly cheap! You just rephrase and ask again. That there are prompts which generate impressive results with GPT is the point. Because anyone can generate prompts - and get impressive results. Whereas hyper parameter tuning is expensive. A system that generates good results with the right tuning doesn’t tell you much about your ability to use it to generate good results, because it will be hard to try lots of different tuning approaches to discover if you can get a useful result. These things seem not remotely comparable.
- akkartik 3y ago
  Think about when you use GPT to generate prompts, something that seems to be growing common. Once you have a pipeline like that, changing the (meta)prompt can be expensive. You designed the pipeline thinking you only have to come up with the meta-prompt once, and it would work from now on. But you find you have to keep tuning your Kubernetes/upgrading your dependencies/tweaking your meta prompt. "Scam" is overblown, but I think OP is right to warn of a possible future issue. It's an issue endemic to all software, so not something worth calling out recent AI advances in particular for. But it seems something we should all be trying to get better about. Monitor the ongoing costs of the systems we create. Do they really pay for themselves or are we waiting for a Godot that never arrives?
ChainOfFools 3y ago
Prompt engineering rather resembles the mentalists' trick of cold reading their subject, turned inside out.
adamnemecek 3y ago
All systems are like that.
- quickthrower2 3y ago
  That is quite an insight. A "programmer" is really an "overfitter".
  - version_five 3y ago
    That's not an insight, it's a misunderstanding. Overfitting is only applicable relative to claims of statistical performance. And in any event, there are lots of systems with fewer degrees of freedom (or in the case of deep learning, more generalization potential) than the training data, that are not at particular risk of being overfit, and there are measures and tests to mitigate the risk of overfitting. It's not some inherent characteristic of "systems".

jameshart 3y ago

But generating new prompts for GPT is incredibly cheap! You just rephrase and ask again. That there are prompts which generate impressive results with GPT is the point. Because anyone can generate prompts - and get impressive results. Whereas hyper parameter tuning is expensive. A system that generates good results with the right tuning doesn’t tell you much about your ability to use it to generate good results, because it will be hard to try lots of different tuning approaches to discover if you can get a useful result. These things seem not remotely comparable.

akkartik 3y ago
Think about when you use GPT to generate prompts, something that seems to be growing common. Once you have a pipeline like that, changing the (meta)prompt can be expensive. You designed the pipeline thinking you only have to come up with the meta-prompt once, and it would work from now on. But you find you have to keep tuning your Kubernetes/upgrading your dependencies/tweaking your meta prompt. "Scam" is overblown, but I think OP is right to warn of a possible future issue. It's an issue endemic to all software, so not something worth calling out recent AI advances in particular for. But it seems something we should all be trying to get better about. Monitor the ongoing costs of the systems we create. Do they really pay for themselves or are we waiting for a Godot that never arrives?

ChainOfFools 3y ago

Prompt engineering rather resembles the mentalists' trick of cold reading their subject, turned inside out.

adamnemecek 3y ago

All systems are like that.

quickthrower2 3y ago
That is quite an insight. A "programmer" is really an "overfitter".
- version_five 3y ago
  That's not an insight, it's a misunderstanding. Overfitting is only applicable relative to claims of statistical performance. And in any event, there are lots of systems with fewer degrees of freedom (or in the case of deep learning, more generalization potential) than the training data, that are not at particular risk of being overfit, and there are measures and tests to mitigate the risk of overfitting. It's not some inherent characteristic of "systems".