3 ms·

It's like using a hammer to turn a screw and calling it useless. To envision what a next generation model bound by the same constraints should do, it'd be to r

It's like using a hammer to turn a screw and calling it useless. To envision what a next generation model bound by the same constraints should do, it'd be to recognize that it can't count tokens and use code access to write code that solves the strawberry problem without prompting. Asked to count cells it'd be a model that could write and execute OpenCV tasks. Or to go a step further, be a multimodal model that can synthesize 10000 varations of the target cell, and finetune a model like YOLO on it autonomously. I find arguments that reduce LLMs to "It can't do the simple thing!!!!" come from people unable to apply lateral thinking to how a task can be solved.

deleted 2y ago
[deleted]
doctorpangloss 2y ago
> To envision what a next generation model bound by the same constraints should do, it'd be to recognize that it can't count tokens and use code access to write code that solves the strawberry problem without prompting. The VQA problems I'm describing can be solved seemingly in one case but not combined with counting. Counting is fundamentally challenging for sort of unknown reasons, or perhaps known to the very best labs who are trying to tackle it directly. Another POV is that the stuff you are describing is in some sense so obvious that it has been tried, no?
- BoorishBears 2y ago
  I don't get what you mean by "unknown reasons", we understand that counting tokens requires a type of introspection transformer models can't do while operating on tokens. What I described is tried, and works, but the models are still not cheap/fast/reliable enough to always do what I described for every query. The difference between what I described and directly asking the model to count is that we know the models can get cheaper, faster, and more reliable at what I described without any earth shattering discoveries Like I don't see any reason why GPT 10 will ever be able to count how many letters there are in the word strawberry without a complete paradigm shift in model building... but going from GPT 3 to GPT 4 we already got a model that can always write the dead simple code required to count it out, and the models that can do so are already getting cheaper and faster every few months without any crazy discoveries.
  - doctorpangloss 2y ago
    > What I described is tried, and works Maybe they should make you Chief Scientist at OpenAI, I hear the position is open.
    - BoorishBears 2y ago
      Don't get pissy with me because you decided to talk about something you don't understand the basics of.

deleted 2y ago

[deleted]

doctorpangloss 2y ago

> To envision what a next generation model bound by the same constraints should do, it'd be to recognize that it can't count tokens and use code access to write code that solves the strawberry problem without prompting. The VQA problems I'm describing can be solved seemingly in one case but not combined with counting. Counting is fundamentally challenging for sort of unknown reasons, or perhaps known to the very best labs who are trying to tackle it directly. Another POV is that the stuff you are describing is in some sense so obvious that it has been tried, no?

BoorishBears 2y ago
I don't get what you mean by "unknown reasons", we understand that counting tokens requires a type of introspection transformer models can't do while operating on tokens. What I described is tried, and works, but the models are still not cheap/fast/reliable enough to always do what I described for every query. The difference between what I described and directly asking the model to count is that we know the models can get cheaper, faster, and more reliable at what I described without any earth shattering discoveries Like I don't see any reason why GPT 10 will ever be able to count how many letters there are in the word strawberry without a complete paradigm shift in model building... but going from GPT 3 to GPT 4 we already got a model that can always write the dead simple code required to count it out, and the models that can do so are already getting cheaper and faster every few months without any crazy discoveries.
- doctorpangloss 2y ago
  > What I described is tried, and works Maybe they should make you Chief Scientist at OpenAI, I hear the position is open.
  - BoorishBears 2y ago
    Don't get pissy with me because you decided to talk about something you don't understand the basics of.