4 ms·

I actually read the (first half) of the paper. I can't speak to whether or not Nature should publish this, but I don't find the outcomes in the paper spectacul

I actually read the (first half) of the paper. I can't speak to whether or not Nature should publish this, but I don't find the outcomes in the paper spectacular. In brief, they create an ontology -- in the initial examples, four made up color words, each corresponding to a real color and then three made up 'function' words, representing 'before', 'in the middle of' and 'triple'. They then ask humans and their AI to generate sequences of colors based on short and progressively longer strings of color words and functions. The AI is about as good at this as humans, which is to say 85% successful for instructions longer than three or four words. The headline says that GPT4 is worse at this than MLC. I am doubtful about this claim. I feel a quality prompt engineer could beat 85% with GPT4. The claims in the document are that MLC shows similar error types to humans, and that this backs some sort of theory of mind I don't know anything about. That's as may be. I would be surprised if this becomes a radical new architecture based on what I'm reading. I couldn't find a lot of information as to size of the network used; I suppose if its very small that might be interesting. But this read to me very like an 'insider' paper, its target other academics who care about some of these theories of mind, not people who need to get sentences like 'cax kiki pup wif gep' turned into real color circles right away.

SirMaster 3y ago
If I am evaluating how intelligent an AI is against a human then IMO it's only fair they both get the same prompt. It should be counted as a fault of the AI if it can't understand the prompt as well as the human, or another AI. That's all part of being a useful entity, how well you understand a prompt and how well you can infer the original meaning from what prompt you have been given.
famouswaffles 3y ago
>I feel a quality prompt engineer could beat 85% with GPT4. I generally don't pull the "you're not prompting it good enough" card unless i have direct experience with the task but this does raise a few flags - "between 42 and 86% of the time, depending on how the researchers presented the task." I would have liked them to show how they presented this task, at least something more than a single throw away line, given how integral it is to the main claim.
- randcraw 3y ago
  The article suggested that GPF-4 FAILED between 42 and 86 percent of the time. So its success rate would be 58 to 14 percent, which compared to the novel NN's 80 percent, seems significant.
  - famouswaffles 3y ago
    Yes I'm aware. My point is that 1. The level of variance displayed here is very atypical for a language model unless you're making significant changes to how the information is presented. This alone is cause for more elaboration. 2. With this big a variance already showing, it's hard to say for sure that they hit the top end. Maybe it seems strange but most LLM evaluation papers still don't actually bother to tie in even some extremely well known/basic "output boosters" like chain of thought and so on. But at least in these instances, we know how they presented it. a "this is how we presented the task, this is how it performed" would have been very nice.
- PumpkinSpice 3y ago
  I don't find this argument persuasive. If you need to engineer the prompt in order for the model to give you the answer you want, and if it's performing poorly without that kind of assistance from a human who understands the task and the desired outcome... then it's perfectly OK to not bother and give the LLM a low mark on a benchmark. After all, the whole point was to test the LLM's language and reasoning skills. "Prompt engineering" makes sense if your goal is more utilitarian, but it's not a license to cheat on tests. If you want a machine learning model to generate aesthetically pleasing images, or to follow certain rules in a conversation, then it's OK to rely on hacky solutions to get there.
  - famouswaffles 3y ago
    I kind of dislike the term "Prompt Engineering". It mystifies what to me is an ordinary process. I'm not talking about chanting magic words, i'm just talking about taking the structure of an LLM into account, even taking how humans would approach the problem into account. In microsoft's agi paper, there is a test for planning they put out for GPT-4. They give detailed instructions on the constraint of a poem and expect it to spit out the constrained poem in one shot. Of course it fails and the conclusion is that it can't plan. But if you think about it, this is a ridiculous assertion. No human on earth would have passed that test with working memory alone. Even by our standards, it's a weird way to present the information but they did so anyway. This is actually a major issue for most plan benchmark assessment for LLMs i've come across. These are the kind of problems i'm talking about. If i changed the request to encourage for a draft/revise generation process and it consistently passed these tests then it's very fair to say it can plan. That's not "hacky". It's just truth. and saying otherwise is denying real world usage for a misplaced sense of propriety to benchmarks. a benchmark is only as useful as the capabilities it can assess. a "this is how this is presented, this is how the model performed" would have been very prudent in my opinion. Maybe these guys already covered the kind of things i'm talking about(i can accept that!)....but i can't know that if they don't tell us.
b_and_again 3y ago
[dead]
avgcorrection 3y ago
That’s seriously it? What an insulting (to the reader) headline by Nature.
8bitsrule 3y ago
>I couldn't find a lot of information as to size of the network used In the original paper,[0] in the section called 'Implementation of MLC', there's a description of sorts (Greek to me): "Both the encoder and decoder have 3 layers, 8 attention heads per layer, input and hidden embeddings of size 128, and a feedforward hidden size of 512. Following GPT63, GELU64 activation functions are used instead of ReLU. In total, the architecture has about 1.4 million parameters." [0] https://www.nature.com/articles/s41586-023-06668-3 https://www.nature.com/articles/s41586-023-06668-3
sashank_1509 3y ago
Honestly I’m surprised by how low Nature has fallen. Is it even a useful signal of paper quality anymore? NeurIPS and ICLR aren’t great but in general I’ve found their work to be more rigorous than that of Nature despite of the fact that they are shorter conference papers compared to Nature’s journal papers. Right now sadly the only useful signal in Deep Learning research is research group. If OpenAI releases a paper I know it’s something good that works at scale, similarly if Kaiming He, Piotr Dollar and team in Meta AI release a paper, it tends to be really good and SOTA. Google Deepmind maintains high quality, Google brain has been more of a mixed bag. If Berkeley releases a paper, it has 50% chance of directly going to trash, Stanford has a much lower percent (I also disambiguate) based on specific groups. Of course I’m going to be heavily biased and this system is not great, but conferences and journals have managed to become such a useless signal that I find this method to be more accurate.
tokai 3y ago
>The claims in the document are that MLC shows similar error types to humans, and that this backs some sort of theory of mind The same thing was claimed with latent semantic analysis in the late 90s.
jug 3y ago
But isn’t that their point? That their new neural network needs less prompt crafting because it is part better at reading intent and meaning of words and part less likely to hallucinate? Sure, I also think GPT-4 can be brute forced with a very precise prompt into generating correct answers but that’s not how you would describe the problem to a human, and I assume the humans performing well in this test weren’t either, but given a fairly informal description of the exercise.

SirMaster 3y ago

If I am evaluating how intelligent an AI is against a human then IMO it's only fair they both get the same prompt. It should be counted as a fault of the AI if it can't understand the prompt as well as the human, or another AI. That's all part of being a useful entity, how well you understand a prompt and how well you can infer the original meaning from what prompt you have been given.

famouswaffles 3y ago

>I feel a quality prompt engineer could beat 85% with GPT4. I generally don't pull the "you're not prompting it good enough" card unless i have direct experience with the task but this does raise a few flags - "between 42 and 86% of the time, depending on how the researchers presented the task." I would have liked them to show how they presented this task, at least something more than a single throw away line, given how integral it is to the main claim.

randcraw 3y ago
The article suggested that GPF-4 FAILED between 42 and 86 percent of the time. So its success rate would be 58 to 14 percent, which compared to the novel NN's 80 percent, seems significant.
- famouswaffles 3y ago
  Yes I'm aware. My point is that 1. The level of variance displayed here is very atypical for a language model unless you're making significant changes to how the information is presented. This alone is cause for more elaboration. 2. With this big a variance already showing, it's hard to say for sure that they hit the top end. Maybe it seems strange but most LLM evaluation papers still don't actually bother to tie in even some extremely well known/basic "output boosters" like chain of thought and so on. But at least in these instances, we know how they presented it. a "this is how we presented the task, this is how it performed" would have been very nice.
PumpkinSpice 3y ago
I don't find this argument persuasive. If you need to engineer the prompt in order for the model to give you the answer you want, and if it's performing poorly without that kind of assistance from a human who understands the task and the desired outcome... then it's perfectly OK to not bother and give the LLM a low mark on a benchmark. After all, the whole point was to test the LLM's language and reasoning skills. "Prompt engineering" makes sense if your goal is more utilitarian, but it's not a license to cheat on tests. If you want a machine learning model to generate aesthetically pleasing images, or to follow certain rules in a conversation, then it's OK to rely on hacky solutions to get there.
- famouswaffles 3y ago
  I kind of dislike the term "Prompt Engineering". It mystifies what to me is an ordinary process. I'm not talking about chanting magic words, i'm just talking about taking the structure of an LLM into account, even taking how humans would approach the problem into account. In microsoft's agi paper, there is a test for planning they put out for GPT-4. They give detailed instructions on the constraint of a poem and expect it to spit out the constrained poem in one shot. Of course it fails and the conclusion is that it can't plan. But if you think about it, this is a ridiculous assertion. No human on earth would have passed that test with working memory alone. Even by our standards, it's a weird way to present the information but they did so anyway. This is actually a major issue for most plan benchmark assessment for LLMs i've come across. These are the kind of problems i'm talking about. If i changed the request to encourage for a draft/revise generation process and it consistently passed these tests then it's very fair to say it can plan. That's not "hacky". It's just truth. and saying otherwise is denying real world usage for a misplaced sense of propriety to benchmarks. a benchmark is only as useful as the capabilities it can assess. a "this is how this is presented, this is how the model performed" would have been very prudent in my opinion. Maybe these guys already covered the kind of things i'm talking about(i can accept that!)....but i can't know that if they don't tell us.

b_and_again 3y ago

[dead]

avgcorrection 3y ago

That’s seriously it? What an insulting (to the reader) headline by Nature.

8bitsrule 3y ago

>I couldn't find a lot of information as to size of the network used In the original paper,[0] in the section called 'Implementation of MLC', there's a description of sorts (Greek to me): "Both the encoder and decoder have 3 layers, 8 attention heads per layer, input and hidden embeddings of size 128, and a feedforward hidden size of 512. Following GPT63, GELU64 activation functions are used instead of ReLU. In total, the architecture has about 1.4 million parameters." [0] https://www.nature.com/articles/s41586-023-06668-3 https://www.nature.com/articles/s41586-023-06668-3

sashank_1509 3y ago

Honestly I’m surprised by how low Nature has fallen. Is it even a useful signal of paper quality anymore? NeurIPS and ICLR aren’t great but in general I’ve found their work to be more rigorous than that of Nature despite of the fact that they are shorter conference papers compared to Nature’s journal papers. Right now sadly the only useful signal in Deep Learning research is research group. If OpenAI releases a paper I know it’s something good that works at scale, similarly if Kaiming He, Piotr Dollar and team in Meta AI release a paper, it tends to be really good and SOTA. Google Deepmind maintains high quality, Google brain has been more of a mixed bag. If Berkeley releases a paper, it has 50% chance of directly going to trash, Stanford has a much lower percent (I also disambiguate) based on specific groups. Of course I’m going to be heavily biased and this system is not great, but conferences and journals have managed to become such a useless signal that I find this method to be more accurate.

tokai 3y ago

>The claims in the document are that MLC shows similar error types to humans, and that this backs some sort of theory of mind The same thing was claimed with latent semantic analysis in the late 90s.

jug 3y ago

But isn’t that their point? That their new neural network needs less prompt crafting because it is part better at reading intent and meaning of words and part less likely to hallucinate? Sure, I also think GPT-4 can be brute forced with a very precise prompt into generating correct answers but that’s not how you would describe the problem to a human, and I assume the humans performing well in this test weren’t either, but given a fairly informal description of the exercise.