4 ms·

I have to admit, that discussing this topic is forcing a precision in the vocabulary used to discuss reason. However, it is a TEDIOUS process. It’s simplest to

I have to admit, that discussing this topic is forcing a precision in the vocabulary used to discuss reason. However, it is a TEDIOUS process. It’s simplest to just make something - the parallel prompts scenario for example. Can you leave your LLMs to their own business, and will they have a workable product at the end of it? As you said it would seem as if they have the “knowledge” of the task. It should not be an issue. However you will not get anywhere. The issue isnt in the LLM - the issue is in misunderstanding what is going on, and therefore expectations. LLMs don’t notice things - in the human manner you assumed they do. It isnt point out gaps, its repeating text patterns. Our habit of dealing with humans is filling in these gaps and supporting an assumption of ‘noticing’ or ‘improvement’. I think it is hard for us to consider changes in text, without seeing the changes in meaning as well. With LLMs you have to accept that it’s not seeing meaning, it’s just seeing correlation.

resters 3y ago
> LLMs don’t notice things - in the human manner you assumed they do. It isn't point out gaps, its repeating text patterns. Our habit of dealing with humans is filling in these gaps and supporting an assumption of ‘noticing’ or ‘improvement’. True. It is important to avoid anthropomorphizing LLMs, etc. I do this intentionally now and then but I agree it is dangerous to do it accidentally. > I think it is hard for us to consider changes in text, without seeing the changes in meaning as well. True, but since LLMs were trained on text sequences that had meaning, much of the meaning was accidentally embodied in the resulting model. Areas where LLMs seem to reason well happen to be the areas where the training data was sufficiently generalized and the language tokens used similarly enough... such as recipes. > With LLMs you have to accept that it’s not seeing meaning, it’s just seeing correlation. True. I think it is interesting how much it often feels like knowledge. I think we also have to be careful not to overly glorify human "knowledge" as something other than producing a pattern of output signals in response to a pattern of input signals.
- intended 3y ago
  I agree. I think precision in all cases makes it possible to pursue the technology, as opposed to almost dabbling in superstition and philosophy. However, it’s taken an inordinate amount of time to achieve even the finesse shared in this comment chain. It’s a challenging topic. We have to carve out levels of utility for systems that generate output given inputs. Some way to distinguish between what our wetware achieves, and what LLMs achieve - Without putting our models on a pedestal.

resters 3y ago

> LLMs don’t notice things - in the human manner you assumed they do. It isn't point out gaps, its repeating text patterns. Our habit of dealing with humans is filling in these gaps and supporting an assumption of ‘noticing’ or ‘improvement’. True. It is important to avoid anthropomorphizing LLMs, etc. I do this intentionally now and then but I agree it is dangerous to do it accidentally. > I think it is hard for us to consider changes in text, without seeing the changes in meaning as well. True, but since LLMs were trained on text sequences that had meaning, much of the meaning was accidentally embodied in the resulting model. Areas where LLMs seem to reason well happen to be the areas where the training data was sufficiently generalized and the language tokens used similarly enough... such as recipes. > With LLMs you have to accept that it’s not seeing meaning, it’s just seeing correlation. True. I think it is interesting how much it often feels like knowledge. I think we also have to be careful not to overly glorify human "knowledge" as something other than producing a pattern of output signals in response to a pattern of input signals.

intended 3y ago
I agree. I think precision in all cases makes it possible to pursue the technology, as opposed to almost dabbling in superstition and philosophy. However, it’s taken an inordinate amount of time to achieve even the finesse shared in this comment chain. It’s a challenging topic. We have to carve out levels of utility for systems that generate output given inputs. Some way to distinguish between what our wetware achieves, and what LLMs achieve - Without putting our models on a pedestal.