4 ms·

You misunderstand what a language model is. What a human can or cannot do is frankly irrelevant. A GPT (Generative pretrained Transformer) is not a simulator n

You misunderstand what a language model is. What a human can or cannot do is frankly irrelevant. A GPT (Generative pretrained Transformer) is not a simulator nor an imitator. It is a predictor. Predictors try to reverse engineer the computation that could have led to a certain output so they don't just grok what is explicitly stated in a dataset but also what is implied by its structure. As an example, In a language model trained on protein sequences alone and nothing else, you will find biological structure and function emerge in the inner layers. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 https://www.pnas.org/doi/full/10.1073/pnas.2016239118 In fact, with a capable enough architecture and sufficient data (quantity, qualit, variance), there is nothing a predictor couldn't divine with enough training. You assert >Actually generating a recipe is obviously much harder, and should be effectively impossible unless you include a humanoid robot that can cook and taste. In the pursuit of recipe prediction, a predictor will learn to internally model taste because taste is implicit in the data. You don't need the hypothetical perfect predictor to demonstrate your assertion as false because a capable predictor already does. >If you're wondering whether a task is theoretically possible for an AI, I think a good rule of thumb is to ask "could a human domain expert do this given a few days and as much research material as they need". You are wrong. Train a language model on descriptions of the functions of proteins and an equivalent protein sequence and you get a language model that can predict novel functioning sequences from function descriptions alone. https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2 Not only is this not something a human expert can achieve in a couple days, It's not something a human expert can achieve at all. It is a Super-Human ability.

uoaei 3y ago
I don't think the commenter's understanding of LLMs is relevant here. It's what the authors of TFA think LLMs are for and, arguably more importantly, what kinds of problems (nails) they think about when presented with the idea of applying LLMs (hammer). It's a bit ridiculous that this is the best they can come up with, and I don't really think the idea is worth defending, especially by telling other people that they just don't get it.
- famouswaffles 3y ago
  His/Her primary assertion is that it is impossible for a LLM to generate novel recipes without physically tasting it first. His/her assertion is wrong and one easily corrected by understanding a LLM is and does. It's not a perfect predictor so it's not impeccable for all conceivable situations but GPT-4 can already generate novel recipes that taste nice. This is not some far flung science fiction ability.
  - ska 3y ago
    This is true. On the other hand, the recipes aren't typically good, and generating novel ones isn't really what people mostly want to do... so the value seems limited.
    - famouswaffles 3y ago
      That's part of it yes but people tend to forget that the road to capable prediction is not instantaneous. It's possible the data is enough to get a good predictor but the one you tested simply wasn't good enough yet.
      - ska 3y ago
        Possibly, but the easily available data itself seems is pretty poor for systemic reasons.
  - ForestCritter 3y ago
    Salt to taste, cook until brown, kneed until the dough is no longer sticky, everything about cooking is tasting and feeling. Moreover everybody has differences in their tastes and consistency preferences. Good recipe books teach you how to use ingredients and apply that knowledge to recipes and how to modify those recipes to your personal preferences. Sure an LLM can generate novel recipes but without the premise of taste, texture and food pairing tuned in to personal preference, the novel recipe is a crap shoot in the dark. The reality is, the more you learn to cook, the less you need recipes and a good cookbokk will teach you alot more than a pile of recipes.
deleted 3y ago
[deleted]
burkaman 3y ago
I understand what you're saying, I just think we have a fundamental disagreement over "taste is implicit in the data" that I'm not sure we can resolve here. Intuitively it makes sense to me that if you know the entire sequence of a protein's fundamental building block then you can deduce its higher-level properties, but it does not make sense that if you know a lot of high-level descriptions of an ingredient and how it tastes then you can deduce behaviors like how it will react to a certain application of heat or another ingredient. I appreciate your links though, I think this is the first I've heard of an LLM doing something that a human can't do. I can't claim to understand a lot of the first paper and I can't access the second, but assuming your descriptions are right I agree that my rule of thumb was wrong.
- famouswaffles 3y ago
  I see. First let me explain what I mean by implicit incentive with our biology example. You have the protein sequence, G46AKT5778FAG4 The predictor is given "G46A____" The only way to predict this correctly consistently across multiple proteins is learn the underlying biological structure. Now, I pulled this snippet of a recipe randomly from the web. "Add the tomato purée and turn up the heat slightly, cooking until it has darkened, about 2-3 mins." Block out some information. "Add the tomato purée and turn up the heat slightly, cooking until it has ____" The only way a predictor is getting what kind of completion to make consistently correct across multiple unseen recipes and food items is if it has some model of the visible effects of heat on food in general. The predictor will be forced learn what kind of foods darken in heated water and group them together internally so it doesn't have to memorize every single instance (more effort than simply learning it) Now let's put back darkened and block out the time. "Add the tomato purée and turn up the heat slightly, cooking until it has darkened, about____." The only way a predictor is consistently correct on what time interval this should be across multiple recipes is if it has some model of the effects of heat and time on whatever it is you're cooking. Implicit incentives are everywhere.
deleted 3y ago
[deleted]

uoaei 3y ago

I don't think the commenter's understanding of LLMs is relevant here. It's what the authors of TFA think LLMs are for and, arguably more importantly, what kinds of problems (nails) they think about when presented with the idea of applying LLMs (hammer). It's a bit ridiculous that this is the best they can come up with, and I don't really think the idea is worth defending, especially by telling other people that they just don't get it.

famouswaffles 3y ago
His/Her primary assertion is that it is impossible for a LLM to generate novel recipes without physically tasting it first. His/her assertion is wrong and one easily corrected by understanding a LLM is and does. It's not a perfect predictor so it's not impeccable for all conceivable situations but GPT-4 can already generate novel recipes that taste nice. This is not some far flung science fiction ability.
- ska 3y ago
  This is true. On the other hand, the recipes aren't typically good, and generating novel ones isn't really what people mostly want to do... so the value seems limited.
  - famouswaffles 3y ago
    That's part of it yes but people tend to forget that the road to capable prediction is not instantaneous. It's possible the data is enough to get a good predictor but the one you tested simply wasn't good enough yet.
    - ska 3y ago
      Possibly, but the easily available data itself seems is pretty poor for systemic reasons.
- ForestCritter 3y ago
  Salt to taste, cook until brown, kneed until the dough is no longer sticky, everything about cooking is tasting and feeling. Moreover everybody has differences in their tastes and consistency preferences. Good recipe books teach you how to use ingredients and apply that knowledge to recipes and how to modify those recipes to your personal preferences. Sure an LLM can generate novel recipes but without the premise of taste, texture and food pairing tuned in to personal preference, the novel recipe is a crap shoot in the dark. The reality is, the more you learn to cook, the less you need recipes and a good cookbokk will teach you alot more than a pile of recipes.

deleted 3y ago

[deleted]

burkaman 3y ago

I understand what you're saying, I just think we have a fundamental disagreement over "taste is implicit in the data" that I'm not sure we can resolve here. Intuitively it makes sense to me that if you know the entire sequence of a protein's fundamental building block then you can deduce its higher-level properties, but it does not make sense that if you know a lot of high-level descriptions of an ingredient and how it tastes then you can deduce behaviors like how it will react to a certain application of heat or another ingredient. I appreciate your links though, I think this is the first I've heard of an LLM doing something that a human can't do. I can't claim to understand a lot of the first paper and I can't access the second, but assuming your descriptions are right I agree that my rule of thumb was wrong.

famouswaffles 3y ago
I see. First let me explain what I mean by implicit incentive with our biology example. You have the protein sequence, G46AKT5778FAG4 The predictor is given "G46A____" The only way to predict this correctly consistently across multiple proteins is learn the underlying biological structure. Now, I pulled this snippet of a recipe randomly from the web. "Add the tomato purée and turn up the heat slightly, cooking until it has darkened, about 2-3 mins." Block out some information. "Add the tomato purée and turn up the heat slightly, cooking until it has ____" The only way a predictor is getting what kind of completion to make consistently correct across multiple unseen recipes and food items is if it has some model of the visible effects of heat on food in general. The predictor will be forced learn what kind of foods darken in heated water and group them together internally so it doesn't have to memorize every single instance (more effort than simply learning it) Now let's put back darkened and block out the time. "Add the tomato purée and turn up the heat slightly, cooking until it has darkened, about____." The only way a predictor is consistently correct on what time interval this should be across multiple recipes is if it has some model of the effects of heat and time on whatever it is you're cooking. Implicit incentives are everywhere.

deleted 3y ago

[deleted]