3 ms·

> inventorying my opinions but you assume that your opinions aren't formed from a preexisting body of knowledge from which your past self has learnt and intern

by chii 4y ago

> inventorying my opinions but you assume that your opinions aren't formed from a preexisting body of knowledge from which your past self has learnt and internalized. > If I were to start with ChatGPT I would, necessarily, only say what has been said. And i would also imagine that this isn't too different from someone with a vast experience behind them. And these ML models don't just repeat what has been said - i think there's enough complexity in the model where they could genuinely extract knowledge (or patterns) across different areas, and have it be applied to specific circumstances.

iudqnolq 4y ago
I'm less confident it innovates based on my practical experience with Copilot. It writes boring, verbose code. And it assumes APIs are written in that style as well. That effect is worse the less it knows about an API. So if you want it to be able to complete based on the rest of your code in your project you'd have to tailor your APIs to it, which I won't do. TabNine has a much more primitive model, but I like that they weigh the code on your hard drive much heavier.
- piva00 4y ago
  Boring code is not bad code, overly verbose might be but descriptive verbose code is pretty damn good when you are maintaining something for 5-10 years.
  - saurik 4y ago
    In my experience maintaining things for even longer than that, if your code is verbose you are way more likely to make a mistake many years later because you don't remember all of the rules required to correctly edit or generate new code in the same form. You are much better off compressing that code using domain-specific abstractions that force all development to fully follow every mental model you ascribed to the model, even if it means you have to load more mental state and it takes you a bit longer to "get back up to speed" in the project you haven't edited in a while, as it helps guarantee you actually load the correct state.
  - iudqnolq 4y ago
    It depends on the specifics. I think a correct AI should look at the relevant context and give code with matching verbosity.
Etherlord87 4y ago
> but you assume that your opinions aren't formed from a preexisting body of knowledge from which your past self has learnt and internalized. I think the OP's argument is, the ChatGPT can only average inputs into an output, as opposed to a human mind that can extrapolate. Regardless, the obvious difference is we humans build bigger understanding, a complex structure of ideas based on our inputs, and so not only we have a lot more ideas to combine, but also we use acquired ideas to rule which ideas we preliminarily choose to combine and how we combine them. So as to quote the meme > We are not the same.
- lordnacho 4y ago
  But this is only an effect of reward maximization. If I write a poem the way everyone writes it, I won't be found out as a fraud. Which is what we've told AI to do, just do whatever you need to get a passing grade like all the human kids in the class. Why couldn't we create truly novel things by allowing for some failures? It's a bit like how there was a bunch of new musical styles after the war. Terrible flops mixed in there, but we got some interesting things as well.
  - throwaway4aday 4y ago
    I think this is something OpenAI is working on, updating their RLHF practices so that there is a wider variety of input shaping the selected responses. There's a huge difference between 3.5 and 4 due to this. Sam Altman has mentioned it a few times in podcasts he's appeared on so they are aware that it is a problem to have too narrow of a selection of reviewers. Ideally, you'd have a really broad selection of people from many different areas of expertise and tacit knowledge train a generalist model. You could then have more specialist models as well that were trained by different professions although you'd probably still want some amount of training from other areas for cross-pollination. It's kind of crazy how the model learns to act this way, we funnel in more knowledge than any one person would ever be exposed to in their lifetime and it learns some latent structures in language and knowledge from this, then we teach it to interact with humans in a more natural way. It's both different and similar to how humans learn language and then social skills. Makes me wonder what else we could teach it to do through interacting with it.
mjburgess 4y ago
Modern AI is just a variation on `mean()` -- the apparent intelligence comes from what it's averaging. (In particular, it's just an `Expectation_weightDistribution[TrainingData|Query]` ) If you wanted to be, I think pathologically cynical about animal intelligence, and reduce it down to the same operation, then at the very least the "TrainingData" and the "weightDistribution" is specialised to the individual -- and so different from the average. However, we cannot really be this extremely cynical (I find this pseudoscience) -- animals acquire novel training data by changing their environments. Animals are a source of data. So even on this cynical reading we have to at least go towards, while(Alive && ...) MyExperiences = ChangeTheWorld(MyExperiences|Goal) MyIdeas = Expectation_mySenseOfRelevance[MyExperiences|Goal] ... Which makes `MyIdeas` quite radically different from the `mean()`ing of the Ai.
- mitjam 4y ago
  Interesting viewpoint, following this, it would probably be beneficial to have many different personal AIs as compared to a few AIs that serve many.
  - caddemon 4y ago
    Yeah this is a good point, independent of AI (but probably not the internet) things are already becoming more homogeneous IMO. If that's expedited by everybody using e.g. ChatGPT I don't think that would be good.
    - pas 4y ago
      How many degrees of freedom these networks have? There's a lot of room in high dimensional space :) Sure, same prompts same answers, but people are pretty creative already in their questions.
  - CuriouslyC 4y ago
    It makes more sense to have a really strong base model that is "general" and then apply LoRAs to it to "specialize" it.
koonsolo 4y ago
ChatGPT for sure can be creative, but it lacks 1 thing: real life world experience. For example, as a senior software developer, I'm not convinced pure unit testing is always the way to go. I was thinking about writing my own alternative and how to weigh the benefits against the drawbacks. I was querying ChatGPT about it, but it's just able to restate the current "common knowledge". So yes, ChatGPT can come up with new concepts. It however, cannot find out the problems, issues and unexpected consequences of applying it in the real world. Somehow, these things will have to be fed into it again, in either the form of text, images or video. 99% of current writing can be replaced by ChatGPT. But this 1% of things only people with real world experience can come up with, it cannot do.
- edwardso 4y ago
  Yeah, with my interaction with ChatGPT it just spits out general concepts that almost make it seem like it exhaustively covers the prompt, but from those concepts sometimes there'll be 1 or 2 that just won't be applicable to the reality of the real world. Large language model lacks the acuteness to anticipate any unintended consequences and that 1% of edge case is where things might truly matter.
- pillefitz 4y ago
  ChatGPT lists (common) downsides, remedies and situations when to avoid unit testing. Do you have the expected answer in mind, and could chatGPT have possibly deduced this from your prompt?
  - koonsolo 4y ago
    My problem and solution is probably too wide to explain here, so let's take an easier example. TDD where you write the test before the code, is a widespread concept now. Imagine a time before this was the case. Would ChatGPT be able to come up with such a concept? Probably. Would it be able to really understand all the consequences of such a setup? I highly doubt it. Even if it did, would we trust it? In my opinion, such things can only be tested in the real world, where you work with real projects, developers and testers, and gather all the impacts. I don't think a LLM could derive such things.
rcme 4y ago
> but you assume that your opinions aren't formed from a preexisting body of knowledge from which your past self has learnt and internalized. Of course opinions are partially formed by preexisting knowledge learnt by your past self. But the fundamental difference between yourself and an ML model, which TFA touches on, is that an ML model stops learning after training whereas your past self did learn and your current self continues to learn. “Learn” here means something akin to updating the model weights of your brain. And you see this in practice. Ask ChatGPT about something not in the training set and it doesn’t answer well whereas a human is capable of learning about a topic and incorporating that knowledge in the future.
tomxor 4y ago
> but you assume that your opinions aren't formed from a preexisting body of knowledge from which your past self has learnt and internalized. Knowledge != experience, there is overlap for sure, but they are qualitatively different things. And that's before we even get into the underlying differences in the process: when GPT writes a book it's predicting desired output based on what it's learned from it's training data, but it's only as deep as language, coherant logic that appears deeper is incidental and only as common as it is due to the sheer amount of data it's consumed and statistical correlation - all of the amusement we get from ChatGPT is when this lack of depth is revealed. When a human writes a book they are using the same mechanisms for sure, predicting desired output, but that's only the language level, bellow that is a more abstract process driving the direction and ideation, introspection and interrogating experiences, exploring and validating them before attempting to express them in language. I think part of what makes these LLMs appear to be smarter and more creative than they have any right to be, is due to the inhuman level of input... they are extremely shallow models, but have massive bandwidth, way more than a human could consume - as a result clever stuff emerges more though the sheer volume than something intrinsic to the model itself.

by chii 4y ago

iudqnolq 4y ago

I'm less confident it innovates based on my practical experience with Copilot. It writes boring, verbose code. And it assumes APIs are written in that style as well. That effect is worse the less it knows about an API. So if you want it to be able to complete based on the rest of your code in your project you'd have to tailor your APIs to it, which I won't do. TabNine has a much more primitive model, but I like that they weigh the code on your hard drive much heavier.

piva00 4y ago
Boring code is not bad code, overly verbose might be but descriptive verbose code is pretty damn good when you are maintaining something for 5-10 years.
- saurik 4y ago
  In my experience maintaining things for even longer than that, if your code is verbose you are way more likely to make a mistake many years later because you don't remember all of the rules required to correctly edit or generate new code in the same form. You are much better off compressing that code using domain-specific abstractions that force all development to fully follow every mental model you ascribed to the model, even if it means you have to load more mental state and it takes you a bit longer to "get back up to speed" in the project you haven't edited in a while, as it helps guarantee you actually load the correct state.
- iudqnolq 4y ago
  It depends on the specifics. I think a correct AI should look at the relevant context and give code with matching verbosity.

Etherlord87 4y ago

> but you assume that your opinions aren't formed from a preexisting body of knowledge from which your past self has learnt and internalized. I think the OP's argument is, the ChatGPT can only average inputs into an output, as opposed to a human mind that can extrapolate. Regardless, the obvious difference is we humans build bigger understanding, a complex structure of ideas based on our inputs, and so not only we have a lot more ideas to combine, but also we use acquired ideas to rule which ideas we preliminarily choose to combine and how we combine them. So as to quote the meme > We are not the same.

lordnacho 4y ago
But this is only an effect of reward maximization. If I write a poem the way everyone writes it, I won't be found out as a fraud. Which is what we've told AI to do, just do whatever you need to get a passing grade like all the human kids in the class. Why couldn't we create truly novel things by allowing for some failures? It's a bit like how there was a bunch of new musical styles after the war. Terrible flops mixed in there, but we got some interesting things as well.
- throwaway4aday 4y ago
  I think this is something OpenAI is working on, updating their RLHF practices so that there is a wider variety of input shaping the selected responses. There's a huge difference between 3.5 and 4 due to this. Sam Altman has mentioned it a few times in podcasts he's appeared on so they are aware that it is a problem to have too narrow of a selection of reviewers. Ideally, you'd have a really broad selection of people from many different areas of expertise and tacit knowledge train a generalist model. You could then have more specialist models as well that were trained by different professions although you'd probably still want some amount of training from other areas for cross-pollination. It's kind of crazy how the model learns to act this way, we funnel in more knowledge than any one person would ever be exposed to in their lifetime and it learns some latent structures in language and knowledge from this, then we teach it to interact with humans in a more natural way. It's both different and similar to how humans learn language and then social skills. Makes me wonder what else we could teach it to do through interacting with it.

mjburgess 4y ago

Modern AI is just a variation on `mean()` -- the apparent intelligence comes from what it's averaging. (In particular, it's just an `Expectation_weightDistribution[TrainingData|Query]` ) If you wanted to be, I think pathologically cynical about animal intelligence, and reduce it down to the same operation, then at the very least the "TrainingData" and the "weightDistribution" is specialised to the individual -- and so different from the average. However, we cannot really be this extremely cynical (I find this pseudoscience) -- animals acquire novel training data by changing their environments. Animals are a source of data. So even on this cynical reading we have to at least go towards, while(Alive && ...) MyExperiences = ChangeTheWorld(MyExperiences|Goal) MyIdeas = Expectation_mySenseOfRelevance[MyExperiences|Goal] ... Which makes `MyIdeas` quite radically different from the `mean()`ing of the Ai.

mitjam 4y ago
Interesting viewpoint, following this, it would probably be beneficial to have many different personal AIs as compared to a few AIs that serve many.
- caddemon 4y ago
  Yeah this is a good point, independent of AI (but probably not the internet) things are already becoming more homogeneous IMO. If that's expedited by everybody using e.g. ChatGPT I don't think that would be good.
  - pas 4y ago
    How many degrees of freedom these networks have? There's a lot of room in high dimensional space :) Sure, same prompts same answers, but people are pretty creative already in their questions.
- CuriouslyC 4y ago
  It makes more sense to have a really strong base model that is "general" and then apply LoRAs to it to "specialize" it.

koonsolo 4y ago

ChatGPT for sure can be creative, but it lacks 1 thing: real life world experience. For example, as a senior software developer, I'm not convinced pure unit testing is always the way to go. I was thinking about writing my own alternative and how to weigh the benefits against the drawbacks. I was querying ChatGPT about it, but it's just able to restate the current "common knowledge". So yes, ChatGPT can come up with new concepts. It however, cannot find out the problems, issues and unexpected consequences of applying it in the real world. Somehow, these things will have to be fed into it again, in either the form of text, images or video. 99% of current writing can be replaced by ChatGPT. But this 1% of things only people with real world experience can come up with, it cannot do.

edwardso 4y ago
Yeah, with my interaction with ChatGPT it just spits out general concepts that almost make it seem like it exhaustively covers the prompt, but from those concepts sometimes there'll be 1 or 2 that just won't be applicable to the reality of the real world. Large language model lacks the acuteness to anticipate any unintended consequences and that 1% of edge case is where things might truly matter.
pillefitz 4y ago
ChatGPT lists (common) downsides, remedies and situations when to avoid unit testing. Do you have the expected answer in mind, and could chatGPT have possibly deduced this from your prompt?
- koonsolo 4y ago
  My problem and solution is probably too wide to explain here, so let's take an easier example. TDD where you write the test before the code, is a widespread concept now. Imagine a time before this was the case. Would ChatGPT be able to come up with such a concept? Probably. Would it be able to really understand all the consequences of such a setup? I highly doubt it. Even if it did, would we trust it? In my opinion, such things can only be tested in the real world, where you work with real projects, developers and testers, and gather all the impacts. I don't think a LLM could derive such things.

rcme 4y ago

> but you assume that your opinions aren't formed from a preexisting body of knowledge from which your past self has learnt and internalized. Of course opinions are partially formed by preexisting knowledge learnt by your past self. But the fundamental difference between yourself and an ML model, which TFA touches on, is that an ML model stops learning after training whereas your past self did learn and your current self continues to learn. “Learn” here means something akin to updating the model weights of your brain. And you see this in practice. Ask ChatGPT about something not in the training set and it doesn’t answer well whereas a human is capable of learning about a topic and incorporating that knowledge in the future.

tomxor 4y ago

> but you assume that your opinions aren't formed from a preexisting body of knowledge from which your past self has learnt and internalized. Knowledge != experience, there is overlap for sure, but they are qualitatively different things. And that's before we even get into the underlying differences in the process: when GPT writes a book it's predicting desired output based on what it's learned from it's training data, but it's only as deep as language, coherant logic that appears deeper is incidental and only as common as it is due to the sheer amount of data it's consumed and statistical correlation - all of the amusement we get from ChatGPT is when this lack of depth is revealed. When a human writes a book they are using the same mechanisms for sure, predicting desired output, but that's only the language level, bellow that is a more abstract process driving the direction and ideation, introspection and interrogating experiences, exploring and validating them before attempting to express them in language. I think part of what makes these LLMs appear to be smarter and more creative than they have any right to be, is due to the inhuman level of input... they are extremely shallow models, but have massive bandwidth, way more than a human could consume - as a result clever stuff emerges more though the sheer volume than something intrinsic to the model itself.