4 ms·
humans have learnt by looking at a few examples, and from there can code almost anything. Humans can also learn without examples, although it's harder. But I ex
by bombolo 3y ago
humans have learnt by looking at a few examples, and from there can code almost anything. Humans can also learn without examples, although it's harder. But I expect the 1st C programmers didn't have a K&R book to learn from.
AI instead has looked at all the existing code and it has learnt to do no more than what the existing code can do.
Do you see a difference?
- krisoft 3y ago> AI instead has looked at all the existing code and it has learnt to do no more than what the existing code can do. Are you sure about that?
- bombolo 3y ago> Are you sure about that? Yes I'm quite sure. Unless you're claiming that source code existed before humans?
- krisoft 3y agoI’m sorry but I don’t understand you. It sounds like what you are saying is that the AI cannot write code which would do something it hasn’t seen in the training set. That is how I interpreted the “it has learnt to do no more than what the existing code can do”. Do i understand you right? If so, how do you know that? Are you talking about the limitations of a particular implementation or a limitation of all AIs as a concept?
- stavros 3y agoNo, because I can ask it for a piece of code that isn't in the training set and it will write it for me.
- mikojan 3y agoYou are missing the point. Humans make sense of the world with an extremely limited amount of information. I have not read all of GitHub. Comparatively, I have not read a fraction of it. And I could not read all of GitHub even if I wanted to. However, I do not need to read all of GitHub anyway because, as a human, I am capable of understanding. Current generation "AI" is not capable of understanding. Current generation "AI" merely computes the probability of any given word appearing next in this sentence based on an utterly absurd amount of raw data. That is not what humans do at all. We do not need this information. We cannot even process it.
- stavros 3y agoYou make a logical leap. "Machines learn differently from humans, therefore they cannot understand". My definition for "understanding" isn't just "whatever humans (and nobody else) do".
- mikojan 3y agoWhatever your definition of "understanding" may be, we appear to agree that these are different things which is all parent asked about: > Do you see a difference?
- vidarh 3y agoWe do not know whether or not that is what humans do or how qualitatively different it is to what humans do, because we don't know all that much about human reasoning works. Put another way: Some Markov models are Turing complete, so "merely" computing probabilities can be Turing complete with only minor steps, so trying to downplay the potential capabilities of models like this by handwaving about things we don't know is foolish. We don't need that scale of input, but we also don't know if LLMs need that much input to do well, or if our current training protocols are simply poor. With ongoing work on reducing the training cost, it is at a minimum clear that current training methods are far from optimal.
- bombolo 3y agoyou can ask for things that are combinations of stuff that is in the training set.