4 ms·

This (popular) opinion makes no sense. How are you going to improve something you dont understand? Throwing ramdom software pieces to see what sticks? The recen

This (popular) opinion makes no sense. How are you going to improve something you dont understand? Throwing ramdom software pieces to see what sticks? The recent progress was possible because people understood intuitively enough of the limitations of earlier models to think and invent a remedy.

randomlogic 3y ago
(Reads:) "Did you know that time-traveling was invented, cos time was seen as a threat to life ?" P-:
Tuna-Fish 3y ago
Not that I necessarily agree with the above poster, but this: > How are you going to improve something you dont understand? Is just nonsense. Evolution understands nothing, yet produced a mind. Closer to us, the early people who produced all the crops that led to the shift to agriculture, and it's later improvements, absolutely did not understand how any of it worked. Evaluation and selection are sufficient to improve things. Understanding is useful, but optional.
- nologic01 3y ago
  These are just absurdly remote analogies that have nothing to with how ML algorithms have developed thus far, nor how they will develop in the the immediate future. So how would you "improve" here an now any algorithm. Create and insert random code and evaluate? Jeepers people are losing touch with reality.
Xcelerate 3y ago
> This (popular) opinion makes no sense. How are you going to improve something you dont understand? How does that opinion not make sense? There are numerous things humans have invented for which we have little understanding of how they work: medical drugs, anesthesia, certain quantum phenomena utilized in semiconductors, etc. I would argue that for current state-of-the-art LLMs, the implementation is likewise ahead of the theory at the moment.
- nologic01 3y ago
  "Understanding" does not need internalizing a process or algorithm at low level. We are not stochastic parrots. It is simply possesing sufficient insight so as to be able to explore nearby designs, formulate and test hypotheses, narrow the search space etc. There is a strange emerging AI cult that is also in force here in HN that seems to believe these algorithms have evolved themselves or were some random trial and error. Ergo, they can keep evolving and the researchers dont need to understand a thing about how they work. Serendipity in combining ideas that prove effective plays a role but within a fairly well defined conceptual sandbox. But progress with AI is more or less conditional on people having sufficient understanding to coax algorithmic structures in the desired direction.
  - imtringued 3y ago
    For decades ML researchers were walking on eggshells, thinking that they are going to run into overfitting and bias/variance tradeoffs, adding regularisation, worrying about local maxima, etc. Then one day a certain company kept making their models bigger without any concern and suddenly everyone thinks that there are unlimited returns to scale, when in reality you merely bought yourself a one time victory.

randomlogic 3y ago

(Reads:) "Did you know that time-traveling was invented, cos time was seen as a threat to life ?" P-:

Tuna-Fish 3y ago

Not that I necessarily agree with the above poster, but this: > How are you going to improve something you dont understand? Is just nonsense. Evolution understands nothing, yet produced a mind. Closer to us, the early people who produced all the crops that led to the shift to agriculture, and it's later improvements, absolutely did not understand how any of it worked. Evaluation and selection are sufficient to improve things. Understanding is useful, but optional.

nologic01 3y ago
These are just absurdly remote analogies that have nothing to with how ML algorithms have developed thus far, nor how they will develop in the the immediate future. So how would you "improve" here an now any algorithm. Create and insert random code and evaluate? Jeepers people are losing touch with reality.

Xcelerate 3y ago

> This (popular) opinion makes no sense. How are you going to improve something you dont understand? How does that opinion not make sense? There are numerous things humans have invented for which we have little understanding of how they work: medical drugs, anesthesia, certain quantum phenomena utilized in semiconductors, etc. I would argue that for current state-of-the-art LLMs, the implementation is likewise ahead of the theory at the moment.

nologic01 3y ago
"Understanding" does not need internalizing a process or algorithm at low level. We are not stochastic parrots. It is simply possesing sufficient insight so as to be able to explore nearby designs, formulate and test hypotheses, narrow the search space etc. There is a strange emerging AI cult that is also in force here in HN that seems to believe these algorithms have evolved themselves or were some random trial and error. Ergo, they can keep evolving and the researchers dont need to understand a thing about how they work. Serendipity in combining ideas that prove effective plays a role but within a fairly well defined conceptual sandbox. But progress with AI is more or less conditional on people having sufficient understanding to coax algorithmic structures in the desired direction.
- imtringued 3y ago
  For decades ML researchers were walking on eggshells, thinking that they are going to run into overfitting and bias/variance tradeoffs, adding regularisation, worrying about local maxima, etc. Then one day a certain company kept making their models bigger without any concern and suddenly everyone thinks that there are unlimited returns to scale, when in reality you merely bought yourself a one time victory.