4 ms·

Do you think researchers that really have 0 conflicts or interest with other researchers working in those companies really are up-to-date as to what the real ca

by bsaul 3y ago

Do you think researchers that really have 0 conflicts or interest with other researchers working in those companies really are up-to-date as to what the real capabilities of NN will be in the coming years ? We all know there's a huge gap between what researchers think is possible in theory and what's actually released as a working product in the industry. My intuition is nobody saw chatGPT coming, and only if you were inside openAI a few months before the release could you have anticipated the impact.

GTP 3y ago
I think it's exactly the opposite: researchers are currently studying models that are better than ChatGPT. The reason why when we use one of the open source models those tend to feel worse than ChatGPT is twofold: 1. Those models are usually optimized to run on an average PC, at the cost of accuracy 2. The researchers behind it don't have the kind of money that OpenAI has for training, but this doesn't mean that the model itself is less advanced than ChatGPT. I also suggest to read a somewhat famous Google memo that leaked a few months ago [1]. [1] https://www.semianalysis.com/p/google-we-have-no-moat-and-neither https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...
- bsaul 3y ago
  that's why i was mentioning the gap between better in theory and in practice. My feeling is that the transition between something that looks fantastic in a lab, and something that can actually be used as a product, like chatGPT is pretty hard.
  - GTP 3y ago
    But this has nothing to do with the quality of a consultancy from university researchers. They know very well how the models work and the direction the research is taking, they not having the same amount of money available for training is irrelevant w.r.t. their ability to provide advice and guidance in policy making. And even if I was wrong about this, the link I provided in the previous comment explains how the money required to effectively train those models is rapidly dropping, making the advantage of companies like OpenAI smaller and smaller.

by bsaul 3y ago

GTP 3y ago

I think it's exactly the opposite: researchers are currently studying models that are better than ChatGPT. The reason why when we use one of the open source models those tend to feel worse than ChatGPT is twofold: 1. Those models are usually optimized to run on an average PC, at the cost of accuracy 2. The researchers behind it don't have the kind of money that OpenAI has for training, but this doesn't mean that the model itself is less advanced than ChatGPT. I also suggest to read a somewhat famous Google memo that leaked a few months ago [1]. [1] https://www.semianalysis.com/p/google-we-have-no-moat-and-neither https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

bsaul 3y ago
that's why i was mentioning the gap between better in theory and in practice. My feeling is that the transition between something that looks fantastic in a lab, and something that can actually be used as a product, like chatGPT is pretty hard.
- GTP 3y ago
  But this has nothing to do with the quality of a consultancy from university researchers. They know very well how the models work and the direction the research is taking, they not having the same amount of money available for training is irrelevant w.r.t. their ability to provide advice and guidance in policy making. And even if I was wrong about this, the link I provided in the previous comment explains how the money required to effectively train those models is rapidly dropping, making the advantage of companies like OpenAI smaller and smaller.