3 ms·

I'm glad Lex chose to spend half of it discussing Jordan Peterson and Elon Musk. Also as great and useful as the LLMs are, it's still a bit of a leap to claim

by m_ke 4y ago

I'm glad Lex chose to spend half of it discussing Jordan Peterson and Elon Musk. Also as great and useful as the LLMs are, it's still a bit of a leap to claim it's getting us that much closer to AGI. GPT does a great job compressing and distilling the knowledge found in text on the internet but it still doesn't have the ability to do real reasoning. As a simple example I recommend trying to have it multiply two large numbers, it does a decent job guestimating but obviously can't do the recursive steps required to generate a correct answer. EDIT: Also people worried about AGI implications should slow down and worry what humans and society will do with tools that can automate a large portion of our jobs and accrue all of the gains of that to a few well capitalized players.

uxcolumbo 4y ago
Has Lex started to be more critical of Musk or is he still behaving quasi sycophantic?
sebzim4500 4y ago
>As a simple example I recommend trying to have it multiply two large numbers, it does a decent job guestimating but obviously can't do the recursive steps required to generate a correct answer. Is it worse than the average human at mental arithmetic? I haven't done detailed benchmarks, but for my usecase it only gets maths wrong once it gets way past the range that I could do myself. Personally, I find the fact that it can't say "I don't know" to be much more offputting than the dodgy maths.
- ftxbro 4y ago
  > Personally, I find the fact that it can't say "I don't know" to be much more offputting This is a problem with their PPO "finishing school" not with their base model. Check Figure 8 in https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf its base model has an uncannily well calibrated confidence in the correctness of its decisions. Basically, their PPO is for training it to talk to normies, and normies don't like uncertainty. Reinforcement learning using Proximal Policy Optimization (PPO): They optimize a policy against the reward model using reinforcement learning. For each new prompt sampled from the dataset, the policy generates an output. The reward model calculates a reward for the output, and the reward is used to update the policy using the PPO algorithm.
  - sebzim4500 4y ago
    True, but even with the base model it is hard to go from logits to a confidence level for freeform answers. For multiple choice questions it is easy, since you just look at the probabilities for each of the allowable tokens in the answer (which is presumably what they did in that benchmark) but I have no idea how to do it more generally.
r0uv3n 4y ago
It is absolutely not obvious it can't do the recursive steps to multiply two numbers. In fact, you can just teach it the proper algorithm and ask it to apply it: https://twitter.com/random_walker/status/1600336556425826304?s=20 https://twitter.com/random_walker/status/1600336556425826304...
- m_ke 4y ago
  The feedforward nature of transformers means that it can only do a limited number of computational steps. Having it explain an algorithm is a good way to have it use its output as a tape during inference/decoding, but even then it just fudges things and makes mistakes for largish numbers.
ActorNightly 4y ago
>it still doesn't have the ability to do real reasoning. It technically does. Humans solve problems in a recursive way, much like the generative transformer networks generate the next piece of text. The big difference of course is the depth of reasoning. My bet is that the next big effort is going to be reduce the size of the models while retaining performance (perhaps using ML tasked to do so). Then once you have that, from the other side there will be more and more specialized hardware with ASIC to run the transformers. Eventually we will be able to run these things at scale and do on the fly transfer learning, which will be about the same as a human growing reasoning skills as they mature.
ftxbro 4y ago
This parent post with its EDIT is a perfect example of what I wrote in another comment: Beliefs are changing so fast right now. The term "AGI skeptic" will soon (if not already) mean "I don't trust AGIs in positions of authority or power" rather than "I don't think the technology is capable of matching our level of cognition."
- ChatGTP 4y ago
  Hang on, we’ve also changed the definition of AGI, now this thing is increasingly “marketed” as an AGI, and as someone has used it, I don’t want it anywhere near critical decision making. There is marketing and there’s reality. Don’t conflate the two. There is untold money which has been spent on these systems. They’re going to be pushed into our lives whether we like it or not. That’s not to say there is no value in machine learning, but there also a lot of money to be made from it. Think opioid crisis, you had it prescribed even if it wasn’t in your interest. This is America…
djaychela 4y ago
> it still doesn't have the ability to do real reasoning I don't think that matters. It does significantly more than algorithms which are designed to simply maximise engagement, and they have revolutionised the online world in the last decade. This will be orders of magnitude more powerful. And given what the simple 'maximise engagement' algorithms have done to various aspects of society and the people who live in it, I don't think this will be good given how little attention seems to be paid to these aspects by those who stand to profit from their use.
- madeofpalk 4y ago
  It might not matter to whether it's a useful product, but it matters as to whether it is artificial general inteligence or not.
  - m_ke 4y ago
    Yes, my post was mostly a response to the first few segments of the interview where Sam talks about how experts in the field didn’t believe in their (openai) focus on AGI. I’ve been working in deep learning for over 10 years now and was always bullish on the technology but as great of a tool as these LLMs might be, I wouldn’t bet on them getting us to true AGI anytime soon.
gautamdivgi 4y ago
I find Peterson to be an insufferable coot. Musk is obviously more dynamic.

by m_ke 4y ago

uxcolumbo 4y ago

Has Lex started to be more critical of Musk or is he still behaving quasi sycophantic?

sebzim4500 4y ago

>As a simple example I recommend trying to have it multiply two large numbers, it does a decent job guestimating but obviously can't do the recursive steps required to generate a correct answer. Is it worse than the average human at mental arithmetic? I haven't done detailed benchmarks, but for my usecase it only gets maths wrong once it gets way past the range that I could do myself. Personally, I find the fact that it can't say "I don't know" to be much more offputting than the dodgy maths.

ftxbro 4y ago
> Personally, I find the fact that it can't say "I don't know" to be much more offputting This is a problem with their PPO "finishing school" not with their base model. Check Figure 8 in https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf its base model has an uncannily well calibrated confidence in the correctness of its decisions. Basically, their PPO is for training it to talk to normies, and normies don't like uncertainty. Reinforcement learning using Proximal Policy Optimization (PPO): They optimize a policy against the reward model using reinforcement learning. For each new prompt sampled from the dataset, the policy generates an output. The reward model calculates a reward for the output, and the reward is used to update the policy using the PPO algorithm.
- sebzim4500 4y ago
  True, but even with the base model it is hard to go from logits to a confidence level for freeform answers. For multiple choice questions it is easy, since you just look at the probabilities for each of the allowable tokens in the answer (which is presumably what they did in that benchmark) but I have no idea how to do it more generally.

r0uv3n 4y ago

It is absolutely not obvious it can't do the recursive steps to multiply two numbers. In fact, you can just teach it the proper algorithm and ask it to apply it: https://twitter.com/random_walker/status/1600336556425826304?s=20 https://twitter.com/random_walker/status/1600336556425826304...

m_ke 4y ago
The feedforward nature of transformers means that it can only do a limited number of computational steps. Having it explain an algorithm is a good way to have it use its output as a tape during inference/decoding, but even then it just fudges things and makes mistakes for largish numbers.

ActorNightly 4y ago

>it still doesn't have the ability to do real reasoning. It technically does. Humans solve problems in a recursive way, much like the generative transformer networks generate the next piece of text. The big difference of course is the depth of reasoning. My bet is that the next big effort is going to be reduce the size of the models while retaining performance (perhaps using ML tasked to do so). Then once you have that, from the other side there will be more and more specialized hardware with ASIC to run the transformers. Eventually we will be able to run these things at scale and do on the fly transfer learning, which will be about the same as a human growing reasoning skills as they mature.

ftxbro 4y ago

This parent post with its EDIT is a perfect example of what I wrote in another comment: Beliefs are changing so fast right now. The term "AGI skeptic" will soon (if not already) mean "I don't trust AGIs in positions of authority or power" rather than "I don't think the technology is capable of matching our level of cognition."

ChatGTP 4y ago
Hang on, we’ve also changed the definition of AGI, now this thing is increasingly “marketed” as an AGI, and as someone has used it, I don’t want it anywhere near critical decision making. There is marketing and there’s reality. Don’t conflate the two. There is untold money which has been spent on these systems. They’re going to be pushed into our lives whether we like it or not. That’s not to say there is no value in machine learning, but there also a lot of money to be made from it. Think opioid crisis, you had it prescribed even if it wasn’t in your interest. This is America…

djaychela 4y ago

> it still doesn't have the ability to do real reasoning I don't think that matters. It does significantly more than algorithms which are designed to simply maximise engagement, and they have revolutionised the online world in the last decade. This will be orders of magnitude more powerful. And given what the simple 'maximise engagement' algorithms have done to various aspects of society and the people who live in it, I don't think this will be good given how little attention seems to be paid to these aspects by those who stand to profit from their use.

madeofpalk 4y ago
It might not matter to whether it's a useful product, but it matters as to whether it is artificial general inteligence or not.
- m_ke 4y ago
  Yes, my post was mostly a response to the first few segments of the interview where Sam talks about how experts in the field didn’t believe in their (openai) focus on AGI. I’ve been working in deep learning for over 10 years now and was always bullish on the technology but as great of a tool as these LLMs might be, I wouldn’t bet on them getting us to true AGI anytime soon.

gautamdivgi 4y ago

I find Peterson to be an insufferable coot. Musk is obviously more dynamic.