3 ms·

Look at the trajectory, not where we are. S curve, yada, yada. Sure it may top out and we will be left with something that many will argue is not AI. But on the

Look at the trajectory, not where we are. S curve, yada, yada. Sure it may top out and we will be left with something that many will argue is not AI. But on the other hand the curve may continue and it will be GAI. I don't see any sign of it topping out yet.

ChatGTP 4y ago
Translation: “something might happen”.
jszymborski 4y ago
I don't blame people for being optimistic, LLMs represent a huge jump in what we can do with computers. But there are two huge problems here with this article that none of the progress we've made with LLMs are close to solving. 1. Most of the time, you don't actually want to talk to your computer. You have a huge database. You need to make a number of non-trivial queries with subtle but precise requirements against it. Your livelihood depends on these results being correct. You have limited time to work on it. Would you rather (A) write an SQL query or (B) text Fred to do it for you. Fred doesn't have any special knowledge of your task, but he's capable of writing SQL, and he's of an obsequious and loyal disposition; willing to do whatever you ask over SMS. Fred is a really confident guy, and his deliverable, while they are to the best of his ability, are delivered expediently regardless of how correct they are or how good Fred is at the task. Choosing B means that I need to brief Fred on the task. I need to layout explicitly what records I want, how optimized I need the query to be for my tight timeline, and I need to impress upon him how deeply important the results of the outcome are. It is of the utmost importance that I explain away every possible misconception because if Fred is unsure of something, he might just shrug it off and give me his best guess. Suddenly, the language I'm using to communicate to Fred becomes more... formal. I'm writing these verbose documents to Fred that are starting to look like ISO documents. I'm starting to see the appeal of constructed languages like Lojban which are formally verified and designed to avoid ambiguity. Wait... maybe what I actually want is a special language that would be both unambiguous and high-bandwidth. Ah yes, I think SQL would suit me just fine actually. 2. LLMs are as much building an AGI as building a keyboard is building a computer. I'll not get into it here, but there's a lot of burden of proof that LLMs are a microcosm of computers reasoning about things. LLMs are super impressive, and they are able to do some neat tricks like do math like a drunk 7 year old or generate plausible SQL injections. The rub here is that while that _feels_ like it's reasoning and has a mental model of arithmetic or cyber-security, it's just predicting what word is most likely to go next.
- plutonorm 4y ago
  Completely disagree with 2. Look up AIXI. It looks and feels to me like a completely plausible and mathematically grounded theory of intelligence. One principle aspect of the theory is that it allows you to see that intelligence is merely compression. This is the origin of the Hutter prize. Compression is almost the same thing as prediction, look into the math. Llms are prediction machines. Predicting the next word isn't some stupid goal it's exactly what you need to do to build an intelligent machine.
  - jszymborski 4y ago
    I'm failing to see what AIXI, a theoretical RL agent, has to do with LLMs. I never said that predicting the next word was a stupid goal, just that it isn't AGI.
  - atoav 4y ago
    Whether you are fine with 2 depends on how much you are willing to shrug any possible outcome off and say: "I guess it's fine". And, more essentially, how much the rest of the company just shrugs it off and says: "I guess it's fine", so when it blows up you can defend with the famous defence my Nazi grandfather also told me: "But everybody else did it too." Remember the Xerox printer that was swapping out (and therefore falsifying) numbers in their compression algorithm whenever you made a copy? Imagine we could bring this to all of our business processes, but in new and unexpected ways.
- d1sxeyes 4y ago
  > The rub here is that while that _feels_ like it's reasoning and has a mental model of arithmetic or cyber-security, it's just predicting what word is most likely to go next. I have yet to hear a convincing argument that 'reasoning' is any more than simply predicting what comes next. We intuitively think that human intelligence is special, and that somehow we are categorically different to some black box of ones and zeros in any form other than substrate.
  - jszymborski 4y ago

ChatGTP 4y ago

Translation: “something might happen”.

jszymborski 4y ago

I don't blame people for being optimistic, LLMs represent a huge jump in what we can do with computers. But there are two huge problems here with this article that none of the progress we've made with LLMs are close to solving. 1. Most of the time, you don't actually want to talk to your computer. You have a huge database. You need to make a number of non-trivial queries with subtle but precise requirements against it. Your livelihood depends on these results being correct. You have limited time to work on it. Would you rather (A) write an SQL query or (B) text Fred to do it for you. Fred doesn't have any special knowledge of your task, but he's capable of writing SQL, and he's of an obsequious and loyal disposition; willing to do whatever you ask over SMS. Fred is a really confident guy, and his deliverable, while they are to the best of his ability, are delivered expediently regardless of how correct they are or how good Fred is at the task. Choosing B means that I need to brief Fred on the task. I need to layout explicitly what records I want, how optimized I need the query to be for my tight timeline, and I need to impress upon him how deeply important the results of the outcome are. It is of the utmost importance that I explain away every possible misconception because if Fred is unsure of something, he might just shrug it off and give me his best guess. Suddenly, the language I'm using to communicate to Fred becomes more... formal. I'm writing these verbose documents to Fred that are starting to look like ISO documents. I'm starting to see the appeal of constructed languages like Lojban which are formally verified and designed to avoid ambiguity. Wait... maybe what I actually want is a special language that would be both unambiguous and high-bandwidth. Ah yes, I think SQL would suit me just fine actually. 2. LLMs are as much building an AGI as building a keyboard is building a computer. I'll not get into it here, but there's a lot of burden of proof that LLMs are a microcosm of computers reasoning about things. LLMs are super impressive, and they are able to do some neat tricks like do math like a drunk 7 year old or generate plausible SQL injections. The rub here is that while that _feels_ like it's reasoning and has a mental model of arithmetic or cyber-security, it's just predicting what word is most likely to go next.

plutonorm 4y ago
Completely disagree with 2. Look up AIXI. It looks and feels to me like a completely plausible and mathematically grounded theory of intelligence. One principle aspect of the theory is that it allows you to see that intelligence is merely compression. This is the origin of the Hutter prize. Compression is almost the same thing as prediction, look into the math. Llms are prediction machines. Predicting the next word isn't some stupid goal it's exactly what you need to do to build an intelligent machine.
- jszymborski 4y ago
  I'm failing to see what AIXI, a theoretical RL agent, has to do with LLMs. I never said that predicting the next word was a stupid goal, just that it isn't AGI.
- atoav 4y ago
  Whether you are fine with 2 depends on how much you are willing to shrug any possible outcome off and say: "I guess it's fine". And, more essentially, how much the rest of the company just shrugs it off and says: "I guess it's fine", so when it blows up you can defend with the famous defence my Nazi grandfather also told me: "But everybody else did it too." Remember the Xerox printer that was swapping out (and therefore falsifying) numbers in their compression algorithm whenever you made a copy? Imagine we could bring this to all of our business processes, but in new and unexpected ways.
d1sxeyes 4y ago
> The rub here is that while that _feels_ like it's reasoning and has a mental model of arithmetic or cyber-security, it's just predicting what word is most likely to go next. I have yet to hear a convincing argument that 'reasoning' is any more than simply predicting what comes next. We intuitively think that human intelligence is special, and that somehow we are categorically different to some black box of ones and zeros in any form other than substrate.
- jszymborski 4y ago