4 ms·

You presuppose that intelligence is like flight in the ways you've outlined (so solutions are going to converge). Frankly I don't know whether that's true or n

by flir 3y ago

You presuppose that intelligence is like flight in the ways you've outlined (so solutions are going to converge). Frankly I don't know whether that's true or not, but I want to suggest that it's a bad bet: I would have sworn blind that consciousness is an essential component of intelligence, but the chatbots are starting to make that look like a poor assumption on my part. When we know so little about intelligence, can we really assume there's only one way to be intelligent? To extend your analogy, I think that the intelligence equivalents of helicopters and rockets are out there somewhere, waiting to be found. I think I'm with Dijkstra on this one: "The question of whether machines can think is about as relevant as the question of whether submarines can swim" I think we're going to end up with submarines (or helicopters), not dolphins (or birds). No animal has evolved wheels, but wheels are a pretty good solution to the problem of movement. Maybe it's truer to say there's only one way to evolve an intelligent mammal, because you have to work with what already exists in the mammalian body. But AI research isn't constrained in that way. (Not saying you're wrong, just arguing we don't know enough to know if you're right).

mitthrowaway2 3y ago
I agree we don't know enough to know if I'm right! I tried to use a lot of hedgy-words. But it's not a presupposition, merely a line of argument why it's not a complete absurdity to think LLMs might be a step towards AGI. I do think consciousness is beside the point, as we have no way to test whether LLMs are conscious, just like we can't test anything else. We don't know what consciousness is, nor what it isn't. I don't think Dijkstra's argument applies here. Whether submarines "swim" is a good point about our vague mental boundaries of the word "swim". But submarine propellers are absolutely a convergent structure for underwater propulsion: it's the same hydrodynamic-lift-generating motion of a fin, just continuous instead of reciprocating. That's very much more structurally similar than I expect LLMs are to any hardware we have in our heads. It's true that the solution space for AI is in some ways less constrained than for biological intelligence, but just like submarines and whales operate under the same Navier-Stokes equations, humans and AI must learn and reason under the same equations of probability. Working solutions will probably have some mathematical structure in common. I think more relevant is Von Neumann: "If you will tell me precisely what it is that a machine cannot do, then I can always make a machine which will do just that!" Whether a submarine swims is a matter of semantics, but if there's a manuever that a whale can execute that a submarine cannot, then at least we can all agree about the non-generality of its swimming. For AGI, I can't say whether it's conscious or really thinks, but for the sake of concrete argument, it's dangerous enough to be concerned if: - it can form and maintain an objective; it can identify plausible steps to achieve that objective; it can accurately predict human responses to its actions; it can decently model the environment, as we can; it can hide its objectives from interrogators, and convince them that its actions are in their interests; it can deliver enough value to be capable of earning money through its actions; it can propose ideas that can convince investors to part with $100 billion; it can design a chemical plant that appears at a cursory inspection to manufacture high-profit fluorochemicals, but which also actually manufactures and stores CFCs in sufficient quantity to threaten the viability of terrestrial agriculture.
- kubanczyk 3y ago
  > it can form and maintain an objective; Yes and yes. > it can identify plausible steps to achieve that objective; It can predict. > it can accurately predict human responses to its actions; It can predict. > it can decently model the environment, as we can; It can predict. > it can hide its objectives from interrogators, and convince them that its actions are in their interests; It can predict. > it can deliver enough value to be capable of earning money through its actions; It can predict. > it can propose ideas that can convince investors to part with $100 billion; It can predict. > it can design a chemical plant... It can predict.
  - mitthrowaway2 3y ago
    I don't think a prediction is truly a prediction when it's not being compared against a reference. It's really only a prediction during training; the rest of the time it's synthesis. But again I'll repeat my point: "I also wouldn't underestimate the power of token prediction". It's very well possible that accurate token prediction may be the only necessary fundamental ingredient to weather forecasting, writing a successful novel, compiling a pitch deck for investors, designing a chemical plant... Humans can eat, talk, predict, reproduce, and wiggle our limbs and fingers, but it turns out that there's a lot of complex recipes that you can bake with those ingredients.
    - kubanczyk 3y ago
      Who cares about weather or factories? The missing big ingredient is predicting humans a bit better than another human can. This would unlock a humongous multiplier. All the rest seems peanuts, really. Re: synthesis, I wasn't aware of such distinction at all, it looks more like a misunderstanding.
simiones 3y ago
> I think I'm with Dijkstra on this one: "The question of whether machines can think is about as relevant as the question of whether submarines can swim" Just a nitpick, but this is Turing, not Dijkstra. And it is in fact his argument in the famous "Turing Test" paper - he gives his test (which he calls "the imitation game") as an objective measure of something like AGI instead of the vague notion of "thinking", analogously to how we test successful submarines by "can it move underwater for some distance without killing anyone inside" rather than "can it swim".
- flir 3y ago
  Thanks, that's not a nitpick at all. Can you provide a citation? It's all over the internet as a Dijkstra quote, and I'd like to be correct.
  - 1xdevnet 3y ago
    Fairly certain it is Dijkstra, in his own handwriting in 1984. https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD898.html https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD898... https://www.cs.utexas.edu/~EWD/ewd08xx/EWD898.PDF https://www.cs.utexas.edu/~EWD/ewd08xx/EWD898.PDF
  - simiones 3y ago
    It seems I made some confusions, you were actually right. Apologies...
    - flir 3y ago
      Nah, I think it's good to get the context. I never realised Dijkstra probably had Turing's paper in mind when he said that.

by flir 3y ago

mitthrowaway2 3y ago

I agree we don't know enough to know if I'm right! I tried to use a lot of hedgy-words. But it's not a presupposition, merely a line of argument why it's not a complete absurdity to think LLMs might be a step towards AGI. I do think consciousness is beside the point, as we have no way to test whether LLMs are conscious, just like we can't test anything else. We don't know what consciousness is, nor what it isn't. I don't think Dijkstra's argument applies here. Whether submarines "swim" is a good point about our vague mental boundaries of the word "swim". But submarine propellers are absolutely a convergent structure for underwater propulsion: it's the same hydrodynamic-lift-generating motion of a fin, just continuous instead of reciprocating. That's very much more structurally similar than I expect LLMs are to any hardware we have in our heads. It's true that the solution space for AI is in some ways less constrained than for biological intelligence, but just like submarines and whales operate under the same Navier-Stokes equations, humans and AI must learn and reason under the same equations of probability. Working solutions will probably have some mathematical structure in common. I think more relevant is Von Neumann: "If you will tell me precisely what it is that a machine cannot do, then I can always make a machine which will do just that!" Whether a submarine swims is a matter of semantics, but if there's a manuever that a whale can execute that a submarine cannot, then at least we can all agree about the non-generality of its swimming. For AGI, I can't say whether it's conscious or really thinks, but for the sake of concrete argument, it's dangerous enough to be concerned if: - it can form and maintain an objective; it can identify plausible steps to achieve that objective; it can accurately predict human responses to its actions; it can decently model the environment, as we can; it can hide its objectives from interrogators, and convince them that its actions are in their interests; it can deliver enough value to be capable of earning money through its actions; it can propose ideas that can convince investors to part with $100 billion; it can design a chemical plant that appears at a cursory inspection to manufacture high-profit fluorochemicals, but which also actually manufactures and stores CFCs in sufficient quantity to threaten the viability of terrestrial agriculture.

kubanczyk 3y ago
> it can form and maintain an objective; Yes and yes. > it can identify plausible steps to achieve that objective; It can predict. > it can accurately predict human responses to its actions; It can predict. > it can decently model the environment, as we can; It can predict. > it can hide its objectives from interrogators, and convince them that its actions are in their interests; It can predict. > it can deliver enough value to be capable of earning money through its actions; It can predict. > it can propose ideas that can convince investors to part with $100 billion; It can predict. > it can design a chemical plant... It can predict.
- mitthrowaway2 3y ago
  I don't think a prediction is truly a prediction when it's not being compared against a reference. It's really only a prediction during training; the rest of the time it's synthesis. But again I'll repeat my point: "I also wouldn't underestimate the power of token prediction". It's very well possible that accurate token prediction may be the only necessary fundamental ingredient to weather forecasting, writing a successful novel, compiling a pitch deck for investors, designing a chemical plant... Humans can eat, talk, predict, reproduce, and wiggle our limbs and fingers, but it turns out that there's a lot of complex recipes that you can bake with those ingredients.
  - kubanczyk 3y ago
    Who cares about weather or factories? The missing big ingredient is predicting humans a bit better than another human can. This would unlock a humongous multiplier. All the rest seems peanuts, really. Re: synthesis, I wasn't aware of such distinction at all, it looks more like a misunderstanding.

simiones 3y ago

> I think I'm with Dijkstra on this one: "The question of whether machines can think is about as relevant as the question of whether submarines can swim" Just a nitpick, but this is Turing, not Dijkstra. And it is in fact his argument in the famous "Turing Test" paper - he gives his test (which he calls "the imitation game") as an objective measure of something like AGI instead of the vague notion of "thinking", analogously to how we test successful submarines by "can it move underwater for some distance without killing anyone inside" rather than "can it swim".

flir 3y ago
Thanks, that's not a nitpick at all. Can you provide a citation? It's all over the internet as a Dijkstra quote, and I'd like to be correct.
- 1xdevnet 3y ago
  Fairly certain it is Dijkstra, in his own handwriting in 1984. https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD898.html https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD898... https://www.cs.utexas.edu/~EWD/ewd08xx/EWD898.PDF https://www.cs.utexas.edu/~EWD/ewd08xx/EWD898.PDF
- simiones 3y ago
  It seems I made some confusions, you were actually right. Apologies...
  - flir 3y ago
    Nah, I think it's good to get the context. I never realised Dijkstra probably had Turing's paper in mind when he said that.