3 ms·

Did you read carefully? Your second link is a game different than Sumplete's rules, so not sure why you linked it. You have one source, the Digital Trends art

Did you read carefully? Your second link is a game different than Sumplete's rules, so not sure why you linked it. You have one source, the Digital Trends article. It speaks about a mobile game being similar. The link here is at best circumstantial. The app page doesn't describe the rules of the game. Instead it says this: "It is very similar to classical game of Kakuro" I checked Kakuro, and it has similar, but distinctly DIFFERENT rules, as well. So, what's the theory here? GPT downloaded the game and ran it on its phone? No. Maybe the description of this game is somewhere. Maybe. But I couldn't find it. Where is it? The excuse "it's somewhere, so it didn't invent it" will always be used for why an AI can't produce a "novel idea". While the reality of novel ideas is very simple: it's like conservation of matter & energy. You can never make something or destroy it. You can only rearrange it. New ideas are a rearrangement of other existing ideas. It's the case for everything we ever made. And everything we ourselves are.

pksebben 4y ago
This is a discussion in the art world that you'd hear a lot, before stable diffusion came out. The whole idea of "steal like an artist". It's one of those questions about the nature of us that's on the level of "is there a ghost in the machine"; "do we have actually novel ideas?". I'm kind of leaning the way you are; I don't think we do. What seems to be happening that we _call_ novelty is just an advanced form of recursive synthesis - we take information in and remix it. But we're capable of levels of abstraction, so we can remix it really well. I think that getting the abstraction part into these models might just be as simple as wiring them together with a meta-model. A network responsible for identifying similarity between outputs of different types, or one that serves as a connective layer, distributing tokens through other networks and then synthesizing the results. What I'm almost certain of, is that the important changes are going to be architectural.
- BulgarianIdiot 4y ago
  The important changes will be architectural, yes. Three things I'm noticing: 1. We're reinventing structured programming in AI in fast forward. First it's a plain Markov chain. Then it's an "attention" directed acyclic graph. Then we realize we need loops. Then we realize we need to jump to different points in the loop. Then we realize it's useful to recursively call yourself or parts of yourself as a subroutine, parametrized with specific input. Etc. 2. Even before we fully realize this framework of thought into a model, I'm almost sure the model EVOLVES some of these structures during training. In the form of crude unrolled loops etc. Simply because it's inevitable for processing certain types of input data. 3. In order to preserve pragmatic outcomes, I'd bet the future is not one giant monolithic model for AI, but many medium-sized models, communicating in a meta network, like meta neurons, sending meta (high-level) messages to each other. Essentially, we need to make neural networks more like a fractal. I have this rule of thumb that always works somehow: "no concept definition is complete, until it's made recursive". Neural networks will get there.

pksebben 4y ago

This is a discussion in the art world that you'd hear a lot, before stable diffusion came out. The whole idea of "steal like an artist". It's one of those questions about the nature of us that's on the level of "is there a ghost in the machine"; "do we have actually novel ideas?". I'm kind of leaning the way you are; I don't think we do. What seems to be happening that we _call_ novelty is just an advanced form of recursive synthesis - we take information in and remix it. But we're capable of levels of abstraction, so we can remix it really well. I think that getting the abstraction part into these models might just be as simple as wiring them together with a meta-model. A network responsible for identifying similarity between outputs of different types, or one that serves as a connective layer, distributing tokens through other networks and then synthesizing the results. What I'm almost certain of, is that the important changes are going to be architectural.

BulgarianIdiot 4y ago
The important changes will be architectural, yes. Three things I'm noticing: 1. We're reinventing structured programming in AI in fast forward. First it's a plain Markov chain. Then it's an "attention" directed acyclic graph. Then we realize we need loops. Then we realize we need to jump to different points in the loop. Then we realize it's useful to recursively call yourself or parts of yourself as a subroutine, parametrized with specific input. Etc. 2. Even before we fully realize this framework of thought into a model, I'm almost sure the model EVOLVES some of these structures during training. In the form of crude unrolled loops etc. Simply because it's inevitable for processing certain types of input data. 3. In order to preserve pragmatic outcomes, I'd bet the future is not one giant monolithic model for AI, but many medium-sized models, communicating in a meta network, like meta neurons, sending meta (high-level) messages to each other. Essentially, we need to make neural networks more like a fractal. I have this rule of thumb that always works somehow: "no concept definition is complete, until it's made recursive". Neural networks will get there.