3 ms·

Running the model multiple times doesn't reinforce the model. In general, you should not anthropomorphize algorithms as human cognition does not give any bearin

Running the model multiple times doesn't reinforce the model. In general, you should not anthropomorphize algorithms as human cognition does not give any bearing on how algorithms work.

Scene_Cast2 5y ago
It can. Check out "zero shot learning" -> both sentences would be part of a single "evaluation", and the first sentence would prime for the output of the second. (You basically combine multiple "evaluations" into one, and context is held in tensors / blobs) https://towardsdatascience.com/zero-and-few-shot-learning-c08e145dc4ed https://towardsdatascience.com/zero-and-few-shot-learning-c0...
- hervature 5y ago
  Sure, but I feel like we're talking about different things. I consider "context held in tensors" as part of the model. That is, if you zero out these registers, then the model evolves in a deterministic way every time. In this case, when you perform a query, I assume those tensors are always initialized before your query.
twofornone 5y ago
>you should not anthropomorphize algorithms as human cognition does not give any bearing on how algorithms work I disagree. As an ML engineer I see vast parallels between how neural networks essentially parameterize high dimensional joint probability distributions, and how humans reason about the world. Including the learning phase. Through gradual exposure to data, it seems humans and deep nets are both mapping examples to similar topological representations - as evidenced by, for example, totally accidental learning of token arithmetic by large language models, e.g. king + female = queen. I think this humanlike behavior is effectively a form of rudimentary machine reasoning and it makes me rather hopeful that we have stumbled upon the early components of general artificial intelligence. All of human reasoning can be emulated with complex joint probability distributions. Even emotions boil down to heuristics on these same probabilistic relationships. Though artificial neurons and learning are only very loose approximations of biological neurons and the hebbian principle, I think it's clear at this point that the way that neural networks compress information is already quite humanlike; and furthermore I would hazard a guess that this similarity is a profound manifestation of the nature of information in our reality.

Running the model multiple times doesn't reinforce the model. In general, you should not anthropomorphize algorithms as human cognition does not give any bearing on how algorithms work.

Scene_Cast2 5y ago

It can. Check out "zero shot learning" -> both sentences would be part of a single "evaluation", and the first sentence would prime for the output of the second. (You basically combine multiple "evaluations" into one, and context is held in tensors / blobs) https://towardsdatascience.com/zero-and-few-shot-learning-c08e145dc4ed https://towardsdatascience.com/zero-and-few-shot-learning-c0...

hervature 5y ago
Sure, but I feel like we're talking about different things. I consider "context held in tensors" as part of the model. That is, if you zero out these registers, then the model evolves in a deterministic way every time. In this case, when you perform a query, I assume those tensors are always initialized before your query.

twofornone 5y ago

>you should not anthropomorphize algorithms as human cognition does not give any bearing on how algorithms work I disagree. As an ML engineer I see vast parallels between how neural networks essentially parameterize high dimensional joint probability distributions, and how humans reason about the world. Including the learning phase. Through gradual exposure to data, it seems humans and deep nets are both mapping examples to similar topological representations - as evidenced by, for example, totally accidental learning of token arithmetic by large language models, e.g. king + female = queen. I think this humanlike behavior is effectively a form of rudimentary machine reasoning and it makes me rather hopeful that we have stumbled upon the early components of general artificial intelligence. All of human reasoning can be emulated with complex joint probability distributions. Even emotions boil down to heuristics on these same probabilistic relationships. Though artificial neurons and learning are only very loose approximations of biological neurons and the hebbian principle, I think it's clear at this point that the way that neural networks compress information is already quite humanlike; and furthermore I would hazard a guess that this similarity is a profound manifestation of the nature of information in our reality.