4 ms·
How is that an argument to the comment you replied to?
by gtirloni 3mo ago
How is that an argument to the comment you replied to?
- JacobAsmuth 3mo agoAstra is clearly able to acquire new knowledge in context and apply it. It was the whole thing that his ARC-AGI benchmarks have been measuring. It's a direct refutation of the original comment.
- fny 3mo agoNone of these LLMs are plastic. They lack neurodiversity. Their thought space and their traversal are likely constrained in someway that humanity's isn't as a collective.
- angusturner 3mo agoWhat does this even mean. There is strong evidence of LLMs doing in context learning. Some of the linear RNN layers in recent models are provably doing SGD in hidden space during inference
- oblio 3mo ago> What does this even mean. There is strong evidence of LLMs doing in context learning. 1. Is this learning persistent? 2. Do they verify these new lessons against core principles? 3. Do they and protect themselves/ignore requests if these new lessons contradict those core principles? Humans do that from the time they're 3 years old (not that well, but they do do it).
- NuclearPM 3mo agoYes. Yes. Yes.
- oblio 3mo agoIn my experience all those claims are false. So the next step is to ask for evidence and ideally independent and peer reviewed research.
- CamperBob2 3mo agoYou know they can take notes, right? And ICL dates all the way back to 2020, at least: https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165
- oblio 3mo agoThat's training during training. They can't learn afterwards.
- deleted 3mo ago[deleted]
- DoctorOetker 3mo agodo you have a reference where that claim is demonstrated?
- gtirloni 3mo agoBy in context learning do you mean latent space? https://transformer-circuits.pub/2021/framework/index.html https://transformer-circuits.pub/2021/framework/index.html
- Yiin 3mo agohow do you imagine that would work? you being able to. influence globally stored weights with some prompts? we have fine-tuning for that.
- oblio 3mo agoYet I can't randomly order another person to steal a car for me, just because I tell them to. Alignment for an intelligent system is a hard problem and at this stage is seems close to unsolvable. My guess is that we'll just ignore it and make money along the way and every 2-3 months we'll have the equivalent to "Equifax gets hacked and millions of user records are stolen", etc. (this time with the LLM itself doing the hacking at someone's behest - accidental or not).
- fzeindl 3mo agoIs it possible to have neuroplasticity and still keep them aligned?
- fny 3mo agoI have neuroplasticity, and I am aligned. ;)
- olmo23 3mo agoHumans are famously horribly aligned, plenty of examples in history.
- chrisweekly 3mo agoHmm. Examples of horrible alignmnent don't necessarily outweigh the fact that most people, most of the time, mostly behave in a way that is socially aligned. (Though I'm speaking in terms of intent, conveniently ignoring the side effects / negative externalities of our collective behavior.)
- gtirloni 3mo agoWhen threatened with physical or social harm. What they think in their little heads though :)
- ZiiS 3mo agoBut are other people?
- fzeindl 3mo agoAre you (at least partially) aligned out of existential fear of repercussions (getting fired, losing your life, going to prison), which LLMs don't have?
- alienbaby 3mo agosurely it can only acquire new knowledge if it were updating it's weights as it is used?