4 ms·
> Either I'm a super genius or this has been discussed before and made it's way into the training data. If an LLM can tell you the relatonship between a compos
by EMM_386 3y ago
> Either I'm a super genius or this has been discussed before and made it's way into the training data.
If an LLM can tell you the relatonship between a compost heap and nuclear bomb, that doesn't mean that was in the training data.
It could be because a compost heap "generates heat", and a nuclear bomb also "generates heat" and due to that relationship they have something in common. The model will pick up on these similar patterns. They tokens are positioned closer to each other in the high dimensional vector space.
But for any given "what does x have in common with y", that doesn't necessarily mean someone has asked that before and it's in the training data. Is that reasoning? I don't know ... how does the brain do it?
- moffkalast 3y ago> how does the brain do it? It's a lot of organic matmuls. ;)
- bamboozled 3y agoI mean that’s what sucks about Open AI isn’t it ? They won’t tell us what is in the training data so we don’t know. All I’m saying is that it wouldn’t be surprising if this was discussed previously somewhere in a pop science book. That answer was close btw !