4 ms·
I wonder if emergent abilities are just the result of better clustering. For example, a clustering algorithm with rich datasets could improve the quality and nu
by didnotreadit 4y ago
I wonder if emergent abilities are just the result of better clustering. For example, a clustering algorithm with rich datasets could improve the quality and number of clusters so to provide emergent abilities. As the number of parameters in the LLM is increased and the trained allows the embedding to construct a richer hypothesis space, the clustering algorithm could construct new centers that are the origin of the emergent capabilities.
Also it could be related to the percolation threshold (1)
(1) https://en.wikipedia.org/wiki/Percolation_threshold https://en.wikipedia.org/wiki/Percolation_threshold