3 ms·
The ML community keeps rediscovering the work of Steve Grossberg. This is very similar to his decades-old ART model.
by intalentive 3y ago
The ML community keeps rediscovering the work of Steve Grossberg. This is very similar to his decades-old ART model.
- vardhanw 3y agoCould you explain in simple terms (if possible) what is the similarity? For context, I worked for a brief time with ART before Y2K in BU CNS, and took a few courses there, but had to leave it for 'reasons'.
- intalentive 3y agoSome ART concepts 1) Normalize input (batch norm, 2015) 2) Competitive dynamics / lateral inhibition (softmax in attention layers, 2017) 3) Cluster best matching activation vectors (top-k keys, 2023)