3 ms·
Feed forward networks are pairwise linear in respect to inputs, they are effectively DAGs. In theory you could represent an entire LLM as a single 2 dimensiona
by nyrikki 3y ago
Feed forward networks are pairwise linear in respect to inputs, they are effectively DAGs.
In theory you could represent an entire LLM as a single 2 dimensional graph of linear line segments.
It wouldn't be useful for much as the parameters are clustered in dense patches.
- kridsdale3 3y agoSo are my synapses, and by and large my cortex is 2-dimensional as well (folded and scrunched in to a super complex shape by the limitations of my roughly spherical cranium)
- nyrikki 3y agoEverything from ion channel response up is non-linear and not a deterministic Turing machine. Random cite https://www.jneurosci.org/content/13/1/334?ijkey=9f8730dd3d028900055afd202f57b98d5498b776&keytype2=tf_ipsecsha https://www.jneurosci.org/content/13/1/334?ijkey=9f8730dd3d0...