3 ms·
I'd love to watch a LLM run in WebGL where everything is Textures. Would be neat to visually see the difference in architectures.
by koolala 2y ago
I'd love to watch a LLM run in WebGL where everything is Textures. Would be neat to visually see the difference in architectures.
- vanviegen 2y agoWouldn't that be just like watching static noise?
- archerx 2y agoI think some patterns would appear.
- Twirrim 2y agoWhy would you expect patterns to appear?
- koolala 2y agoEverything is a different pattern. Noise isn't random - we just call 1 very specific unique pattern of it 'random'.
- archerx 2y agoI was really into Convolutional Neural Networks back in the day and when you visualize the layers you’d get interesting patterns. I’m guessing the same would happen with LLMs but the patterns will be a lot more abstract but still some what recognizable. Here’s an example of visualizing CNN layers; https://www.researchgate.net/figure/Visualization-of-example-features-of-eight-layers-of-a-deep-convolutional-neural_fig2_279068412 https://www.researchgate.net/figure/Visualization-of-example...
- iml7 2y agoDoesn't Google have a tool that allows you to check the activation status of the matrix? Gemma scope