5 ms·
Finding the Words to Say: Hidden State Visualizations for Language Models
- jalammar 6y agoHello HN, author here. Language models are absolutely fascinating tools. I believe it would pay for software engineers to have a sense of their capabilities and how they function. The article showcases a few views to expose the inner workings of the model, but also simple UI for interacting with a language model to get a sense for how they work and generate words. If you prefer video, I have also recently released a video [1] with PyData to provide an intro to language models and their applications and how we're trying to make Transformer-based ones more transparent with Ecco[2]. Contributors are welcome! [1] https://www.youtube.com/watch?v=rHrItfNeuh0 https://www.youtube.com/watch?v=rHrItfNeuh0 [2] https://www.eccox.io/ https://www.eccox.io/ and https://github.com/jalammar/ecco https://github.com/jalammar/ecco Thanks mods for merging submissions. Happy to get feedback , thoughts, or questions.
- deepnet 6y agoNice article, thanks for posting :-)
- amelius 6y agoDoes some text generating tool exist which takes a badly written piece of prose (perhaps even in the form of bullet-points), and produces a high(er) quality text? (E.g., richer in vocabulary by using synonyms, removing repetition, rephrasing sentences so they don't all have the same general form, et cetera.)
- jalammar 6y agoI didn't come across one yet personally