4 ms·

Attention Viz

67 points by djoldman 3y ago

mrwnmonm 3y ago
What does attention visualization mean?
GaggiX 3y ago
Did the page load for someone? In the meanwhile I have found the paper page interesting: https://catherinesyeh.github.io/attn-docs/ https://catherinesyeh.github.io/attn-docs/
- banga 3y ago
  The UI tries connecting to http://18.219.70.154:8500/getMatrixData/vit-nat http://18.219.70.154:8500/getMatrixData/vit-nat for the data, but that target appears to be unavailable.
- ape4 3y ago
  There are probably thousands of us trying to load it then bailing part way thru.
  - swyx 3y ago
    maybe its open source and some of us can rehost it?
- swyx 3y ago
  it just loaded for me after 1 failed attempt
Nzen 3y ago
tl;dr a 'parse tree' for how their (transformer based) LLM weights the import of the tokens in a given prompt. The authors noticed patterns in the visualization that they felt imitated relationships that we would regard as related. That's as much as I can summarize, given my ignorance.
- kordlessagain 3y ago
  Use the force! user-aRxb[attentionviz.pdf]> summarize the document's contents bot> Querying GPT... bot> This paper describes AttentionViz, a tool designed to provide a global view of Transformer Attention that can be used to identify sentence view, clusters of search results, semantic behavior, and fine-grained attention patterns; experts have suggested additional interaction modes and global search patterns to be used to quickly scan for and compare attention trends.
kordlessagain 3y ago
I've been working on a project I've called DocGPT: https://github.com/featurebasedb/DocGPT https://github.com/featurebasedb/DocGPT. This is a work in progress and I'll be putting together a release next week, once I've got the instructions for running it done. It was my assumption that GPT-4 could be used for keyterm and question extraction from the fragments of a document and perhaps used to build a semantic graph that could be leveraged for rapid prompt assembly, augmenting the vector searches that are normally done. The idea here is to set "attention flags" on the document during discussion. In this example, I indexed the arxiv PDF for the paper: https://arxiv.org/abs/2305.03210 https://arxiv.org/abs/2305.03210. Here are a few example keyterms it extracted, along with the count of fragments containing the terms: intermediate artifacts of self-attention 1 attention patterns 3 key vectors 5 gpt-2 7 bert 15 Here's are some questions it asked and answered, using keyterms in each fragment to identify relevant document fragments for assembly (in addition to using nearest neighbor vectors): system> How can attention head comparison and model pruning be used to identify anomalies in transformer behaviour? bot> Attention head comparison and model pruning can be used to identify irregularities and potential behavioural issues with transformers through attention pattern exploration, which can provide insights into why large language models fail at reasoning tasks or math. Comparing heads could allow to find parts of the model that are actually useful, and tracking attention flow throughout the model could help with hypothesis testing and visualizing induction head pairs. system> How does Attention Viz help to visualize multiple datasets from different NLP tasks? bot> AttentionViz can identify consistent shapes and patterns across different NLP tasks and can also identify anomalies and unexpected behavior, as well as facilitate attention pattern comparisons. system> What are the mean correlations between query-key distances and dot products for BERT and GPT on Wiki-Auto data? bot> -0.938 for BERT and -0.792 for GPT system> What technologies are used in the Attention Viz prototype? bot> The AttentionViz prototype uses a Python/Flask backend, a Vue/Typescript frontend, a DeepLabv3 segmentation model and Deck.gl to visualize the corresponding query-key joint embeddings.
- qingcharles 3y ago
  This sounds like exactly what I need. I tried ChatPDF last night and paid the Plus fee, but had terrible results, sadly. I have about 9,000,000 PDFs I need summaries of and I can't find a good AI solution as yet...
Lio 3y ago
The site is not responding for me. I'm guessing the page is temporarily hugged to death. In the meantime, I originally thought this might be linked to Viz magazine[1][2]. Spoilers, it's not. Warning: Viz deals with a very specific sort of British humour. 1. https://viz.co.uk/ https://viz.co.uk/ 2. https://en.wikipedia.org/wiki/Viz_(comics) https://en.wikipedia.org/wiki/Viz_(comics)

67 points by djoldman 3y ago

What does attention visualization mean?

Did the page load for someone? In the meanwhile I have found the paper page interesting: https://catherinesyeh.github.io/attn-docs/ https://catherinesyeh.github.io/attn-docs/

banga 3y ago
The UI tries connecting to http://18.219.70.154:8500/getMatrixData/vit-nat http://18.219.70.154:8500/getMatrixData/vit-nat for the data, but that target appears to be unavailable.
ape4 3y ago
There are probably thousands of us trying to load it then bailing part way thru.
- swyx 3y ago
  maybe its open source and some of us can rehost it?
swyx 3y ago
it just loaded for me after 1 failed attempt

Nzen 3y ago

tl;dr a 'parse tree' for how their (transformer based) LLM weights the import of the tokens in a given prompt. The authors noticed patterns in the visualization that they felt imitated relationships that we would regard as related. That's as much as I can summarize, given my ignorance.

kordlessagain 3y ago
Use the force! user-aRxb[attentionviz.pdf]> summarize the document's contents bot> Querying GPT... bot> This paper describes AttentionViz, a tool designed to provide a global view of Transformer Attention that can be used to identify sentence view, clusters of search results, semantic behavior, and fine-grained attention patterns; experts have suggested additional interaction modes and global search patterns to be used to quickly scan for and compare attention trends.

kordlessagain 3y ago

I've been working on a project I've called DocGPT: https://github.com/featurebasedb/DocGPT https://github.com/featurebasedb/DocGPT. This is a work in progress and I'll be putting together a release next week, once I've got the instructions for running it done. It was my assumption that GPT-4 could be used for keyterm and question extraction from the fragments of a document and perhaps used to build a semantic graph that could be leveraged for rapid prompt assembly, augmenting the vector searches that are normally done. The idea here is to set "attention flags" on the document during discussion. In this example, I indexed the arxiv PDF for the paper: https://arxiv.org/abs/2305.03210 https://arxiv.org/abs/2305.03210. Here are a few example keyterms it extracted, along with the count of fragments containing the terms: intermediate artifacts of self-attention 1 attention patterns 3 key vectors 5 gpt-2 7 bert 15 Here's are some questions it asked and answered, using keyterms in each fragment to identify relevant document fragments for assembly (in addition to using nearest neighbor vectors): system> How can attention head comparison and model pruning be used to identify anomalies in transformer behaviour? bot> Attention head comparison and model pruning can be used to identify irregularities and potential behavioural issues with transformers through attention pattern exploration, which can provide insights into why large language models fail at reasoning tasks or math. Comparing heads could allow to find parts of the model that are actually useful, and tracking attention flow throughout the model could help with hypothesis testing and visualizing induction head pairs. system> How does Attention Viz help to visualize multiple datasets from different NLP tasks? bot> AttentionViz can identify consistent shapes and patterns across different NLP tasks and can also identify anomalies and unexpected behavior, as well as facilitate attention pattern comparisons. system> What are the mean correlations between query-key distances and dot products for BERT and GPT on Wiki-Auto data? bot> -0.938 for BERT and -0.792 for GPT system> What technologies are used in the Attention Viz prototype? bot> The AttentionViz prototype uses a Python/Flask backend, a Vue/Typescript frontend, a DeepLabv3 segmentation model and Deck.gl to visualize the corresponding query-key joint embeddings.

qingcharles 3y ago
This sounds like exactly what I need. I tried ChatPDF last night and paid the Plus fee, but had terrible results, sadly. I have about 9,000,000 PDFs I need summaries of and I can't find a good AI solution as yet...

Lio 3y ago

The site is not responding for me. I'm guessing the page is temporarily hugged to death. In the meantime, I originally thought this might be linked to Viz magazine[1][2]. Spoilers, it's not. Warning: Viz deals with a very specific sort of British humour. 1. https://viz.co.uk/ https://viz.co.uk/ 2. https://en.wikipedia.org/wiki/Viz_(comics) https://en.wikipedia.org/wiki/Viz_(comics)