4 ms·

Transformer – Spreadsheet

250 points by next_xibalba 2y ago

antidnan 2y ago
You can even build GPT-2 in Excel: https://www.youtube.com/watch?v=yjD2n_e9E3w&ab_channel=Spreadsheetsareallyouneed https://www.youtube.com/watch?v=yjD2n_e9E3w&ab_channel=Sprea...
- xnx 2y ago
  Still a marvel. https://news.ycombinator.com/item?id=39700256 https://news.ycombinator.com/item?id=39700256
esafak 2y ago
Their title says Spreadsheet, and it uses Google Sheets, so I don't know why it says Excel here.
- dang 2y ago
  We've reverted it now. (Submitted title was "Transformer Implemented Using Excel")
fragmede 2y ago
props for using the /copy endpoint feature of Google docs. saves a lot of hassle when teaching
fangpenlin 2y ago
So, it turns out "Spreadsheet Is All You Need"
rmnclmnt 2y ago
On the related topic to teach AI fundamentals and intuition of the inner workings, there is a wonderful material named "AI Unplugged"[1] to perform activities with pen/pencils, cards, etc. in a game manner. I've been using this material on several occasions with various audiences not familiar with the field of AI/ML (kids and grown-ups), and each time people seem to have been enjoying it and earning a bit of understanding of the modern world they live in. [1] http://www.aiunplugged.org http://www.aiunplugged.org
nobodywillobsrv 2y ago
Genuine question: What do people feel they understand after going through this? If we wrote out every matrix multiplication for linear regression, would we say we've truly grasped it? Is the takeaway about implementation mechanics, or does it build deeper intuition? I see the value in visualization, but what’s the real educational gain beyond that? (I'm an ML person/mathematician, but I haven’t lectured in over a decade—maybe I just don’t get it. I tend to prefer tutorials that build connections to known ideas, so this might be a blind spot for me.)
- remuskaos 2y ago
  I think it's supposed to quite the opposite, at least it was like that for me. When gpt3 first launched, it seemed almost like magic for me, and I have some limited ml background too. Much later I saw the 3blue1brown Video about transformers, and I was almost disappointed to see the math itself is rather simple. My main take away was that even simple basics can produce astonishing results, I this case if they're just scaled large enough. That incredibly complex and useful emergent behavior can result from what seems like conways-game-of-life like principles.
  - goatlover 2y ago
    It's the large training data which contains the knowledge for that complex and useful emergent behavior. It's like if you could import vast information about the world into Conway's game of life to enable increasingly complex levels of emergence.
    - elcritch 2y ago
      Oh idea of "vast patterns" in Conway's game of life seems like a great metaphor for LLM transformers. Explaining hidden markov chains with some extras or whatnot to a layman doesn't give a good mental image but is what I guess it's like. Probability chains alone don't seem to give a good mental visualization of how such a system comes to certain "decisions" or "thought patterns". But watching the Game of Life you can see fascinating patterns which emerge and lead to interesting patterns. That's easy to extrapolate. Maybe in the future NNs will be understable sorta like Game of Life, "oh that NN section is running pattern 27 on XY input data. That's gonna be an unstable behavioral element combined with pattern 38c over here." Not sure if that's a fascinating or dreadfully boring prospect though.
nyclounge 2y ago
One can also argue this can be done within db such as postgres have everything store procedures/functions/triggers. In some ways DBs are just Spreadsheets with really robust querying languages.
raju 2y ago
Similar HN thread: https://news.ycombinator.com/item?id=42967173 https://news.ycombinator.com/item?id=42967173
deleted 2y ago
[deleted]
alok-g 2y ago
See also: (Classical) Computer vision basics in Excel, using just formulas: https://github.com/amzn/computer-vision-basics-in-microsoft-excel https://github.com/amzn/computer-vision-basics-in-microsoft-... Original HN Submission: https://news.ycombinator.com/item?id=22357374 https://news.ycombinator.com/item?id=22357374 PS: I am the primary author.

250 points by next_xibalba 2y ago

You can even build GPT-2 in Excel: https://www.youtube.com/watch?v=yjD2n_e9E3w&ab_channel=Spreadsheetsareallyouneed https://www.youtube.com/watch?v=yjD2n_e9E3w&ab_channel=Sprea...

xnx 2y ago
Still a marvel. https://news.ycombinator.com/item?id=39700256 https://news.ycombinator.com/item?id=39700256

esafak 2y ago

Their title says Spreadsheet, and it uses Google Sheets, so I don't know why it says Excel here.

dang 2y ago
We've reverted it now. (Submitted title was "Transformer Implemented Using Excel")

fragmede 2y ago

props for using the /copy endpoint feature of Google docs. saves a lot of hassle when teaching

fangpenlin 2y ago

So, it turns out "Spreadsheet Is All You Need"

rmnclmnt 2y ago

On the related topic to teach AI fundamentals and intuition of the inner workings, there is a wonderful material named "AI Unplugged"[1] to perform activities with pen/pencils, cards, etc. in a game manner. I've been using this material on several occasions with various audiences not familiar with the field of AI/ML (kids and grown-ups), and each time people seem to have been enjoying it and earning a bit of understanding of the modern world they live in. [1] http://www.aiunplugged.org http://www.aiunplugged.org

nobodywillobsrv 2y ago

Genuine question: What do people feel they understand after going through this? If we wrote out every matrix multiplication for linear regression, would we say we've truly grasped it? Is the takeaway about implementation mechanics, or does it build deeper intuition? I see the value in visualization, but what’s the real educational gain beyond that? (I'm an ML person/mathematician, but I haven’t lectured in over a decade—maybe I just don’t get it. I tend to prefer tutorials that build connections to known ideas, so this might be a blind spot for me.)

remuskaos 2y ago
I think it's supposed to quite the opposite, at least it was like that for me. When gpt3 first launched, it seemed almost like magic for me, and I have some limited ml background too. Much later I saw the 3blue1brown Video about transformers, and I was almost disappointed to see the math itself is rather simple. My main take away was that even simple basics can produce astonishing results, I this case if they're just scaled large enough. That incredibly complex and useful emergent behavior can result from what seems like conways-game-of-life like principles.
- goatlover 2y ago
  It's the large training data which contains the knowledge for that complex and useful emergent behavior. It's like if you could import vast information about the world into Conway's game of life to enable increasingly complex levels of emergence.
  - elcritch 2y ago
    Oh idea of "vast patterns" in Conway's game of life seems like a great metaphor for LLM transformers. Explaining hidden markov chains with some extras or whatnot to a layman doesn't give a good mental image but is what I guess it's like. Probability chains alone don't seem to give a good mental visualization of how such a system comes to certain "decisions" or "thought patterns". But watching the Game of Life you can see fascinating patterns which emerge and lead to interesting patterns. That's easy to extrapolate. Maybe in the future NNs will be understable sorta like Game of Life, "oh that NN section is running pattern 27 on XY input data. That's gonna be an unstable behavioral element combined with pattern 38c over here." Not sure if that's a fascinating or dreadfully boring prospect though.

nyclounge 2y ago

One can also argue this can be done within db such as postgres have everything store procedures/functions/triggers. In some ways DBs are just Spreadsheets with really robust querying languages.

raju 2y ago

Similar HN thread: https://news.ycombinator.com/item?id=42967173 https://news.ycombinator.com/item?id=42967173

deleted 2y ago

[deleted]

alok-g 2y ago

See also: (Classical) Computer vision basics in Excel, using just formulas: https://github.com/amzn/computer-vision-basics-in-microsoft-excel https://github.com/amzn/computer-vision-basics-in-microsoft-... Original HN Submission: https://news.ycombinator.com/item?id=22357374 https://news.ycombinator.com/item?id=22357374 PS: I am the primary author.