5 ms·
Show HN: GPT-2 implemented using graphics shaders
Back in the old days, people used to do general-purpose GPU programming by using shaders like GLSL. This is what inspired NVIDIA (and other companies) to eventually create CUDA (and friends).
This is an implementation of GPT-2 using WebGL and shaders. Enjoy!
- throwaway314155 1y agoThis is pretty rad. I've built and run the project on my MacBook and can't seem to get the visualizations to show as they appear in the GitHub demonstration. I'm running Firefox but I think that should support WebGL features you're using? Happy to file an issue on the repository as well if you'd prefer that.
- nathan-barry 1y agoYeah it'd be great if you could file an issue and show what you see! I was using Safari but I'm gonna check other browsers to test if it works on them
- throwaway314155 1y agoGreat! I've filed an issue.
- SamosaGuru 1y agoWould be really cool to have it hosted on a website if it can be a static page so we could try it out
- nathan-barry 1y agoWorking on getting it up on Github pages, will let you know. Not sure if there will be an issue with hosting the weights, I'll look into it
- Philpax 1y agoYou may be able to fetch the weights directly from Hugging Face. I'd try that first.
- ianand 1y agoCheckout https://github.com/jseeio/gpt2-tfjs https://github.com/jseeio/gpt2-tfjs fetches the weights for GPT2 from huggingface on the fly.
- jasonjmcghee 1y agoI haven't looked at your implementation, but if you can read ONNX weights... https://huggingface.co/Xenova/gpt2 https://huggingface.co/Xenova/gpt2
- Philpax 1y agoI'm reminded of this VRChat world, which runs Qwen2-0.5B in a shader: https://vrchat.com/home/launch?worldId=wrld_e919c1f4-8399-43cc-8305-d890dc8e2e98 https://vrchat.com/home/launch?worldId=wrld_e919c1f4-8399-43...
- ianand 1y agoAs the guy who did GPT2 in Excel, very cool and kudos!! Curious why you chose WebGL over WebGPU? Just to show it can be done? (Also see my other comment about fetching weights from huggingface)
- ronsor 1y agoProbably because WebGPU support is still rather iffy.
- nathan-barry 1y agoThis was a final project for a graphics class where we used WebGL a lot. Also I was just more familiar with OpenGL and haven't looked that much into webGPU
- littlestymaar 1y ago> Curious why you chose WebGL over WebGPU? Just to show it can be done? For a WebGPU implementation, one can use transformers.js directly (or many other libraries actually), maybe WebGL is more original. [1]: https://huggingface.co/docs/transformers.js/index https://huggingface.co/docs/transformers.js/index
- 85392_school 1y agoTransformers.js wraps the ONNX runtime which is rather versatile (WASM, WebGL, WebGPU, and WebNN). It's not the backend that makes it novel.
- _dijs 1y agoianand, I immediately thought of you when I saw this post. Miss you friend.
- ianand 1y agoDude, been forever. Thanks. Will DM you.
- divan 1y ago
- Narishma 1y agoNow do it with the fixed function pipeline so I can run it on my 3dfx Voodoo card.
- deleted 1y ago[deleted]
- helloplanets 1y agoWould absolutely read a 'making of' post on this project!
- jasonjmcghee 1y agoIncredible. Was going to suggest tossing up a github page with it, but looks like one is currently in-progress? (https://nathan-barry.github.io/gpt2-webgl/ https://nathan-barry.github.io/gpt2-webgl/) but doesn't seem to load for me currently. Looks like it's trying to load the typescript directly / hosting the source files: `<script type="module" src="src/main.ts"></script>` - instead of hosting the dist folder.
- nathan-barry 1y agoYeah, right now the weights aren't even pushed to the repo so there's no way for the github page to load them in. I've looked a tiny bit into it and might end up making a fix for it, having it load the weights by fetching them from somewhere else on load, but probably not today. There is a link to the zipped up folder with the weights here: https://github.com/nathan-barry/gpt2-webgl/releases/download/v1.0/weights.tar.gz https://github.com/nathan-barry/gpt2-webgl/releases/download... If anyone wants to make a PR which fetches the file from there, unzips it, then loads in the weights from there that would be greatly appreciated.
- badmonster 1y agoSuper cool project! How much of the inference time is actually GPU-bound vs CPU-bound in the browser? Curious if WebGPU would offer a big perf boost over WebGL2 here.
- deleted 1y ago[deleted]
- gitroom 1y agoNice, that takes some serious grinding to pull off in shaders. Makes me kinda wonder - does pushing old tools like WebGL actually force deeper understanding of how these models work compared to just grabbing a new library?
- supuun 1y ago> Back in the old days, people used to do general-purpose GPU programming by using shaders like GLSL. This is what inspired NVIDIA (and other companies) to eventually create CUDA (and friends). I learned about this at the lecture last week. So fun to see this project.