4 ms·
I see this is using GPT4o, any plans for something more sustainable? Would be interesting to see an https://openfreemap.org https://openfreemap.org for LLMs. P
by FraserGreenlee 1y ago
I see this is using GPT4o, any plans for something more sustainable?
Would be interesting to see an https://openfreemap.org https://openfreemap.org for LLMs.
Perhaps via an RNN like in https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-2 https://huggingface.co/spaces/BlinkDL/RWKV-Gradio-2
Or even just leverage huggingface gradio spaces? (most are Gradio apps that expose APIs https://www.gradio.app/guides/view-api-page https://www.gradio.app/guides/view-api-page)
- ivape 1y agoI wonder if a 1B model could be close to free to host. That's an eventuality, but I wonder how long it'll take for that to be real.
- giantrobot 1y agoA 1B model at 2-bit quantization is about the size of the average web page anymore. With some WebGPU support you could run such a model in a browser. I'm half joking. Web pages are ludicrously fat these days.
- reverius42 1y agoSomething like this? https://github.com/huggingface/transformers.js-examples/tree/main/llama-3.2-webgpu https://github.com/huggingface/transformers.js-examples/tree...
- giantrobot 1y agoThat was the exact thing I was thinking of but couldn't remember the project name. Thanks!
- codazoda 1y agoI’m planning to deploy a 1B model, feed it all the documents I’ve ever written, host it on a $149 mini-PC in my bedroom, and enable you to chat with it. I’ve released similar projects before. I’ll drop a post about my plans in the coming days and I’ll build and document it about two weeks later if there’s enough interest. joeldare.com
- indigodaddy 1y agoSounds cool!
- codazoda 1y agoThe post about my plan is now online: https://joeldare.com/my_plan_to_build_an_ai_chat_bot_in_my_bedroom https://joeldare.com/my_plan_to_build_an_ai_chat_bot_in_my_b...