5 ms·
Show HN: Tiny LLMs – Browser-based private AI models for a wide array of tasks
- TOMDM 3y agoWhich one of these is an LLM? The youtube summary one maybe? Doesn't seem to run in the browser.
- bilater 3y agoThe other tools section is just extra helpful links to other stuff I've done.
- ilaksh 3y agoNone of these are LLMs. LLM means Large Language Model (for text generation, like ChatGPT).
- kordlessagain 3y agoLooks like we've hit full kleenex on terms for models, maybe. Even if it were a tiny LLM, which there are none on that site, it would technically be a SLM, for small language model.
- ilaksh 3y agoHere is an actual LLM that runs in the browser: https://webllm.mlc.ai/#chat-demo https://webllm.mlc.ai/#chat-demo I cannot make this work on Chrome in Linux. I even tried with --enable-dawn-features=allow_unsafe_apis --enable-unsafe-webgpu
- senseiV 3y agoNice, whats the text to video model? Also, you could try to go for a 1b llm for the browser, would fit.
- bilater 3y agoGood idea! Mistral 1B is supported by transformers.js now so I can add it! The Text to Video model is a pipeline Text to Speech + FFMPEG Magic to stitch together a video.
- woadwarrior01 3y agoWhat’s Mistral 1B? I only know of Mistral 7B.
- bilater 3y agoI think it might be a < 1B fine tuned model. Read about it in this release https://twitter.com/xenovacom/status/1722661501180256311 https://twitter.com/xenovacom/status/1722661501180256311
- woadwarrior01 3y agoThe announcement seems somewhat disingenuous. The PR[1] found from their release notes[2] seems to contain only boilerplate and no real support for Mistral models or their weights. [1]: https://github.com/xenova/transformers.js/pull/379 https://github.com/xenova/transformers.js/pull/379 [2]: https://github.com/xenova/transformers.js/releases/tag/2.8.0 https://github.com/xenova/transformers.js/releases/tag/2.8.0
- albertzeyer 3y agoThe name "Tiny LLMs" does not make sense. LLM stands for large language model. So tiny is kind of the opposite of large. Large usually means >=7B parameters (2023). If it's less, we would just call it language model, not large language model. And language model has a very specific meaning: It models text. Usually we mean an auto-regressive language model, i.e. input is partial text, and output is the the prediction of the following text. Although there are also other kind of language models. Language model always means text-only. E.g, a model for speech recognition is a speech recognition model, not a language model. You might use a language model in addition here (shallow fusion etc), but it's not necessary (end-to-end models). None of the models I see here are language models. So, to put a title here, you could maybe "neural network models in browser" or so. Or maybe "natural language models". "Natural language processing (NLP)" has a different meaning (https://en.wikipedia.org/wiki/Natural_language_processing https://en.wikipedia.org/wiki/Natural_language_processing) (compared to "language model", https://en.wikipedia.org/wiki/Language_model https://en.wikipedia.org/wiki/Language_model) and includes speech recognition. Sometimes this was also referred to as "human language technology (HLT)" (https://en.wikipedia.org/wiki/Language_technology https://en.wikipedia.org/wiki/Language_technology), which would also include all those models. It would also be nice to add a bit more details on what kind of models we see here, how large they are, etc. E.g. for speech recognition, I see that this uses a port of Whisper to the web (https://github.com/xenova/whisper-web https://github.com/xenova/whisper-web) based on the Transformers.js library (https://github.com/xenova/transformers.js https://github.com/xenova/transformers.js). That uses ONNX, and the standard conversion is via Hugging Face Optimum (https://github.com/huggingface/optimum https://github.com/huggingface/optimum), and that usually would do some dynamic quantization, i.e. compression of the model. So maybe the "tiny large" is referring to that. But I did not really find out which Whisper model this is based on, i.e. whether it is Whisper-large (which is still not too large with 1.5B parameters).
- unshavedyak 3y agoI could see TinyLLM meaning a method of reducing/compression the effective runtime size of an LLM. Ie quantization stuff, etc. No idea what this is, tho.
- andy99 3y ago
- politelemon 3y agoTiny Models would have been a better name. LLMs are a specific type of model that generate text. Or Tiny GenAI to capitalize on the marketing names?
- jackconsidine 3y agoThe speech recognition tool is impressive! I've tried many different attempts at browser-local speech-to-text from Ermine.ai to Google's Tensorflow.js speech toolkit (recognition only), and this is the best I've seen. And it works on a standard GPU.
- bsndev 3y ago[dead]
- sunshadow 3y agoOriginal project: https://github.com/xenova/transformers.js https://github.com/xenova/transformers.js
- raybb 3y agoLove that the speech to text even works in FF on Mac, and rather well I'd say.
- anigbrowl 3y agoYou should have some text explaining what is going and how they are running locally on the GPU - otherwise everyone sharing the link will need to write an explanation of why they're sharing it. Otherwise, fine work.
- supermatt 3y agoThese aren't LLMs...
- gsuuon 3y agoThis is cool but named pretty poorly. It doesn't seem to have anything to do with LLM's - TinyAIs would've been perfect.
- hilti 3y agoPerfect fit. But some jokers try to sell tinyai.com for $120,000