4 ms·

I want to see more models that can be streamed to a browser and run locally via wasm. That would be my hope for small models. In the <100mb range.

• deleted 1y ago
  [deleted]
• firejake308 1y ago
  After experimenting with 1B models, I am starting to think that any model with 1B parameters or less will probably lack a lot of the general intelligence that we observe in the frontier models, because it seems physically impossible to encode that much information into so few parameters. I believe that in the range of very small models, the winner will be models that are fine tuned to a small range of tasks or domains, such as a model that can translate between English and any other language, or a legal summarization model, etc.
• relaxing 1y ago
  Why? Just so user data stays local?
  • dainiusse 1y ago
    Yes. And also, cost to run it.
    • relaxing 1y ago
      Sounds like a case where egress could easily cost more than compute.
• vindex10 1y ago
  Have you heard of Transformers.js? They are running onnx inside browser: https://huggingface.co/docs/transformers.js/en/index https://huggingface.co/docs/transformers.js/en/index