4 ms·

Show HN: Run LLaMa2 on the Browser with Ggml.js

You can now build serverless AI inference web application with ggml.js's LM backends.

gardenfelder 3y ago
Somewhat amusing: Me: What are the key bullet points? A substantial percentage of patients with ulcerative colitis (UC) have irritable bowel syndrome- (IBS-) like symptoms despite adequate treatment and endoscopic remission. In this study, we evaluated the clinical efficacy of probiotic therapy for residual IBS-like symptoms in patients with UC in endoscopic remission. LLaMa2 (edit typo) What are the key bullet points? A substantial percentage of patients with ulcerative colitis (UC) have irritable bowel syndrome- (IBS-) like symptoms despite adequate treatment and endoscopic remission. In this study, we evaluated the clinical efficacy of probiotic therapy for residual IBS-like symptoms in patients with UC in endoscopic remission. eat. When Halloween arrived,
- sigmoid10 3y ago
  It's not the "actual" LLaMA2, although you have too look at the code to see what it is: It's a miniature mock up trained on the tinystories dataset. The model is orders of magnitude smaller than even the smallest official LLaMA2 release and it should honestly be an immediate giveaway if anyone thought that the real stuff would actually run inside browsers on today's hardware. Quantization or not, we're just not there yet.
quickthrower2 3y ago
What is up with that, seems too fast and not that accurate. Is it the smallest model?
- pawelduda 3y ago
  This is the model used: https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin https://huggingface.co/karpathy/tinyllamas/resolve/main/stor... Makes sense why I ask it to answer 2+2 and it tells me stories about Bob, Alice and so on.
  - quickthrower2 3y ago
    Lol I was going to say it had a tinyGPT feel about it
    - anonymousd3vil 3y ago
      Lol Okay, I should have been clear in the title. It is the TinyStories version of the LLAMA2. Its not good quick maths for sure.
      - PhilippGille 3y ago
        This would also be useful on the website, which currently says > we use ggml.js library to run llama2.c model on browser. Just mention the model.
        SushiHippie 3y ago
        I guess the url contains "tinystories", well, not that obvious.

You can now build serverless AI inference web application with ggml.js's LM backends.

Somewhat amusing: Me: What are the key bullet points? A substantial percentage of patients with ulcerative colitis (UC) have irritable bowel syndrome- (IBS-) like symptoms despite adequate treatment and endoscopic remission. In this study, we evaluated the clinical efficacy of probiotic therapy for residual IBS-like symptoms in patients with UC in endoscopic remission. LLaMa2 (edit typo) What are the key bullet points? A substantial percentage of patients with ulcerative colitis (UC) have irritable bowel syndrome- (IBS-) like symptoms despite adequate treatment and endoscopic remission. In this study, we evaluated the clinical efficacy of probiotic therapy for residual IBS-like symptoms in patients with UC in endoscopic remission. eat. When Halloween arrived,

sigmoid10 3y ago
It's not the "actual" LLaMA2, although you have too look at the code to see what it is: It's a miniature mock up trained on the tinystories dataset. The model is orders of magnitude smaller than even the smallest official LLaMA2 release and it should honestly be an immediate giveaway if anyone thought that the real stuff would actually run inside browsers on today's hardware. Quantization or not, we're just not there yet.

quickthrower2 3y ago

What is up with that, seems too fast and not that accurate. Is it the smallest model?

pawelduda 3y ago
This is the model used: https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin https://huggingface.co/karpathy/tinyllamas/resolve/main/stor... Makes sense why I ask it to answer 2+2 and it tells me stories about Bob, Alice and so on.
- quickthrower2 3y ago
  Lol I was going to say it had a tinyGPT feel about it
  - anonymousd3vil 3y ago
    Lol Okay, I should have been clear in the title. It is the TinyStories version of the LLAMA2. Its not good quick maths for sure.
    - PhilippGille 3y ago
      This would also be useful on the website, which currently says > we use ggml.js library to run llama2.c model on browser. Just mention the model.
      - SushiHippie 3y ago
        I guess the url contains "tinystories", well, not that obvious.