bashbjorn

6 ms·

1.
Use fewer threads for CPU inference(nobodywho.ai)
6 points by bashbjorn 1mo ago|0 comments
2.
by bashbjorn 5mo ago
There is an RSS feed now: https://nobodywho.ooo/feed.xml
3.
by bashbjorn 5mo ago
It's just an eleventy site: https://github.com/nobodywho-ooo/website No RSS feed currently, but it's a good idea to add one!
4.
by bashbjorn 5mo ago
Cool technique, but I'm not sure I'd call it simple. Doing this means that you can't just tokenize the string output of the chat template as one big string. You might need to tokenize things separately, and combine them after
5.
by bashbjorn 5mo ago
You're right, there must be a good and simple way to do it. Obviously the prefix-with-backslash convention won't do it. The escaping system could be something like inserting a character on the second position in the text repr, and
6.
by bashbjorn 5mo ago
The model sees one token per marker - but the overlap with ingested actual text is still relevant, because the tokenizer will ingest regular text, where it will turn "<|turn>" into the same token. For this reason, it can be
7.
by bashbjorn 5mo ago
Yeah, TheBloke era of local LLMs were good times. TBF Unsloth are doing a fantastic job of publishing quants of the major models quickly - they just don't have nearly the volume of "weird" models as TheBloke did.
8.
by bashbjorn 5mo ago
I love mistral, but that model is... not the best. Maybe try out Gemma 4 e4b, it's a similar size to Mistral 7B, and should run great on your 4070 ("E4B" is slightly misleading naming).
9.
by bashbjorn 5mo ago
whoops, my bad. Just a typo in the markdown. Fixed :)
10.
What's in a GGUF, besides the weights – and what's still missing?(nobodywho.ooo)
195 points by bashbjorn 5mo ago|58 comments
11.
by bashbjorn 6mo ago
Oh hey I'm also working on a thing to solve the devx of llama.cpp: https://github.com/nobodywho-ooo/nobodywho In contrast to Ollama, this is a self-contained library, not a server. I wrote some quick notes on this
12.
by bashbjorn 9mo ago
NobodyWho | Copenhagen, Denmark | Software Engineer, ML Engineer, DevRel | Full-time | ONSITE NobodyWho is making developer tools for running small language models in local-first applications. Our core principle is to ship the model weights
13.
by bashbjorn 1y ago
I made something silly to this effect last year: https://gitlab.com/AsbjornOlling/nixllm You provide the prompt to a nix function, and it deterministically generates the source code using llama.cpp.
14.
by bashbjorn 2y ago
I'm working on a plugin[1] that runs local LLMs from the Godot game engine. The optimal model sizes seem to be 2B-7B ish, since those will run fast enough on most computers. We recommend that people try it out with Gemma 2 2B (but it w
15.
by bashbjorn 2y ago
What makes you say that these are all Stenbergs creations? Could it be that these are just projects that use libcurl in some way? I'm having trouble finding any sources that say that Daniel Stenberg actually worked on spotify, utorrent
16.
by bashbjorn 2y ago
Neat. I wish that I could test the regex immediately in the browser. Tools like regexr.com are great for convincing myself that my regex works.
17.
by bashbjorn 3y ago
Python, Typescript, Nim, Rust, C++, C and Elm.
18.
by bashbjorn 3y ago
The MNT Reform[1] is a pretty sexy open hardware laptop with a mechanical keyboard, optional trackball, and a small ARM processor. A cursory glance at geekbench shows that you can get around 50% the performance of a 2020 Macbook Air M1, if
19.
by bashbjorn 5y ago
I did AoC in Nim one year and was very happy with it! I also became infatuated with Nim that year..
20.
by bashbjorn 5y ago
I highly recommend it to explore new programming styles in general. As some others, I use it as an excuse to learn a new language each year. While I think python is a really great language for advent of code, I'm not sure I'd reco
21.
Cactus Comments: Matrix Spec Changes we're excited for(cactus.chat)
6 points by bashbjorn 5y ago|1 comments
22.
by bashbjorn 6y ago
Shameless plug: My project, Cactus Comments[1] does something similar, although for commenting, instead of support chat. We ship a web-embeddable Matrix client. The users' browser connects directly with a Matrix server of their choice
23.
by bashbjorn 6y ago
Hey, thanks! It's a small web I guess.
24.
by bashbjorn 6y ago
Aforementionend friend and Cactus Comments dev here. We don't support any sort of threading yet, although Cerulean-style threading is definitely somewhere down the road. Although stuff like redactions and emoji reactions are a higher p
25.
by bashbjorn 6y ago
There is one bundled with a standard nim installation, and I use it regularly (when working in nim). It works, but it's not very good. I expect the developers to know this, since its accessible only though the `$ nim secret` command.
26.
by bashbjorn 7y ago
I'm currently studying a CS(-ish) bachelors degree (and I'm currently procrastinating on a group assignment) so I have some experience with this. Our uni gives us quite a lot of group assignments. The university-wide solution to t
27.
by bashbjorn 7y ago
Agreed. Although I'd much sooner recommend just using Mopidy. Mopidy supports spotify and many other music services, and has a number of low-effort ways of controlling playback programmatically. But for sure the interesting part of thi
28.
by bashbjorn 7y ago
This is neat and all, but their widget.js file is 888KB. For comparison, Bootstrap is 180KB, Elm is 29KB and Vue.js is 100KB. It might not be much for sites otherwise dealing with large multimedia assets, but for anyone going for lean &#x2F