tarruda

3 ms·

1.
by tarruda 7d ago
> The thing is that as soon as you seriously give a chance to Bcachefs you see how good it is. I'm pretty sure Bcachefs is amazing and better than Btrfs. I also think Zfs is amazing and better than Btrfs. Even so, I still use Btrfs
2.
by tarruda 7d ago
Except for the fact that the developer has sabotaged the project into being removed from mainline?
3.
by tarruda 7d ago
At this size (~400 million parameters), does it become viable running directly on CPU?
4.
by tarruda 15d ago
The exception is if you use CoW filesystems. Btrfs has a workaround: it can create a cow-disabled subvolume for the swapfile. AFAIK you need a separate partition for Zfs.
5.
by tarruda 15d ago
Awesome!
6.
by tarruda 16d ago
Darn, for a moment I thought it would be possible to run in 128G T.T
7.
by tarruda 16d ago
200B of those 552B is PLE, which works more like a database that is read for each token, thus can be offloaded to a fast SSD.
8.
by tarruda 16d ago
> It also includes additional 196B Engram memory which you can put on an SSD. I think You can put Qwen 3.8 Flash Next engram on SSD, but prompt processing takes a good hit. On my mac studio, I get 300 pp and 33 tg with SSD offload, versu
9.
by tarruda 17d ago
I tried dwarfstar when llama.cpp DSV4 support was still very weak, and while it worked, I didn't see anything that would make me want to stick with it vs llama.cpp. llama.cpp is simply better with its awesome built-in webui, router and
10.
by tarruda 17d ago
Don't use my Vision-Exp GGUF though. As I said I built those GGUFs before llama.cpp supported, and they can't be loaded on current master (require my own branch). I already have new GGUFs but haven't uploaded yet. If you want
11.
by tarruda 17d ago
I would rather have just 3 levels: low, medium and high.
12.
by tarruda 17d ago
IQ3_XXS (~3.2 BPW). For me this is an option because my Mac studio is only used for serving LLMs, so I can afford to dedicate most of its RAM to this. I can run with 256k context and only uses ~117G, with the remaining (up to 125G which I c
13.
by tarruda 17d ago
Hopefully it will be open weights and have the same architecture and size as the current v4 flash vision, which is probably the best LLM that can be run on 128G devices.
14.
by tarruda 19d ago
I use my Mac as a headless server to run LLMs, and never even start a desktop session so it always stays at pre-login state without any applications running. Even so, I never saw it using less than 2G in htop when logged via SSH.
15.
by tarruda 19d ago
I'm certain the video is sped up. Still possible to script computer use, and I think this is what happens. One of the suggests this: https://x.com/DanielSMatthews/status/2096311244504572177 Could be hallucina
16.
by tarruda 19d ago
Interesting, thanks for sharing! A scripting API makes the problem much more approachable, but what about those videos where Astra is drawing people from a photo? Here's one using canva: https://x.com/iam_zachi/sta
17.
by tarruda 20d ago
Amazing project. My only blocker for installing Asahi on my Mac Studio (M1 ultra) is that llama.cpp performance is very bad when compared to using the Metal backend on the same hardware.
18.
by tarruda 20d ago
I haven't tried any of these demos, but I'm not surprised they stop impressing once you go deep. What got be mostly impressed were the demos of Astra doing computer use. At my job I do some RPA and can appreciate how challenging i
19.
by tarruda 20d ago
X.com is full of unverified posts, but here's a few: - https://x.com/MengTo/status/2096213835460084184?s=20 - https://x.com/birdabo/status/2096101381925114327?s=20 Seems really impr
20.
by tarruda 22d ago
> even need to understand that it's "bypassing" because it thinks it's just iterating towards its goal. Could they have added a "no internet access" goal constraint?
21.
by tarruda 22d ago
Even the behavior of agents searching for sandbox bypasses must have been in the training data, or at the very least, "suggested" in some way. To be this whole thing feels like a marketing play by OpenAI.
22.
by tarruda 1mo ago
Also proud of having spent $2.5k on a used 128G M1 Ultra back in September 2024. Despite being outdated in terms of compute, it stills let me run very good recent models locally, with Deepseek V4 Flash 0731 being the greatest one right now,
23.
by tarruda 1mo ago
> not that Apple prices weren’t insane before the ram/ssd shortages Funny that after the price started to increase last year (I think October/November), there was a window of a few months where Apple prices stayed the same as t
24.
by tarruda 1mo ago
Can you share the source for the parameter count (125B A6B)? I didn't see it anywhere in the page.
25.
by tarruda 1mo ago
https://www.youtube.com/watch?v=P4VBqTViEx4
26.
by tarruda 1mo ago
> I’m struggling to understand how that was causing such a large difference in the output. It is incremental, the more a pattern appears in the context, the more likely it was to continue appearing in future turns. So the model was like
27.
by tarruda 1mo ago
There's quite a few tangential features that must be implemented correctly or risk affecting the LLM output in significant ways. Parsing/encoding is one example: A couple of months ago I've debugged a reasoning loop bug in St
28.
by tarruda 1mo ago
> It can run 80-100t/s on a laptop That is a lot, what is your laptop hardware? One issue I have with Gemma is that they seem to use old architectures that rely on full attention, requiring a lot of RAM for context and quickly degra
29.
by tarruda 2mo ago
> It has all of the components of a pelican riding a bicycle, though not exactly arranged in the right order! Maybe a sign that they didn't have SVG pelicans in the dataset
30.
by tarruda 2mo ago
I don't have experience with opencode, so I couldn't tell you. Codex is really good in my experience, especially due to its native sandboxing. Deepseek seems really well versed in its tools, including update_plan and knowing when

More ›