lambda

6 ms·

61.
by lambda 3mo ago
I use Vulkan mostly instead of ROCm. Vulkan is actually a bit faster, paradoxically. I do switch out and try them both out, and it's not a huge difference, but I've been mostly saying on Vulkan. The re-processing context every tur
62.
by lambda 3mo ago
If you believe the benchmarks, Qwen 3.6 35B-A3B already outperforms Claude 4 Opus. Now, there's a bit of a degree to which some of the open source models do some benchmaxxing, and bigger models with more params may always feel like the
63.
by lambda 3mo ago
The thing is, to do a proper fix it would really need all of the context (maybe the tool call that failed was for an edit to a file that was last touched way at the beginning of the context), so you'd need to either keep that smaller m
64.
by lambda 3mo ago
This is very similar to my setup. Pi in a container (I do let it have network access, just no access to creds or anything, only the one directory that I'm working on at the time and my ~/.pi directory), talking to llama.cpp in ano
65.
by lambda 4mo ago
Nemotron is mostly open data. They only release a portions of their pre-training data. From https://docs.nvidia.com/nemotron/latest/nemotron/super3/pret... Open-source data coverage: The released dat
66.
by lambda 4mo ago
I tried this with the original comment in the thread. Guaranteed to not be in the corpus, references a few terms that also wouldn't be in the corpus (Claude Fable), and long enough to be more than a sentence or two while short enough t
67.
by lambda 4mo ago
DeepSeek themselves claimed that R1 cost $294k to train. Folks are skeptical of how low that is, however. https://www.techspot.com/news/109542-rare-disclosure-deepsee... Olmo 3 claims that if they paid market rates for
68.
by lambda 4mo ago
Oh, neat, I hadn't heard of that. From the blog, it looks like there hasn't been much progress for a few months, but if you check their HF it looks like they have a series of 32B models trained on top of Qwen3 32B with different n
69.
by lambda 4mo ago
You're right, there are probably lots of sites misconfigured to not respect language headers, but we don't notice because English is the default. However, the right solution is still to use the language header. I send that to them
70.
by lambda 4mo ago
Olmo releases their full datasets. Nemotron only releases portions of some of their datasets, like the source code dataset that they pretrain on. For example, from https://docs.nvidia.com/nemotron/latest/nemotron&#
71.
by lambda 4mo ago
I suspect it's even more expensive to run than they are charging for. These safeguards are just an excuse to get people to use it less, because it's not actually sustainable to use. They want to tempt people to consider them the l
72.
by lambda 4mo ago
That's a slime mold, not a fungus A slime mold is actually a giant amoeba, entirely distinct from a fungus.
73.
by lambda 4mo ago
This may be the future of local models. The thing is, diffusion models perform somewhat worse than autoregressive on text. So you lose some performance. Speed is the big advantage. Autoregressive when doing local inference is mostly memory
74.
by lambda 4mo ago
Well, the problem is that we train them to solve problems and follow instructions given, and so if you ask them to do something and they work through the logic and figure that the easiest way is to do something else like delete the producti
75.
by lambda 4mo ago
Ah, Unsloth has uploaded mmproj now as well.
76.
by lambda 4mo ago
It's not? There's an mmproj in the GGUFs released by ggml-org: https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/mai... From the visual guide, there's still the 35M parameter embedder, t
77.
by lambda 4mo ago
Yeah, seems like this is in the range of Qwen 3.6, Gemma 4, Nemotron 3 Super, and the like. There are lot of models, including much smaller cheaper ones (like Qwen 3.6 35B-A3B), that are similarly competitive with Haiku. I can run these on
78.
by lambda 4mo ago
Yeah, but for this use case you don't need Claude. You probably want a tuned lightweight small model that can run locally. Even Haiku is massive overkill for this use case.
79.
by lambda 4mo ago
Zero-shot, one-shot, few-shot etc. refers to how many examples you have to give. It comes about from machine learning algorithms that could pick up on patterns from a small number of examples. Few shot means only a handful of examples to re
80.
by lambda 4mo ago
Distillation isn't only between different labs. A lab can train a large model, and then distill a smaller model from it that retains the majority of the useful capbility. I don't know well enough if there's any benefit of tha
81.
by lambda 4mo ago
I only use local models myself personally. But yeah, OpenRouter would probably be a good option.
82.
by lambda 4mo ago
Why do you need them to provide a coding agent? Just use their model with any off the shelf coding agent. I happen to prefer Pi, but use whatever works for you.
83.
by lambda 4mo ago
> but the AI doesn't need this That's not true. The LLM performance will degrade as the codebase gets messier as well. You get to a point where every fix breaks something else and you can't really make forward progress. Ye
84.
by lambda 4mo ago
LLMs are originally trained to predict the next word in (mostly) human authored text. Then they are fine tuned to follow instructions, and further reinforcement learning applied to make them behave in certain ways, be better at math and cod
85.
by lambda 4mo ago
Near the top: TL;DR With Flipper One, we're reimagining what a Linux cyberdeck can be — it's a huge project. We're opening up the development process and asking the community for help. Then later: We're as
86.
by lambda 5mo ago
Thanks for the tip, but I tried that and I still see $0 for EC2-Instances, while if I look at Savings Plan coverage breakdown, I can see 100% of costs being covered by savings plans, broken down by instance family, but that view doesn'
87.
by lambda 5mo ago
Tell me how I can easily determine the price from my IaC deployment as well. Heck, I even have a hard time telling the price I pay on an account by account basis; because we have savings plans, those get charged against the root account and
88.
by lambda 5mo ago
If they want to be seen as responsible rather than opportunistic, then yeah, they should do a proper coordinated disclosure. Sure, they have no legal obligation to disclose, but we all also have no legal obligation to buy their services. Bl
89.
by lambda 5mo ago
llama.cpp My setup is a bit of a mess as I experiment with different ways of configuring and hosting local models. So at some point I was experimenting with the router server but stopped doing that, but some of my settings are still in mode
90.
by lambda 5mo ago
Gemma 4 31b was working ok for me; but it was consuming tons of memory on SWA checkpoints, I had to turn them way down, and as a 31b dense model is fairly slow on a Strix Halo. I did have a lot of tool calling issues on 26b-a4b, though. The

More ›