hypfer

15 ms·

301.
by hypfer 2mo ago
Yes to both. The thing is that I can either use the q8 context, or have not enough context window, so I just live with whatever degradation there is. The same can be said about the IQ4_NL. I would not go any lower though. As for the draft c
302.
by hypfer 2mo ago
What makes you say that it would be hard to do that? It's long, I guess, but not cryptic. You tell llama server where the model is, which context size to use, what to use for the K/V cache quant, that it should do MTP, tune some M
303.
by hypfer 2mo ago
According to this shitty vibecoded thing "I" built https://hypfer.github.io/will-it-fit-llama-cpp/ (and I guess according to math too), FP16 K/V would give me something like 90k context at the same model
304.
by hypfer 2mo ago
Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 17
305.
by hypfer 2mo ago
The sentence still doesn't make sense, because "settled on" implies a long testing phase with a verdict eventually emerging out of that. What you're currently doing is "testing out"
306.
by hypfer 2mo ago
> I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting Dude, GLM-5.3 released _today_. The phrasing "I've settled on" is incorrect for this context.
307.
by hypfer 2mo ago
And with that also gain institutional knowledge, skill up your workers and attract talent that wants to work on this stuff. All boils down to short-term/long-term thinking.
308.
by hypfer 2mo ago
I've been persistently insulting Opus 4.8 lately, since it started(?) constantly speaking incomprehensible gibberish and noise. No amount of telling it to phrase stuff differently seems to help there anymore. So either I am seeing patt
309.
by hypfer 2mo ago
Are those watermarks why claude suddenly started being even more unbearable to work with lately? Man. That would make a lot of sense indeed.
310.
by hypfer 2mo ago
I might be just reading my positive bias into that text, but is it possible that it is written less like SV marketing hype trash and more like researchers wrote it? It does feel like it respects both me and my time. Thank you, Z.AI. Amazing
311.
by hypfer 2mo ago
We will try anyway. No need to try to FUD people into passive acceptance of the cloud.
312.
by hypfer 2mo ago
Maybe this is to keep script kiddies away? Like.. having no readme or a super technical one doesn't work anymore in the age of LLMs, but having one that hurts to read might? Because people trying to get an LLM to translate it just get
313.
by hypfer 2mo ago
If you revisit my comment and pay attention to the opener: > but you might not be aware that llama-server can do multi-model for a while now you will see that the sentence structure clearly implies both a change compared with a prior sta
314.
by hypfer 2mo ago
^ This. Intent is the answer and AI has none.
315.
by hypfer 2mo ago
Being legally liable might work. Might "harm innovation" though.
316.
by hypfer 2mo ago
No, it starts much earlier. They should not have given a next token predictor unsupervised Internet access, regardless of what the marketing might have told them about its capabilities or alignment. Don't use technology you do not unde
317.
by hypfer 2mo ago
AI just doing what 17yo hackers with skills but lack of fully developed morals would've done. Just without the skin in the game. ...And with a fully grown adult requesting it to do so with zero care or due diligence. Just a whoopsie af
318.
by hypfer 2mo ago
Old news by now, but you might not be aware that llama-server can do multi-model for a while now, Meaning that you (and by that I mean your AI agent that has read the llama.cpp code) can write an ini file pointing to your models with parame
319.
by hypfer 2mo ago
> whose only job is to say no to some other group I suspect that people will miss just how precise you've been there. Your proposal - if I understand correctly - is not just to spread the "ethics team" onto everyone (which
320.
by hypfer 2mo ago
Having played around with this model a bit, I am fairly confident that it is not competing in the coding space. It can do that, but its actual selling point appears to be a different take on guardrails and safety alignment. Either that or t
321.
by hypfer 2mo ago
Wouldn't it make more sense to instead use the same GDID on thousands of hosts? Or a random GDID on each read? No GDID sounds like it sticks out a lot more.
322.
by hypfer 2mo ago
On HN, a TLDR would be helpful, because just clicking your links would potentially mean falling for shilling of ones own side-hustle. I bet there's complexity to the topic, but I am sure you could put it into a few words instead of jus
323.
by hypfer 2mo ago
I can see why you're saying that but a) Using some other horizontal line is not in the class of "wrong" that would warrant this sort of reaction. You're debating the pixel count. Not the object. The aspect ratio of it. b
324.
by hypfer 2mo ago
> This is not a defense, but nor is "this character is associated with slop" an argument. You're misunderstanding me. It's not an argument. I have no strong opinions about it. I am telling you how people perceive it.
325.
by hypfer 2mo ago
I mean you would stop using the OK Hands thing when trying to communicate with a group that has for (possibly arbitrary) reasons decided that that symbol is bad, and thus anything you say is bad, right? At least temporarily until you got sy
326.
by hypfer 2mo ago
I just do not comprehend how that can be the hill people choose to die on. Like.. I get it. It's wrong. It feels wrong to give the AI nonsense just an inch and all. But in the grand scheme of things, it's just highly ineffective d
327.
by hypfer 2mo ago
> and this post will go down as my not-the-proudest, however, I take your criticism to heart—although not personally—and strive to improve. I just do not understand why people - even when actively denying LLM usage - use the fucking em d
328.
by hypfer 2mo ago
And yield. Good luck getting good yield out of such ginormous chips. That said, I've been wondering if they could go with multiple smaller ones instead. Like one per layer maybe even? What are your thoughts on that? You seem to be more
329.
by hypfer 2mo ago
I'm somewhat doubtful that we will be seeing something as large as Kimi K3 in silicon any time soon. This tech can definitely scale up from the current 8B prototype, but - at least as far as my limited understanding of the tech involve
330.
by hypfer 2mo ago
Ha, thanks, I completely forgot that that was a thing. I got hit by that too, which was incredibly funny and a great throwback to early 4chan culture. So fwiw, these things at least do breed creativity. Same as with the aforementioned &quo

More ›