segmondy

17 ms·

541.
by segmondy 1y ago
Don't listen to this crowd, these are "technical folks". Most of your audience will fail to figure it out. You can provide an option that llama.cpp is missing and give them an option where you auto install it or they ca
542.
by segmondy 1y ago
garbage benchmark, inconsistent mix of "agent tools" and models. if you wanted to present a meaningful benchmark, the agent tools will stay the same and then we can really compare the models. there are plenty of other benchmarks
543.
by segmondy 1y ago
AWS CEO says what he has to say to push his own agenda and obviously to align himself with the most currently popular view.
544.
by segmondy 1y ago
zed is just on the hype train, obviously very talented people, they are thinking hard about LLMs but I'm not really sure where they are going, their pivot is probably going to be more interesting...
545.
by segmondy 1y ago
By your argument, once anything makes it in, then it can't be removed. Billions of people are going to use the web every day and it won't stop. Even the most obscure feature will end up being used by 0.1% of users. Can you na
546.
by segmondy 1y ago
Mark the accounts as kids account, they will not collect data till the birthday on the account turns 18. Pick any recent month and 2025 and you get 17+yrs of minimal data collection. They won't turn it on for new data sharing options
547.
by segmondy 1y ago
As someone that has participated in llama.cpp development, it's simple, Ollama doesn't want to give credit to llama.cpp. If llama.cpp went closed, Ollama would fall behind, they blatantly rip llama.cpp. Who cares tho? All the
548.
by segmondy 1y ago
when all you use is one model it feels like magic, when you try different model, you realize no one company owns the magic. these models do feel like magic, but the author should really try this across the many models out there.
549.
by segmondy 1y ago
You are right on the nuance. "try and catch me" for me, means I'm going to give it my all not to be caught and I believe I will not be caught. Good luck, try and catch me. "Try to catch me" means I think you might
550.
by segmondy 1y ago
They are not even remotely the same. "should of" is a phonetic issue. of is spoken with the schwa vowel uh, so the o sounds like uh, and the f takes the v sound, so "should of" sounds like "should uhv", and &
551.
by segmondy 1y ago
FYI, you are probably going to use up your tokens because there's a total limit of tokens per day, so in about 300 requests it's feasible to use it all up. See https://www.reddit.com/r/LocalLLaMA/comment
552.
by segmondy 1y ago
You can run it locally too. Below are a few of my local models, this is coming in light compared to them. At Q4 it's ~60B. Furthermore being a MoE, most of it can be in system memory and only the shared experts needs to go to GPU,
553.
by segmondy 1y ago
Your neighbor is a smart guy.
554.
by segmondy 1y ago
how many tk/sec are you getting on that setup especially when you have 100k+ tokens?
555.
by segmondy 1y ago
all of them, you can even use claude-code with a local model
556.
by segmondy 1y ago
the secret sauce will be the available tools, prompting, context engineering, etc, yup whatever "agentic algorithm" has been built in.
557.
by segmondy 1y ago
it will add the feature, I saw openAI make that claim that developers are adding their own features, saw Anthrophic make the same claim, and Aider's paul often says Aider wrote most of the code. I started building my own coding CLI f
558.
by segmondy 1y ago
why? it's basic, edit the config file. I just downloaded it, ~/.cache/share/crush/providers.json add your own or edit an existing one Edit api_endpoint, done.
559.
by segmondy 1y ago
you are showing how young you are. ;-) I'm glad this is back as someone that grew up in the BBS era, colorful text based stuff brings back joyful memories. I'm building my own terminal CLI coding agent. My plan is to make it
560.
by segmondy 1y ago
An LLM coding cli tool, not agentic, not small primitives to make navigating code base and coding easy and fast. I won't say the future isn't agentic, but I'm still having fun doing it manual.
561.
by segmondy 1y ago
You are not wrong. I hold the exact same sentiment. The world is way more optimistic than us.
562.
by segmondy 1y ago
Doesn't worry me. I believed AI would replace developers and I still do to some degree. But AI is going to lack context, not just in business domain but how it would intersect with the tech side. Experienced developers will be need
563.
by segmondy 1y ago
I'm married, kids, got an elderly parent at end of life that I'm caring for, and so on and so forth. How do I do it? Balance, right now, the kids are packing their bags to go to camp, so I have about 10 mins. I just replied
564.
by segmondy 1y ago
I beg to differ, especially when it comes to code.
565.
by segmondy 1y ago
i'm not willing to trade any more quality for performance. no draft, no cache for kv either. i'll take the performance cost, it just makes me think carefully about my prompt. i rarely every need more than one prompt to get my ans
566.
by segmondy 1y ago
join us at r/LocalLlama
567.
by segmondy 1y ago
beg to differ, I'm living fine with 1.5tk/sec
568.
by segmondy 1y ago
One keeps up with it, by keeping up with it. Folks keep up with latest social media gossip, the news, TV shows, or whatever interests them. You just stay on it. Weekend I got to running Kimi K2, last 2 days I have been driving Ernie4.5-
569.
by segmondy 1y ago
I can say that this really works great, I'm a heavy user of the unsloth dyanmic quants. I run DeepSeek v3/r1 in Q3, and ernie-300b and KimiK2 in Q3 too. Amazing performance. I run Qwen3-235b in both Q4 and Q8 and can barely te
570.
by segmondy 1y ago
no it doesn't, claude uses anthropic API. you need to run an anthropic2openAPI proxy

More ›