diggan

24 ms·

421.
by diggan 1y ago
I'm not sure I'd be surprised, I've been playing around with GPT-OSS last few days, and the architecture seems really fast for the accuracy/quality of responses, way better than most local weights I've tried for the
422.
by diggan 1y ago
> for instance, they have broad general knowledge about science, but don’t know much about popular culture That seems like a good focus. Why learn details that can change within days of it being released? Instead, train the models to hav
423.
by diggan 1y ago
> I find coding to be harder to benchmark because there are so many ways to write the same solution. A "correct" solution may be terrible in another context due to loss of speed, security, etc. Yeah, I also do my own benchmarks
424.
by diggan 1y ago
> but for the moment here’s the pelican I got from GPT-5 running at its default “medium” reasoning effort: Would been interesting to see a comparison between low, medium and high reasoning_effort pelicans :) When I've played around
425.
Steam Trailer Player Upgrades(store.steampowered.com)
5 points by diggan 1y ago|0 comments
426.
by diggan 1y ago
Seems they're doing rollout over time, I'm not seeing it anywhere yet.
427.
by diggan 1y ago
> Did they just say they're deprecating all of OpenAI's non-GPT-5 models? Yes. But it was quickly mentioned, not sure what the schedule is like or anything I think, unless they talked about that before I started watching the li
428.
by diggan 1y ago
Not even 10 seconds after I started watching the stream, someone said how much more human GPT-5 is, while the people sitting and talking about it don't seem human at all, and it's not an accent/language thing. Seems they&#x27
429.
by diggan 1y ago
Was just skimming along that review, while watching the live-stream, where they just mentioned how much better at writing prose GPT-5 is, while I skimmed across: > It’s actually worse at writing than GPT-4.5 Sounds like we need to wait a
430.
by diggan 1y ago
Hmm, deprecating all previous models because GPT-5 is launched feels like a big move. I wonder how the schedule for the deprecation will look like.
431.
by diggan 1y ago
> they don't actually constrain the behavior They do actually constraint the behavior, to various degrees of success which depends on the model, the system prompt, the inference parameters, the current context length and a lot mor
432.
by diggan 1y ago
Ah, it only works for GGUF, not for .safetensors (which the format HuggingFace themselves came up with :P ) ? I see the checks at https://huggingface.co/unsloth/gpt-oss-20b-GGUF but nothing at https://huggin
433.
by diggan 1y ago
Yeah, what you're suggesting sounds like it could be more useful than the "generalized calculators" people are currently publishing and using.
434.
by diggan 1y ago
> GPT-OSS is not much of a snowflake Yeah, according to the architecture it doesn't seem like a snowflake, but they also decided to invent a new prompting/conversation format ( https://github.com/openai/harm
435.
by diggan 1y ago
Ah, interesting. As someone with a RTX Pro 6000, is it ready today to be able to run gpt-oss-120b inference, or are there still missing pieces? Both linked PRs seems merged already, so unsure if it's ready to be played around with or n
436.
by diggan 1y ago
I tend to use my own "prompt management CLI" ( https://github.com/victorb/prompta ) to setup somewhat reusable prompts, then paste the output into whatever UI/CLI I use at the moment. Then rewriting/p
437.
by diggan 1y ago
This is where you start to fine-tune the weights, you can get pretty great results when it comes to specific tool calls with the right data.
438.
by diggan 1y ago
> The community needs to fight this framework off. Why? The people who been around for a while, already avoid it because they've either tried it before, or poked around in the source and then we ran away quickly. If people start usi
439.
by diggan 1y ago
> Can you elaborate on how the agents degrades from more tools? The more context you have in the requests, the worse the performance, I think this is pretty widely established at this point. For best accuracy, you need to constantly prun
440.
by diggan 1y ago
> I think it's fair to point out that they were never intended to be a good framework for aligning robots and humans. Even in his own stories they lead to problems. They were created precisely to make the point that encoding these t
441.
by diggan 1y ago
> Instead of us programming the AIs by feeding it lots of explicit hand-crafted rules/instructions, we're feeding the things with plain data instead, and the resulting behavior is much more black-box, less predictable and less
442.
by diggan 1y ago
Depends on the model. Each runner needs to implement support when there are new architectures, and they all seemingly focuses on different things. As far as I've gathered so far, vLLM focuses on inference speed, SGLang on parallizing a
443.
by diggan 1y ago
Lets say MCP is used for 100% of the ecosystem's need of tool calling, who exactly is earning the money for that usage?
444.
by diggan 1y ago
Interesting, never knew about that! I filled out my details, then went to https://huggingface.co/openai/gpt-oss-120b but I'm not sure if I see any difference? Where is it supposed to show if I can run it or not?
445.
by diggan 1y ago
Sure, but it still sits in the 'business-grade hardware whose main purpose is AI training or running inference for LLMs" segment parent mentioned, yet have graphics connectors so the only thing I'm saying is that just looking
446.
by diggan 1y ago
> And flash attention doesn't work on 5090 yet, right? Flash attention works with GPT-OSS + llama.cpp (tested on 1d72c8418) and other Blackwell card (RTX Pro 6000) so I think it should work on 5090 as well, it's the same archit
447.
by diggan 1y ago
Maybe I'm spoiled by having great internet connection, but I usually download the weights and try to run them via various tools (llama.cpp, LM Studio, vLLM and SGLang typically) and see what works. There seems to be so many variables i
448.
by diggan 1y ago
> LMStudio/llama.cpp Even though LM Studio uses llama.cpp as a runtime, the performance differs between them. With LM Studio 0.3.22 Build 2 with CUDA Llama.cpp (Linux) v1.45.0 runtime I get ~86 tok/s on a RTX Pro 6000, while wi
449.
by diggan 1y ago
Although the RTX Pro 6000 is not consumer-grade, it does come with graphics ports (four Displayports) and does render graphics like a consumer card :) So seems the difference between the segments is becoming smaller, not bigger.
450.
by diggan 1y ago
> Do we know WHY openAI even released them? Regulations and trying to earn good will of developers using local LLMs, something that was slowly eroding since it was a while ago (GPT2 - 2019) they released weights to the public.

More ›