spmurrayzzz

6 ms·

31.
by spmurrayzzz 9mo ago
Preferences I think I get, but prejudices? The OED defines prejudice as a "preconceived opinion that is not based on reason or actual experience." My day to day work involves: full stack web dev, distributed systems, embedded syst
32.
by spmurrayzzz 9mo ago
I think you're shadow-boxing with a point I never made. I never said experienced devs are not or can not be excited about current AI capabilities. Lots of experienced devs who work in more difficult domains are excited about AI. In fac
33.
by spmurrayzzz 9mo ago
My comment above I hope wasn't read to mean "LLMs are only good at web dev." Only that there are different capability magnitudes. I often do experiments where I will clone one of our private repos, revert a commit, trash the
34.
by spmurrayzzz 9mo ago
This has also been an interesting social experiment in that we get to see what work people think is actually impressive vs trivial. Folks who have spent years effectively snapping together other people’s APIs like LEGOs (and being well-comp
35.
by spmurrayzzz 10mo ago
Those cards can be great for lots of use cases, plenty of small models are very capable at the param counts which can fit in 32GB of VRAM. GPT-OSS-20B for example is a serviceable model for agentic coding use cases and it runs natively in M
36.
by spmurrayzzz 11mo ago
There is an older paper on something related to this [1], where the model outputs reflection tokens that either trigger retrieval or critique steps. The idea is that the model recognizes that it needs to fetch some grounding subsequent to g
37.
by spmurrayzzz 1y ago
I generally agree with this just from a perspective of personal sentiment, it does feel wrong. But statistically speaking, at a 95% confidence level you'd be within a +/- 3.5% margin of error given the 791 sample size, irrespectiv
38.
by spmurrayzzz 1y ago
> I spent at least an hour trying to get OpenCode to use a local model and then found a graveyard of PRs begging for Ollama support Almost from day one of the project, I've been able to use local models. Llama.cpp worked out of the
39.
by spmurrayzzz 1y ago
Ah didn't realize you were referring to NoPE explicitly. And yea, the intuitions gained from that paper are pretty much what I alluded to above, you don't get away with never modeling the positional data, the question is how you m
40.
by spmurrayzzz 1y ago
Do you have a citation for the paper on that? IME, that's not really something you see used in practice, at least not after 2022 or so. Without some form positional adjustment, transformer-based LLMs have no way to differentiate from &
41.
by spmurrayzzz 1y ago
Very much agree re: inscrutability. It gets even more complicated when you add the LLM-specific concept of rotary positional embeddings to the mix. In my experience, it's been exceptionally hard to communicate that concept to even tech
42.
by spmurrayzzz 1y ago
Might end up being some confusion with the RULER benchmark from NVIDIA given the (somewhat shared) domain: https://github.com/NVIDIA/RULER EDIT: by shared I only mean the adjacency to LLMs/AI/ML, RL is a pret
43.
by spmurrayzzz 1y ago
Concrete example from my own workflows: in my IDE whenever I accept or reject a FIM completion, I capture that data (the prefix, the suffix, the completion, and the thumbs up/down signal) and put it in a database. The resultant dataset
44.
by spmurrayzzz 1y ago
Sure. I didn't imply (or didn't mean to imply at least) that I thought MRCR was solved, only pointing out that it's closer to testing raw retrieval than it is testing long range dependency resolution like Longproc does. If re
45.
by spmurrayzzz 1y ago
We're a G-suite shop so I set aside a ton of time trying to get 2.5 pro to work for us. I'm not entirely unhappy with it, its a highly capable model, but the long context implosion significantly limits it for the majority of task
46.
by spmurrayzzz 1y ago
The NIAH performance is a misleading indicator for performance on the tasks people really want the long context for. It's great as a smoke/regression test. If you're bad on NIAH, you're not gonna do well on the more hol
47.
by spmurrayzzz 1y ago
I'd be curious what the results would be with the automated Autogluon fit/evals. I suspect given the results here, a weighted average model would likely win out.
48.
by spmurrayzzz 1y ago
Absolutely agree with that last sentiment. Recently I came across a project by a former google engineer called Dyad: https://github.com/dyad-sh/dyad This is built to be a lovable/bolt alternative and is definitely
49.
by spmurrayzzz 1y ago
I agree. And the real lede was buried here IMO: > This is the single most impressive code-gen project I’ve seen so far. I did not think this was possible yet. To get that sort of acclaim, a human had to build an embedded programming lang
50.
by spmurrayzzz 1y ago
Looks very buttoned up. My local project has some features tuned for my explicit agent flows however (built directly into my inference engine), so can't really jump ship just yet. Looking great so far though!
51.
by spmurrayzzz 1y ago
Yea the proxying/observability is without question the simplest part of this whole problem space. Once you get into the weeds of automating all the eval and prompt optimizing, you realize how irrelevant wireshark actually is in the fee
52.
by spmurrayzzz 1y ago
There's no relation per se (apart from AMD once being involved, before asking the developer to take the project offline). It is worth noting however that they accomplish their goals very differently. ZLUDA is now DoA-ware because of nv
53.
by spmurrayzzz 1y ago
I think the fact that spurious rewards were predominantly only effective for Qwen may suggest that it was triggering some shift in its language distribution. If you use those models long enough you'll see a ton of mandarin that makes i
54.
by spmurrayzzz 1y ago
Supporting local models can be done by overriding one or two environment variables, as long as your local inference server has an OpenAI-compliant endpoint (which the majority of local stacks ship with). Was there some level of support beyo
55.
by spmurrayzzz 1y ago
I've always found it fascinating how frequently I've seen the complaint about Go re: boilerplate and unnecessary toil, but in previous statements Rust was uttered with an uncritical breath. I agree with the complaint about Go, but
56.
by spmurrayzzz 1y ago
The aspirational goal is that the model knows what tools to call and when, without human intervention. In practice, you'll see varying efficacy with that depending on the tools you need. Some of the tool usage is in-distribution /
57.
by spmurrayzzz 1y ago
Zed strikes a good balance I think with their non-agentic text threads feature. You use slash commands to add whatever you need to the context (single files, directories, highlighted selections, etc) and interact with a conventional chat in
58.
by spmurrayzzz 1y ago
You can just use llama.cpp instead (which is what ollama is using under the hood via bindings). Just need to make sure youre using commit `d3bd719` or newer. I normally use this with nvidia/cuda, but tested on my mbp and havent had any
59.
by spmurrayzzz 2y ago
Assuming you're referring to the backend side of things for online/multiplayer games, working through the Hathora tutorials in the docs can be a good first introduction to general idioms related to backend systems architecture: h
60.
by spmurrayzzz 2y ago
Yea it certainly does benefit Claude Desktop to some degree, but most MCP servers are a few hundred SLOC and the protocol schema itself is only ~400 SLOC. If that was the only major obstacle standing in the way of adoption, I'd be very

More ›