dannyw

12 ms·

151.
by dannyw 2mo ago
We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company. The amount of workloads we can shift with an advisor model pattern continues to grow. It’
152.
by dannyw 2mo ago
Still cheaper than API rates for Opus!
153.
by dannyw 2mo ago
I really, really like DSv4 Flash because you see the full, real thinking text. That’s been so useful for helping steer the model; as well as seeing its thoughts and correcting any errors, or expanding on it. It’s so difficult for me to use
154.
by dannyw 2mo ago
Especially for minor workloads that’s like using a dollar of tokens a day. Not worth it unless you’re unhappy with performance or you have nothing else to do.
155.
by dannyw 2mo ago
If you have a validator for if a JPEG is a valid proof in mathematics, and can generate lots of "plausible" JPEGs, then you can arrive at hew results in mathematics.
156.
by dannyw 2mo ago
Security and compliance tends to be a lot less subjective than ethics. Not saying it’s objective, but there’s a huge difference. The downside is it can often feel like a box-checking exercise than actual security or compliance, but “you nee
157.
by dannyw 2mo ago
CUDA has many problems, but I would say less problems than ROCm, etc; even before considering the ecosystem and that more people (or open source projects) have already solved CUDA's problems for you. Ironically, there was an open sourc
158.
by dannyw 2mo ago
I think it should be well defined. I personally think prediction markets for say, federal and state elections, are a net benefit. It provides another source off information, and there’s much less insider trading risk. “”Event outcome predic
159.
by dannyw 2mo ago
Yes. If you take Kalshi out of it, that’s exactly how the law is supposed to work. The State of New York does not have the power to compel NYSE to stop operating in New York either, irrespective of what laws NY passed, as that is with the S
160.
by dannyw 2mo ago
The order actually does legally require Kalshi to continue operating, and unless they successfully appeal, Kalshi would be breaking federal law. Market regulators do have this power.
161.
by dannyw 2mo ago
You don’t even get _any_ logits with closed models for years now. I can’t fault them too much, as logit based distillation is extremely effective. Very useful for making smaller models out of bigger open weight models.
162.
by dannyw 2mo ago
Yeah, it’s also useful for prompt tuning, debugging and understanding how a model interprets your prompt. Also really good for identifying any contradictions in your system prompt and context.
163.
by dannyw 2mo ago
Trace Inversion is fascinating, but it’s more of an independent reconstruction that will give you some coherent-looking generated CoT; but not necessarily anywhere close or related to the underlying model’s CoT.
164.
by dannyw 2mo ago
The provider has the hidden text anyway; this isn’t customer managed encryption.
165.
by dannyw 2mo ago
It’s already patched according to the authors. Details were not specified.
166.
by dannyw 2mo ago
That’s incompatible with zero data retention and so you’ll lose a lot of enterprise customers.
167.
by dannyw 2mo ago
Amongst Chinese labs and netizens, there's MUCH less belief/mindshare on "AGI = existential risk to humanity", "paperclip maximiser", and similar lines of thinking. AI is seen more as just a technology, and les
168.
by dannyw 2mo ago
So many people have outdated notions of what LLMs can do; and tbh, I think a lot of it comes down to free models being weaker.
169.
by dannyw 2mo ago
It's not a conversational model. It's meant as a local tool calling model.
170.
by dannyw 2mo ago
Two DGX sparks will run the full Deepseek V4 Flash full. (It is definitely expensive, but relatively easy and compact; and extremely power efficient)
171.
by dannyw 2mo ago
One of the frontier companies (Anthropic) is already doing prompt injections on the API, which you pay for. Right now, the presence of these injections are still visible: count the API's returned tokens/billing data, and you'
172.
by dannyw 2mo ago
I'd be skeptical w.r.t. "by all metrics". Qwen3.6 is a definitive, significant downgrade from Qwen3.5 for creative writing and prose for example. Yes, it's better at agentic and coding, but it regresses in many non-codi
173.
by dannyw 2mo ago
Some labs go through bad patches, GDM is definitely in one right now and the recent departures are not reassuring, but I think it's too early and dismissive to count them out of the race so far. They just need one good frontier release
174.
by dannyw 2mo ago
Don’t sleep on NVIDIA and Nemotron. It’s not completely open source, but they actually release their pretraining and post-training datasets with some redactions for (cough) pirated content. They also have very good code and playbooks for ac
175.
by dannyw 2mo ago
Small models are still great for lots of “simple intelligence” use cases, like annotating or summarising files and media; or even just basic chat when given web search tools. My local NAS is private and I’m not going to send it off to APIs
176.
by dannyw 2mo ago
Nitpick: Gemma4-E4B is actually a 8 billion param model, but only 4.5B params worth of memory bandwidth needed per decode.
177.
by dannyw 2mo ago
But you could grab a 5090, and paired with some DRAM for MoE offloading of bigger models, and be a happy camper with 1.8TB/s of memory bandwidth. Or just use Luna honestly. Worth considering if you’re ok with hosted APIs.
178.
by dannyw 2mo ago
These kind of tricks will completely break API customers and be super visible, since most companies deploying API at scale have ample telemetry, evals, etc. Although, selectively applying it to consumer subs is probably beyond likely at thi
179.
by dannyw 2mo ago
Qwen thinking is really good in Mandarin; and probably natively trained the most there. Try a system prompt requiring it to think in Mandarin, while still delivering the response in the user’s language.
180.
by dannyw 2mo ago
You can partially tell by the tokeniser; which gives you some hint into the training corpus mix. </div> is four Gemma4 tokens, but one Qwen3.6 token.

More ›