eli

5 ms·

31.
by eli 1mo ago
I have been working on a personal benchmark suite to test new models and ironically one thing all the models are bad at is writing new benchmark tasks. I guess it’s the different layers of abstraction between the task and how it’s evaluated
32.
by eli 1mo ago
But it's an error, not a response.
33.
by eli 2mo ago
https://www.liberalcurrents.com/the-reconstruction-papers/ is explicitly aiming to be a Project 2028 book for the left.
34.
by eli 2mo ago
I think they run whatever models they get paid to run. But mostly from enterprise. They are clearly not interested in consumer dollars.
35.
by eli 2mo ago
Yes - unusually (uniquely?) bad for an official API. Which is a real bummer because it’s otherwise solid with excellent caching.
36.
by eli 2mo ago
The philosophy with Pi is it is minimal (but functional) out of the box and easily extensible. I'm not familiar with that feature but I would not at all be surprised if someone already coded a Pi extension that does it.
37.
by eli 2mo ago
They did release the weights and there are many providers https://openrouter.ai/deepseek/deepseek-v4-flash-0731#provid... Deepseek's official API has a pretty bad privacy policy so I would assume businesses avoid
38.
by eli 2mo ago
Isn't it a good thing to know about price hikes in advance? If I were building a product around it, I would certainly care.
39.
by eli 2mo ago
Anthropic the other day announced future models "launched in the EU" will include watermarking https://support.claude.com/en/articles/16266773-how-claude-m... I was assuming it was something like SynthID
40.
by eli 2mo ago
Luna has one vendor and they can change the price any time. Deepseek is open and has dozens of vendors competing to serve it.
41.
by eli 2mo ago
I wondered if the promised change in pricing is actually going to be deepseek bringing up their cached costs. They're extremely inexpensive.
42.
by eli 2mo ago
The Deepseek official API is good with excellent caching. But their privacy policy is unusually bad - they can train off your prompts and completions.
43.
by eli 2mo ago
Opus 5 medium to Opus 5 max is only 3 points, if that puts it in context
44.
by eli 2mo ago
Official pricing only kinda matters for an open weight model, no?
45.
by eli 2mo ago
Fable's guardrails would never let it do something like Cybergym so at least for that one it's measuring Opus 5
46.
by eli 2mo ago
The top 4 models on that site are all variants of Gemini Flash? That does not match my experience at all.
47.
by eli 2mo ago
I've seen ones that are configurable to pick a trade off point between lower cost (cache stickiness) and routing performance (best model for that turn). But yeah I'm skeptical all this overhead is worth it.
48.
by eli 2mo ago
It’s injected into an outbound api call, not into an env var the agent can read.
49.
by eli 2mo ago
You can run the agent in the gondolin sandbox if you wish. Their example implementation with pi uses a pi extension so that pi runs on the host but the read/write/bash/etc tools run in the guest. Doesn’t have to be that way t
50.
by eli 2mo ago
That incident was with a model that had the guardrails disabled. Still obviously you should run all untrusted code in a sandbox, but extreme actions like that would be very unusual with the model that shipped.
51.
by eli 2mo ago
I mean…yeah…it’s a bug.
52.
by eli 2mo ago
Qwen 3.8 Max is very strong at troubleshooting and code review.
53.
by eli 2mo ago
I assume/hope this is about prices going up for the next release of Pro
54.
by eli 2mo ago
And all that just to allow internet access for npm and pypi? If you've got the bandwidth and disk space, it's very easy to make an offline mirror of both.
55.
by eli 2mo ago
That's got a significant selection bias. Claude and ChatGPT and Gemini and other subs do not go through openrouter.
56.
by eli 2mo ago
I use https://pi.dev/ which works fine out of the box but is fairly minimal and intended to be customized. There are many extensions. OpenCode or oh-my-pi might make more sense if you just want a batteries-included agent. Y
57.
by eli 2mo ago
It's not exactly what you asked for, but the public transit and multimodal focused apps are generally much better at this stuff. I like CityMapper and Transit. Neither is good for turn-by-turn directions, but both are good for "wh
58.
by eli 2mo ago
It's not enough that it's better? Many providers will host it and will compete on price. It also can't easily be taken away because one company (or one government) decides they don't want it around any more. People can f
59.
by eli 2mo ago
I believe it. It's extremely good at troubleshooting. I gave Qwen and Kimi K3 the same annoying, complicated, intermittent bug to track down. Kimi did a bit better in understanding the existing code, but Qwen built some diagnostic tool
60.
by eli 2mo ago
The amount of abuse of free VMs must be absolutely immense even with credit card verification. I’m not sure there needs to be additional secret motives.

More ›