reissbaker

5 ms·

31.
by reissbaker 5mo ago
Most of these folks aren't running on spot/preemptible instances, they're doing 1-2 year reserved rentals. There isn't that much cheap compute floating around and you can be quite profitable on reservations (if you can g
32.
by reissbaker 5mo ago
LLM subscriptions are not "all you can eat," they have rate limits — and fundamentally there is no difference between subscription-with-rate-limit and typical usage-based business practices. Subscriptions are simply usage-based pr
33.
by reissbaker 5mo ago
> Sometimes it really is free though, because the hardware was bought to serve some other existing needs and that capital expense was fully depreciated quite some time ago. No one has 8xRTX Pro 6000s that have depreciated to zero "q
34.
by reissbaker 5mo ago
The local rig is not free and requires very large capital expenditures while producing very low token throughput for large models. Within any time budget, you can get many orders of magnitude more large-model tokens off an 8xB200 than off
35.
by reissbaker 5mo ago
There already are many subscriptions for LLM tokens: OpenAI, Claude, Synthetic (shameless plug), Zai... I'm not sure what you mean by "There will not ever be a monthly subscription for LLM tokens." That already exists!
36.
by reissbaker 5mo ago
Users do not have an existing $80k of hardware, are not going to buy $80k of hardware for worse performance than paying $100/month, and models are continuing to grow in size while memory grows in price.
37.
by reissbaker 5mo ago
RTX 6000 Pro retails for $10k so an 8x is $80k before anything else in the computer, and long-context will have... pretty bad performance (20+ seconds of waiting before any tokens come out), but it's true it technically works. I don&#x
38.
by reissbaker 5mo ago
Probably an experiment due to Bun's PRs to Zig being rejected (Zig does not allow AI use). If Rust works well enough, and the alternative is maintaining a fork of Zig, I'd guess they'd go with Rust.
39.
by reissbaker 5mo ago
For the Claude Code / OpenCode / Crush / etc new wave TUIs, it's not about composability or text streaming. It's basically a combination of a few tailwinds: 1. There's already a large-ish community of engineers
40.
by reissbaker 6mo ago
Caching is pretty simple. If it's a prefix match, it's cacheable. Very long context windows will be much more expensive than shorter ones, even with caching, assuming you're using Claude Code or some similar harness for both.
41.
by reissbaker 6mo ago
Wow, I'm surprised. Grok 4.3 actually is noticeably better than the other two for the close-friend variant. Surprisingly I found Claude the cringiest of the three!
42.
by reissbaker 6mo ago
The U.S. restrictions on Nvidia compute were clear and extremely widely reported on for years prior to Supermicro's sales. That's why Nvidia created special Chinese-market versions of their GPUs, e.g. the H800 and H20, which Sup
43.
by reissbaker 6mo ago
Good models vs bad models are relative: if this was released in 2020 it would be earth shattering. But releasing a model today that's only on par with open-source dense models a quarter of the size and soundly beaten by open-source MoE
44.
by reissbaker 6mo ago
Naive or not, plenty of investors believed that running a company out of Singapore would shield them from the aggressively un-free Chinese market controls. Manus is proving them wrong, which will hurt Chinese companies that try to run out o
45.
by reissbaker 6mo ago
What are you talking about? Here are the concrete differences: 1. The U.S. has had a long-standing, extremely public policy that you Cannot Sell Nvidia Chips to China since 2022. Supermicro is an American company (based in San Jose, Califor
46.
by reissbaker 6mo ago
You've been all over this this thread responding with the same whataboutist comments claiming America does the same thing. And yet, I'm pretty sure America hasn't held American citizens hostage in order to force them to unwin
47.
by reissbaker 6mo ago
I think it's precisely the fact that the founders live in China. The CCP can make them... kick rocks... for the rest of their lives. Generally speaking this seems bad for Chinese companies, though. They were able to raise capital from
48.
by reissbaker 6mo ago
We do both: managed Kubernetes when it's available (AWS, Nebius, others), but for some hardware vendors they just give us raw machines and we self-host K3s on their nodes. We're an open-source LLM inference company so we're b
49.
by reissbaker 6mo ago
K8s isn't even hard! My team of three manages everything on K8s and we spend ~0 minutes per week on it. Write a script to generate some YAML files, stick it in a CI pipeline, and it's basically fire-and-forget. You're going t
50.
by reissbaker 6mo ago
Dense is (much) worse in terms of training budget. At inference time, dense is somewhat more intelligent per bit of VRAM, but much slower, so for a given compute budget it's still usually worse in terms of intelligence-per-dollar even
51.
by reissbaker 6mo ago
Muse Spark doesn't even match GLM-5.1 on most benchmarks. And GLM is open source!
52.
by reissbaker 6mo ago
Well, I see you've graduated from wishcasting the Iron Dome being "blinded" by a radar it doesn't use to being confused that shooting down missiles involves AI.
53.
by reissbaker 6mo ago
None of your links support that claim or even try to make it. The Haaretz article is complaining about a day of unusually short missile notifications on March 7 , a week later than the Iranian strike on the radar (and now a month-old claim
54.
by reissbaker 6mo ago
The problem is you do not understand how these systems work and are making claims that don't pass the sniff test to anyone who does know how these work. For example, you claim multiple times that Shahed drones have somehow exacerbated
55.
by reissbaker 6mo ago
Both you and the Guardian are confused (or perhaps the Guardian is just trying to ride the popular understanding of the "Iron Dome" as a super catch all missile defense system vs reality). The Iron Dome has nothing to do with shoo
56.
by reissbaker 7mo ago
FWIW, Pierre's "Code Storage" project [1] seems like it simplifies a lot of the operational overhead of running git servers, if what you want is "an API for git push". Not affiliated with the company (and I haven&#x
57.
by reissbaker 7mo ago
I run a small open source LLM inference company, Synthetic.new. As far as I can tell, CNBC isn't reporting this accurately: the problem isn't that Oracle is building "yesterday's data centers": they're building
58.
by reissbaker 7mo ago
How does it compare at vision tasks to Kimi K2.5?
59.
by reissbaker 7mo ago
No, it attacked Azerbaijan's airport and Saudi oil refineries, among many other civilian targets. They're just hitting whatever they can.
60.
by reissbaker 8mo ago
Wait, why would you still need a home server if the harness (aka, the agent) is hosted in the cloud?

More ›