sho

12 ms·

31.
by sho 3mo ago
You can! It just might be a little bit outside your budget.
32.
by sho 3mo ago
> When the free money party stops The Openrouter providers the GP referenced were never at the "free money party". The actual cost of running something like GLM5.2 is well understood and tokens from those providers are not sold
33.
by sho 3mo ago
> It's always integers unless you have a VERY good reason to do otherwise I don't really agree. This seems like one of those outdated "greybeard rules" which people love to cargo cult, but to me it just comes with it
34.
by sho 4mo ago
Well, I guess this is the silver lining to the price increases. I'd been thinking about an M5 128GB for local inference (eg DS4), probably off the table now given that it jumped $2k overnight. But I was on the fence about it for a long
35.
by sho 4mo ago
I'm going to leave my above comment for embarrassment/posterity, but since writing it I've driven GLM5.2 much more extensively and I take it back. 5.2 is surprisingly, even shockingly good. It seems MUCH better than when I tr
36.
by sho 4mo ago
You may be right, and I certainly hope so! But the question was about whether the Chinese labs will have fable-equivalence in 1 year. I am by no means some kind of insider, but knowing the vaguest outlines of what went into Mythos, they jus
37.
by sho 4mo ago
I don't think we will. The open model labs are too resource constrained to approach Fable or even Opus on the general case and I don't see that changing within a year. Right now, due to profound shortfalls in both data and hardwar
38.
by sho 4mo ago
> the “Peekaboo World” What a great analogy. And IG/Tiktok reduce it into an even purer state - endless random videos, barely if at all connected, ephemeral stimulation you can't even remember 30 seconds after seeing it. I know
39.
by sho 4mo ago
I think when you follow this stuff every day it's easy to lose perspective of the rate of change and these leads seem more profound than they really are when you zoom out a bit. I'm no super-insider, I only hear industry scuttlebu
40.
by sho 4mo ago
Ed Zitron certainly was right that a constant firehose of denialist AI doom would get him clicks and views from the type of audience who yearns to have their biases confirmed and their fears validated. He's made a lot of hay off that e
41.
by sho 4mo ago
it's down 99% since that peak. But let's compare to it anyway. It's pretty useless to compare raw FLOPS, but as a general hand-waving guesstimate, F@H is currently doing about 25 petaflops in a mix of FP16 and 32. AI usually
42.
by sho 4mo ago
> AI hardware is for inference, not training Not sure what you are referring to, unless you don't think h100/h200/b200 are "AI hardware" > Superpods aren't really power efficient Maybe not compared to a s
43.
by sho 4mo ago
As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better con
44.
by sho 4mo ago
If folding@home is a useful yardstick by which we might estimate the amount of GPU-ish capability that civilians might be coaxed into donating to a shared enterprise, yeah, it doesn't look pretty. This is extremely rough napkin math bu
45.
by sho 4mo ago
I don't think insulting people is a great way to contribute. Not everyone who sees things differently than you has "psychosis". Your reflexively negative comments on anything relating to AI are as insight-free as they are num
46.
by sho 4mo ago
An enduring, confounding quality of LLMs is that even minor differences in prompting content and style, harness type and environment can lead to radical differences in the output and perceived performance and ability. In my environment and
47.
by sho 5mo ago
Same story with me. To be clear, I am a subscriber, though I tend to hold out for the ultra-cheap last ditch retention deals they through at you. But I take them with a grain of salt these days. They have a narrative like anywhere else, and
48.
by sho 5mo ago
I 100% agree with you, but I've been convinced over the last year that it's a time and scale issue, not anything fundamental. The Chinese models right now are in a weird spot. Compared to the frontiers, both their pre and post tra
49.
by sho 5mo ago
> stop bleeding on fixed cost subscription plans What bleeding? Anthropic wants as much of that "bleeding" as possible. The interaction data gathered from genuine human CC subscription usage of their models goes directly into
50.
by sho 5mo ago
I am no-where near as concerned by this as I was a year ago, when I was expecting the axe to fall at any moment before the Chinese labs achieved some sort of escape velocity. I now think it's too late, all the cats are out of all the b
51.
by sho 5mo ago
The final sentence says it all: > The thing is that even if I was wrong (I'm not) and AI was somehow helpful for software engineering (it isn't), I still wouldn't want to use it. So even if you were wrong on the facts (you
52.
by sho 5mo ago
That's actually the approach we took with https://gentility.ai/ - we either provide almost-raw SQL query access to the DBs themselves or we synthesize from API into DuckDB via parquet and make that available to the a
53.
by sho 5mo ago
The problem description is spot on, but the solution isn't. No-one is going to sit in that chat and "collaborate" on each other's stuff in real time all day. You may as well just all sit around a screen. I welcome the ex
54.
by sho 6mo ago
So, this is the version that's able to serve inference from Huawei chips, although it was still trained on nVidia. So unless I'm very much mistaken this is the biggest and best model yet served on (sort of) readily-available chine
55.
by sho 6mo ago
I use and love Hetzner as well. But you have to go into it eyes open. Concrete example - a month ago I was notified that network infra my systems ran over was going down for maintenance. Public link: https://status.hetzner.com&#x
56.
by sho 6mo ago
You're assuming errors are a clear failure that can be identified and retried, rather than a silent drift from user intent that simply feeds bad but well-formed results into the next step. They're not. Well, of course sometimes th
57.
by sho 6mo ago
Taking the article's 5% accuracy improvement at face value: if true, then it's more than worth the token inflation IMO. That's because of tool call chains, where errors compound and accumulate, and small improvements in accur
58.
by sho 6mo ago
three, now!
59.
by sho 7mo ago
Nice, i've been waiting for this capability to show up. I've added support to my side project llmsg.com, here's a video of it in action https://x.com/sho/status/2034898928618152412
60.
by sho 7mo ago
I'm not some kind of OpenAI or Pentagon fanboy, but it's pretty easy to for me to understand why a buyer of a critical technology wants to be free to use it however they want, within the law, and not subject to veto from another e

More ›