martinald

7 ms·

31.
by martinald 2mo ago
Keep in mind all this kind of stuff can make the model less capable. If it has to think in "plain" English, it may well be squashing quality of code etc output. I'm not sure how true this is, but when using "forced"
32.
by martinald 2mo ago
Why would it save the stock market? Cheaper models if anything transfers more value to hardware companies and datacentre companies. The two companies that would be most affected are OpenAI and Anthropic, which aren't public.
33.
by martinald 2mo ago
Yeah I've been sorting of amazed how polished Grok build is. It's also super fast (written in Rust).
34.
by martinald 2mo ago
There's also a bit of selection bias going on here because we forget about labs that don't have a jump and just focus on the ones that do. Notably Google is definitely not having that capability jump.
35.
Watch out for cache read costs(martinalderson.com)
4 points by martinald 2mo ago|0 comments
36.
by martinald 2mo ago
Also looks incredibly fast. 150tps on openrouter (nearly all deepseek providers are around the 50tps mark).
37.
by martinald 2mo ago
It depends. If you are doing blog content with the idea of upselling users to your product, probably not as much (because the LLM can just give the user the answer). However, if you are looking for the best product/service/whateve
38.
by martinald 2mo ago
That's not _entirely_ true. Search Console now has a Generative AI page where you can see impressions per page now. Bing has something similar. Also, I would assume that really "GEO" is just like "SEO". If you rank
39.
I'm (mostly) picking models on speed now, not intelligence(martinalderson.com)
1 points by martinald 2mo ago|0 comments
40.
by martinald 2mo ago
It's interesting to me when I go to the cinema how poor the quality is compared to at home on an OLED TV. The blacks are so grey and dark stuff is hard to make out.The resolution is far less noticeable than that (but obviously worse).
41.
by martinald 2mo ago
now failing for me, esp auto mode classification
42.
The first known runaway AI agent – or a bad marketing stunt?(martinalderson.com)
3 points by martinald 3mo ago|1 comments
43.
by martinald 3mo ago
I don't think so. According to some very basic research there are around 8bn searches a day, or 250bn a month. Let's assume Google serves AI overviews on every SERP (they don't) and don't cache them (they do, afiak). And
44.
by martinald 3mo ago
If it is marketing it's the most silly marketing of all time. They are under extreme pressure from the US Govt to prove safety and saying "our model escaped" is not ideal. Perhaps there is some 4D chess going on to get open w
45.
by martinald 3mo ago
Yes agreed - I wrote this up a while back https://martinalderson.com/posts/whats-going-on-with-gemini/ My view then was they are optimising the models for inference ability on their own hardware AND use cases, whi
46.
by martinald 3mo ago
Will be interesting to see how it stacks up pricing wise on the various inference providers.
47.
Winners and losers in the coming AI margin collapse(martinalderson.com)
2 points by martinald 3mo ago|0 comments
48.
by martinald 3mo ago
GPUs are even more extreme. A 5060 is something like 15,000x faster than a 3dfx Voodoo card from ~2000 by my limited research.
49.
GLM 5.2 and the coming AI margin collapse(martinalderson.com)
694 points by martinald 3mo ago|469 comments
50.
by martinald 3mo ago
MCP makes a lot, lot more sense when you think of it as as a auth standard and not a comparison with CLIs. It obviously does more than just auth, but having standardised auth (which CLIs definitely do not) is the real 'killer' fea
51.
by martinald 3mo ago
I don't think that's inevitable with RL. Imagine in C# you are training the model with RL loops in a harness. One uses C#12 and one uses C#15 (when released), with union types (and importantly - includes the release notes in the h
52.
by martinald 3mo ago
Micron said that they tried to tell 2 of their largest customers (one almost certainly Apple) that the prices they were demanding would result in the cancellation of a lot new construction in 2023, which wasn't in the industries best i
53.
by martinald 3mo ago
Really sad. I grew up reading his writing. I emailed him some thoughts on one of his blog and he immediately replied in a lovely way very recently. What a shock and a loss.
54.
by martinald 4mo ago
Why not? It's the first time many developing countries have had access to high quality internet at an often relatively affordable price?
55.
by martinald 4mo ago
Well, the EU insists that track & train operations are separate. (ironically the UK _is_ combining passenger operations and track somewhat back together, which is only possible because of brexit). The bigger issue tbh is the enormous co
56.
Expert-aware quantisation: near-Q4 quality at near-Q2 size?(martinalderson.com)
3 points by martinald 4mo ago|0 comments
57.
by martinald 4mo ago
Wrote this a while back. https://martinalderson.com/posts/no-it-doesnt-cost-anthropic... OpenRouter is the best guide to real costs.
58.
by martinald 4mo ago
Yes agreed, for example, there was an interesting table on the starlink page I used to check every so often showing which countries had access to starlink as it was rolled out. Was interesting to see the expansion. Of course, some editor de
59.
by martinald 4mo ago
The point is that tok/s/GPU stays ~roughly stable. So you need say 4 GB200s minimum to fit the modules, but this provides 4x the tok/s as 1 GPU.
60.
by martinald 4mo ago
Yes 32B dense is a weird one to choose. But in reality, 32B dense is very similar* to 32B activated on MoE in terms of inference costs. And I highly suspect eg Opus is around that level of active params. A 284ba13b model at scale, is almost

More ›