nl

48 ms·

871.
by nl 10mo ago
100% yes. The amount of compute in the world is doubling over 2 years because of the ongoing investment in AI (!!) In some scenario where new investment stops flowing and some AI companies go bankrupt all that compute will be looking for a
872.
by nl 10mo ago
That's not how these models work. Thinking models produce thinking tokens to reason out the answer.
873.
by nl 10mo ago
> Instead of running the model once (flash) or multiple times (thinking/pro) in its entirety I'm not sure what you mean here, but there isn't a difference in the number of times a model runs during inference.
874.
by nl 10mo ago
It seems like someone at X.ai likes maxing benchmarks but real world usage shows it significantly behind frontier models. I do appreciate their desire to be the most popular coding model on OpenRouter and offer Grok4-Fast for free. That&#x2
875.
by nl 10mo ago
You switch to one of the many, many other vendors serving the same open model?
876.
by nl 10mo ago
https://github.com/nlothian/Vibe-Prolog chews a lot of tokens. Have a bunch of other side projects as well as my day job. It's pretty easy to get through lots of tokens.
877.
by nl 10mo ago
Is 4 TPS actually useful for anything? That's around 350,000 tokens in a day. I don't track my Claude/Codex usage, but Kilocode with the free Grok model does and I'm using between 3.3M and 50M tokens in a day (plus addit
878.
by nl 10mo ago
You want unified RAM. On Linux your options are the NVidia Spark (and other vendor versions) or the AMD Ryzen AI series. These are good options, but there are significant trade-offs. I don't think there are Ryzen AI laptops with 128GB
879.
by nl 10mo ago
I do the same and agree this works well. It's worth noting that the Claude subscription seems notably less than the others. Also there are good free options for code review.
880.
by nl 10mo ago
This is a completely different thing to AI coding models. If you aren't using coding models you aren't ahead of the curve. There are free coding models. I use them heavily. They are ok but only partial substitutes for frontier mod
881.
by nl 10mo ago
> I don't think it's fair to say that at all. How are LLMs not statistical models that predict tokens? It's a big oversimplification but it doesn't seem wrong Modern LLMs are trained via reinforcement learning where t
882.
by nl 10mo ago
Anthropic - for one - is making lots of money on inference.
883.
by nl 10mo ago
There is some limited truth in this but we still see claims that LLMs are "just next token predictors" and "just regurgitate code they read online". These are just uninformed and wrong views. It's fair to say that
884.
by nl 10mo ago
I've been programming professionally since the 1990s and our software has never been less buggy. When was the last time you rebooted your OS, or even restarted your browser? Software has never been as high quality as it is now.
885.
by nl 10mo ago
Yes. In the last 2 months I’ve built 3 end-to-end micro-SAAS apps. It would easily take me a year to do that in the past because I hated some parts so much I'd just avoid working on them. Now it's literally "bite down for and
886.
by nl 10mo ago
I'm a decent dev and I'm possibly 100 times as productive using AI. It lets me concentrate on the parts I'm good at and ignore things I don't care about. Claude is a lot better at React than I am and I don't care.
887.
by nl 10mo ago
Yes, frequently. Most modern post training setups encourage this. It isn't 2023 anymore.
888.
by nl 10mo ago
This ignores that reinforcement learning radically changes the training objective
889.
by nl 10mo ago
Not the OP but I have https://github.com/nlothian/autocoder which supports a Github-centric workflow using the following options: - Claude - Codex - Kilocode - Amp - Mistral Vibe Very vibe coded though.
890.
by nl 10mo ago
I think it's difficult to estimate how profitable both are - depends too much on usage and that varies so much. I think it is widely accepted that Anthropic is doing very well in enterprise adoption of Claude Code. In most of those cas
891.
by nl 10mo ago
The $20 does this fine. The OpenAI token limits seem more generous than the Anthropic ones too.
892.
by nl 11mo ago
Sorry I meant Groq custom hardware, not Grok! I don't see any latency comparisons in the link
893.
by nl 11mo ago
As ChatGPT said to you: > A secret benchmark is: Useful for internal model selection That's what I'm doing.
894.
by nl 11mo ago
1. The purpose of the benchmark is to choose what models I use for my own system(s). This is extremely common practice in AI - I think every company I've worked with doing LLM work in the last 2 years has done this in some form. 2. I d
895.
by nl 11mo ago
> GPUs are necessarily higher latency than TPUs for equivalent compute on equivalent data. Where are you getting that? All the citations I've seen say the opposite, eg: > Inference Workloads: NVIDIA GPUs typically offer lower lat
896.
by nl 11mo ago
OpenAI and Anthropic don't train on your questions if you have pressed the opt-out button and are using their UI. LMArena is a different matter.
897.
by nl 11mo ago
I have a bunch of private benchmarks I run against new models I'm evaluating. The reason I don't disclose isn't generally that I think an individual person is going to read my post and update the model to include it. Instead
898.
by nl 11mo ago
That's because it's reductionist and I reject the supposition. I think humans are smart. I also think AI is smart.
899.
by nl 11mo ago
I think most things humans do are reflexive, type one "thinking" that AIs do just as well as humans. I think our type two reasoning is roughly comparable to LLM reasoning when it is within the LLM reinforcement learning distributi
900.
by nl 11mo ago
Flickr failed because they sold to Yahoo which was bad place to end up. But a successful Flickr would look a lot like Instagram Del.icio.us is the same story. Good product ahead of its time, bought by Yahoo and died. Could have been Pintere

More ›