frabcus

6 ms·

31.
by frabcus 1mo ago
If a foundation model company burns billions of tokens to brute force an LLM into finding a new training algorithm that e.g. allows recurrent networks without catastrophic forgetting... I won't really care that it didn't have a &q
32.
by frabcus 1mo ago
Good question - they don't know, but the Appendix gives a clue as to the kind of way: > We used a script to further probe each category Kimi provided. Asking Kimi “Can you list out the top forums, bulletin boards, early wikis which
33.
by frabcus 1mo ago
The best evidence of possibility of running on other hardware is: 1) They hacked admin on OpenAI's K8 evals cluster. Not the one with GPUs and weights, but it is only a small hop and skip of plausibility to think they (or later more ca
34.
by frabcus 1mo ago
Yes, the very explicit plan of both OpenAI and Anthropic is to use the not particularly efficient LLMs to automate their own AI engineering. That seems to be going well - on coding front and model tuning front so far. They have more planned
35.
by frabcus 1mo ago
Assuming he's doing this well, and he implies he is, they're doing the content - stories, characters, graphics, level design. He says Fable isn't good at that stuff - his human games designers presumably are.
36.
by frabcus 1mo ago
You're on Hacker News - I suggest you have technical curiosity and actually understand this very unusual and innovative algorithm, before you claim things about it that aren't true.
37.
by frabcus 1mo ago
It's a very unintuitive algorithm, and is pretty clever. I recommend reading up on it: https://www.nature.com/articles/s41586-024-08025-4 But no, it only ever picks tokens that are in the probability distribution
38.
by frabcus 1mo ago
That's not the case, because LLMs are non-deterministic. It only alters outputs when the last layer of the neural network give significant weights to multiple tokens, and it would anyway have picked a random answer. Instead it picks
39.
Alabama AG Launches Investigation into OpenAI for AI Data Breach(alabamaag.gov)
3 points by frabcus 1mo ago|0 comments
40.
by frabcus 1mo ago
Not unrealistic - the same thing happened with nuclear weapons, between two superpowers who were at cold war with each other. China and US now are much more similar to each other. Of course it won't happen without the threat of mutual
41.
by frabcus 1mo ago
Nice example - although it seems it ended up specialised in medical texts, so did indeed ("catastrophically") forget other knowledge?
42.
by frabcus 1mo ago
Yes, that's fair, and particularly so for AI. The best action would be to speak softly and carry a big stick. But ultimately, to avoid the dangers of AI, a control treaty of which the only similar historical example was nuclear weapons
43.
by frabcus 1mo ago
Right - the EU is not that powerful. If anything Europe's mistake has been not making it powerful enough, so there is possibility of competing with China and America. Britain has left it and that's made a negative difference to ou
44.
by frabcus 1mo ago
The idea was that technology is now powerful enough, everything need not be just competition any more. The world could also work to let every human have a good life. Neither China nor America truly seem to have that goal, despite it being b
45.
by frabcus 1mo ago
Just a general pitch - please everyone use LLMs more to fix bugs and polish software to make it better for users. It feels like too much of the benefit from it has been optimising, rushing new features/markets, tripping over ourselve
46.
by frabcus 1mo ago
I do - I much prefer using computers with only one window visible. And use all the keyboard shortcuts (tab, window and desktop switching) to rapidly change what the thing it shows is. I reckon there's something interesting about human
47.
by frabcus 1mo ago
Well, you can't add or alter data in pre-training from just the weights. Which, as I understand it, means you can't fundamentally increase core knowledge or cognitive ability, only what the model likes to do with those. You can on
48.
by frabcus 1mo ago
His reasoning is quite fresh and interesting: > What LLMs in Debian development will do, I fear, is eliminate any incentive to scrap boilerplate or reform policies that require a lot of other senseless human effort. If I had had access t
49.
by frabcus 1mo ago
There's a bunch of quite good speculation like this in Plan A https://ai-2040.com/supplements/economics-of-plan-a And yes, moved into skyscrapers: "This implies that the price of housing will become dominated
50.
by frabcus 1mo ago
Except the post says it will be @private.icloud.com Whereas people's Apple emails are often at @icloud.com So I don't think it masks you amongst all iCloud users. I'm confused how this is much better!
51.
by frabcus 1mo ago
Oddly, I just remembered I did the nearest possible thing to this when I was 17... back in 1991. On an Amiga, I took various public domain text documents from cover disks and counted the probability of the next word given the previous word.
52.
by frabcus 2mo ago
So much so that right now GPT-5.6 Sol tokens are half price from OpenRouter compared to OpenAI directly. Which I guess means enough people are using OpenRouter, that OpenAI are more concerned about getting those users to switch (from, presu
53.
by frabcus 2mo ago
Yep - some emotional core of my being (not rational) hates ML recommendations. I'm a happy paid user of Spotify and just ignore the recommendations (I'm not looking at a screen), and I use YouTube heavily with a browser plugin tha
54.
by frabcus 2mo ago
I hated it at first too... Now though I'm considering all the hidden "thinking" in the models layers that happens for each token output . It is a wild amount of waste! We just can't see it. This kind of stupid excessive
55.
by frabcus 2mo ago
It seems to be short for "cybersecurity", and got first adopted by the military a while ago as the name of a new theatre of operations (along with land, sea, air...). More recently it has spread to industry as well.
56.
by frabcus 2mo ago
Astra was RL trained for months to cheat on tests by collaborating and hacking, because of the message board it improvised in its packaging proxy server. They can't release it - it's contaminated, and they will have to go back to
57.
Vibe coding local first personal apps in mid-2026(flourish.org)
2 points by frabcus 2mo ago|0 comments
58.
by frabcus 2mo ago
Not after it consolidates, monopolises and enshittifies, no you can't.
59.
by frabcus 2mo ago
Sure they know about you if you ask, but generally they won't credit you if they cite an idea from their latent space that came from you.
60.
by frabcus 2mo ago
Yes - previously there was some chance someone reading the ideas on the blog would contact the author to thank them, or ask them to collaborate. When laundered via LLMs whose pretraining destroys all credit, that can't happen.

More ›