5 ms·
It's Known as 'The List'–and It's a Secret File of AI Geniuses
- pretext 1y agohttps://archive.ph/Xp9cN https://archive.ph/Xp9cN
- amelius 1y ago[flagged]
- 3abiton 1y agoI'm curious about that syllabus.
- petesergeant 1y agoThis feels like one of those bell-curve memes with the idiot and savant on the opposite sides saying “building LLMs is magical work undertaken by wizards”, where in the middle someone is saying “it’s just high school math lol”
- _Algernon_ 1y agoThere is obviously a difference between understanding what smarter people have already done, and actually innovating and building on what they have done. The latter requires a much deeper understanding.
- roenxi 1y agoIt isn't actually an obvious limiting factor here. If I wanted to copy what, say, OpenAI has done the biggest stumbling block seems to be budget and GPU hours. It isn't obvious what the academic or engineering breakthroughs here really are. The maths does seem to be pretty trivial. I wouldn't want to say it is easy, but this doesn't seem to be the work of geniuses as much as the work of people who work at large companies with lots of hardware.
- _Algernon_ 1y agoThe breakthrough isn't the matrix multiplication you'd learn in high school math. The breakthroughs are the likes of transformers, deep learning, etc.. In short the stuff that happens in a research lab, not what is thrown at you with flashy confetti by sales people.
- roenxi 1y agoI've read the transformers paper, it is still pretty simple math. And the architecture isn't particularly interesting from a theoretical sense, the concepts are basic - we're not talking Gauss, Newton or Euler here, we're talking 2-10% of software engineers [0]. The limit is most software engineers don't have enough hardware to test whether their ideas work or not. And it is actually the same matrix multiplication people might learn at high school. The standard definition of matrix multiplication is generally accepted. [0] We'd expect the figuring-things-out process to be faster for more cluey people, so the people who actually pick all this stuff up first probably are geniuses. But a more normal engineer with the hardware access would still figure it out independently sooner or later. It is just too neatly on the beaten path where we'd expect to be finding optimisations.
- _Algernon_ 1y agoMy point is that a discovery always seems simple and inevitable once it has happened. Newton's laws are pretty simple, all things considered, but it still took most of human history for those to be discovered. An average high school student doing physics today could derive Newton's laws. That doesn't mean that that student put in the same situation as Newton, would have been capable of discovering newton's laws.
- clauderoux 1y agoGradient descent is to AI, what a loop is to understand programming. However, understanding a loop doesn't mean you can program a full video game from scratch. Organizing hundreds of layers in an efficient way is pretty complex, even if the work today has been simplified thanks to PyTorch or Tensorflow, it remains pretty complicated. You have to understand how to organize your data, how to size your batches, how to make your code resilient enough to survive GPU cards crashing. Train a model over hundreds of GPU is really really complicated. New algorithms are proposed all the time, because we have no idea how to handle these interconnected layers in an efficient way, but with cumbersome heuristics. However, salary inflation is never a good thing, because it will create a gap between decent engineers and other people pronounced geniuses. The AI teams will suffer from these decisions... Badly. It will be like these samouraïs who would kill peasants after a battle to increase their head count, because this was how people were rewarded after a battle. Some of these people, in order to justify their salaries, will feel pressure to poach other people's ideas...
- pinoy420 1y ago[dead]
- ribcage 1y ago[dead]
- trashchomper 1y agoIs it that secret? I imagine you could get 90% the way there within a day or two with just a combo of highly cited papers + Linkedin scraping
- sonorous_sub 1y agoK
- _tk_ 1y agoSo… where is the list?
- tveita 1y agoMight be worth legally changing your name to an entry on the list, just in case.
- bgwalter 1y ago"they all know each other" Stop right there. In this industry, a clique of people who all know each other attest each other superhuman abilities? Unheard of! Shouldn't "AI" write itself by now and make them redundant? This WSJ article is like a Cisco article in 1999. Beware of the crash.
- patrickhogan1 1y agoInteresting how DeepSeek didn’t need the list and just needed people to GSD instead of politics.