10 ms·
What does Alan Kay think about LLMs?
- skadamat 3y agoThis answer is amazing and classically Alan Kay! There's so much here to unpack because of all the different areas Alan draws from in his work (he's like a computing philosopher). All I will say is that for people who want to understand his perspective, there's a large epistemological load to overcome. Sampling his talks is a good starting point though: https://tinlizzie.org/IA/index.php/Talks_by_Alan_Kay https://tinlizzie.org/IA/index.php/Talks_by_Alan_Kay
- agomez314 3y agowow this list is super useful, thanks for sharing!
- layer8 3y agoHe always criticizes how everyone got computing wrong (fair enough), but never offers anything but the most vague suggestions of how to do things differently.
- detourdog 3y agoI think trustability is pretty specific and without a foundation of trust the rest of the effort”s value is unknowable.
- layer8 3y agoThat’s “what we would like to have” (so mostly just a criticism that we don’t have it), but without much of a suggestion of how to actually achieve it in practice.
- detourdog 3y agoThe current industry leaders of the effort want to proclaim that it is impossible to know where the answers come from. If that is the industry norm then I understand how achieving trustability could be mysterious. My line of thinking is that without an audit trail of thoughts there won't be any trustability. I'm unable to describe in detail what any one specific thing leads to trustability. I can say that being able to demonstrate where answers come from would work for me.
- theGnuMe 3y agoTrue message passing is one of the key ideas behind smalltalk which would improve computer security considerably. This is analogous to the design of the internet that he references. The internet scales well. So imagine building entire computing systems this way.
- vajrabum 3y agoIf smalltalk is late binding and message passing, I think it's interesting that modern large scale webapps (say Netflix or Google) do just that just not inside process and server boundaries. I'd guess that Alan Kay would think that's a bastardization, and there are loads of complications but it's a definite improvement over the large php ball of mud monolith you might have seen at say Box or Facebook 10-15 years ago. Because the 'object' boundaries are so coarse you can also better leverage the few people inside an org who know how to design, scale, improve robustness and secure systems. If architecture is the metaphor, it's still pretty much piling mud bricks but at least there are corbeled arches, streets, city walls, public buildings and a rudimentary notion of style. As for LLMs I think that long form typeahead is the right way to think about it.
- fzzzy 3y agoI don't think he would think it is a bastardization. HTTP is 100% message passing. You have an object reference (url) and you pass a message to it with POST. State and behavior is encapsulated. In fact, one of the desireable properties of message passing is that it is easy to distribute across process and machine boundaries.
- layer8 3y agoHow would it improve it, though? Dynamically-typed languages tend to introduce gradual typing after a while to increase reliability and predictability, and the end game of that (static typing) is the antithesis of late binding. Message passing only improves security by adding a validation layer, but managed languages (JVM, CLR, etc.) effectively do the same. > The internet scales well. So imagine building entire computing systems this way. That’s very similar to microservices, and look how messy these tend to get. The internet scales well because it’s a network with flexible capacity and attached processing power, and because its clients are either human users who can deal with failure modes, or software with stable and well-specified point-to-point protocols.
- sebastianconcpt 3y agoHe did tho. He mentioned more than once that OOP was more about message passing than behavior + state.
- layer8 3y agoYes, that’s what I mean by vague suggestions. How do you actually build and maintain a system of any complexity with that? How do you ensure it will do what it is supposed to do? And Smalltalk wasn’t that, it is mostly just regular method calls and not “message passing”.
- Xelynega 3y agoErlang(and more modernly elixir/Gleam) are an example of this in practice, and were created to make it easier to ensure the working and manage the complexity of large systems.
- layer8 3y agoFair enough. But Alan Kay doesn’t say “use Erlang”, nor does he go into any depths about what the desirable feature set is and what would make it work or would fail to make it work. He’s always just waxing around nebulous desiderata and complaining about the status quo.
- MisterTea 3y agoMaybe because there is no one/right/simple answer. You really have to think for yourself and seek out answers instead of expecting them. You're doing that now but with the mistaken impression you were to be handed them.
- asdajksah2123 3y agoIf he said "use Erlang" then that eliminates the actual message he's trying to convey, which is message passing is good. I bet if he simply said "use Erlang", 99% of headlines and discussion would be "Alan Kay said Erlang is the greatest language evah!" I do appreciate that he respects his audience, ie me, enough that he thinks we can read "message passing is good" and go from there to choosing a message passing language that is suitable for our needs, or even using a non message passing language due to other factors, but recognizing that OOP is about message passing and not state encapsulation, which would impact how I code even in Java.
- mempko 3y agoWhat? He literally built systems that did things differently. Look at smalltalk.
- agumonkey 3y agoI think I get what parent means, Kay is mostly seen in conference mode, with simplified slides and big ideas, it's frustrating if you never saw who he got taught or worked with.
- carapace 3y agoYou are impressively wrong. He just wrapped up an incredible research program that demonstrated many concrete implementations of great computing ideas. > After 16 years of continuous research and important contributions toward its mission - "Improve 'powerful ideas education' for the world's children and to advance the state of systems research and personal computing" - Viewpoints Research Institute concluded its operations at the beginning of 2018. http://www.vpri.org/ http://www.vpri.org/
- layer8 3y agoAnd what exactly is the output, other than talks and some programming learning tools for children? I haven't seen any significant new insights or advances.
- carapace 3y ago> And what exactly is the output, other than talks and some programming learning tools for children? The papers and demos are the output. They were a research outfit, not a startup. OP claimed that Kay "never offers anything but the most vague suggestions" when he (et. al.) provided several very concrete working demos. It's hardly his fault if no one took him up on them, is it? (FWIW I suspect some of their work had an effect on MS Word & Excel UI but I don't know for sure.) > I haven't seen any significant new insights or advances. Where have you looked? Did you read the papers that VPRI published? OMeta has been mentioned else-thread, I like that Nile programming language, the COLA system seems neat.
- joshmarinacci 3y agoHis entire 60 year career is a suggestion of how to do things differently. ;) https://en.wikipedia.org/wiki/Alan_Kay https://en.wikipedia.org/wiki/Alan_Kay
- vinnyvichy 3y agoEventually, after distancing themselves from their curmudgeonly creator, his ideas land in the mainstream. I'm convinced, eg, that what's loved about python has been elucidated by AK at some point, but maybe it's even good marketing not to attribute anything to him ;)
- agumonkey 3y agoone real experiment he contributed to was VPRI ometa and the os built on top of that (every part of the OS built on top of an eDSL)
- fmap 3y agoOthers have already mentioned this, but Alan Kay spent his entire career experimenting with approaches to radically simplify computing. The progress reports from vpri give a good overview of a recent project that has since wrapped up, e.g.: https://tinlizzie.org/VPRIPapers/tr2012001_steps.pdf https://tinlizzie.org/VPRIPapers/tr2012001_steps.pdf
- YeGoblynQueenne 3y ago>> Others have already mentioned this, but Alan Kay spent his entire career experimenting with approaches to radically simplify computing. Yes. And yet, he created OOP [1]. Strange. ________________ [1] Not on his own.
- jdlshore 3y agoYou’re saying this like it’s a “gotcha,” which makes me believe you’re confusing OOP-as-practiced-in-enterprise with OOP-as-invented-by-Kay. The sad truth is that most great ideas are doomed to drown in a sea of mediocrity and misinterpretation.
- YeGoblynQueenne 3y agoI'm not confusing anything. OOP is a horribly OOT over-engineering of a problem looking for a solution, from the get go. That it morphed and twisted into a bloated mess when it was taken up by people who didn't get it to begin with is just the inevitable result of starting with a big pile of no need to do all that in the first place.
- Phiwise_ 3y ago[Citation Needed]
- Osmose 3y agoHe has generally great insights about principles to aim for, which is part of the answer, but no, he does not have the entire answer. His insights are inspiration for others who contribute their own parts to potential answers—that's how open collaboration works.
- deleted 3y ago[deleted]
- KorematsuFredt 3y agoI wish someone could provide a tl;dr of that answer to us simpletons. His writing style is something I am unable to comprehend.
- dpflan 3y agoThere is a lot in here, various paths to venture off, but the bottom line seems to be trust is important when running commands on a machine, and LLMs are not trustable. What else?
- altruios 3y agowhat would be required for trusting an LLM? 1: 100% transparency. Open Source code, fully (and correctly) attributed training data. 2: A predictable model of what these models are actually encoding (so that hypothetical new models (or modifications) can be reasoned about).
- thesz 3y agoI think I have to know what chain of reasoning is behind this or that fact and/or deduction. i would like to be able to verify that. For example, the proof of the absence of solution in SAT should be accompanied with the easily verifiable chain of reasoning. This shows the absence of incorrect deductions and missing assignments. Another example is autovectorization in contemporary compilers, they can show you why parts of your loops are not eligible for vectoriztion. All LM's can do is to show me that these parts of those inputs are important for that output, but nothing else. Thus, they cannot be trusted even for minimally critical tasks.
- pixl97 3y agoEven you can't do that yourself. You can only mad post ad-hoc justifications for the choices you made.
- thesz 3y agoWhat cannot I do myself? Look at the output of gcc -ftree-vectorizer-verbose=2? Run the verifier after picosat? I did both.
- thesz 3y agoTo illustrate: https://arxiv.org/abs/2401.05566 https://arxiv.org/abs/2401.05566 One can teach LLM to write good code now and bad code in future (when everyone lower their guards). And no one can prove who and how made LLM to do that. Also, no one can prove formally the absence of such a plant.
- sebastianconcpt 3y agoQuestioning trustfulness to ontological hallucinatory automatons. Nailed it!
- sebastianconcpt 3y agoIt is by definition that the silicon automatons cannot escape from their ontologically stochastic imitative hallucinations. All they can do is be observed by us and with that expose us to get induced by association, ontologically human hallucinations and deal with its outcome of real consequences.
- devjab 3y agoI use LLMs quite a lot to help me in my work, but they are wrong so often that it’s ridiculous. This isn’t a major issue when you’re an expert using the tools to be more efficient, because you’ll spot and laugh at the errors it makes. Sometimes it’ll be things that anyone would notice, like how a LLM will simply “invent” a library function that has never existed. Even if you’re not an expert, you’re not going to get that PNP-whatever function to work in Powershell if it never existed in the module to begin with. Where it becomes more dangerous, at least in my opinion, is when the LLM only gets it sort of wrong. Maybe the answer it gives you is old, maybe it’s inefficient, maybe it’s insecure or a range of other things, and if you’re new to programming, you’re probably not going to notice. Hell, I’ve reviewed code from senior programmers that pulled in deprecated things with massive security vulnerabilities and never noticed because they were too focused on fast delivery and “it worked”. I can’t imagine how that would work out for people trying to actually learn things. I’m not sure what we can really do about it though. I work a side gig as an external examiner for CS students. A lot of the curriculum being taught (at least here in Denmark) are things I’ve seen the industry move “beyond” in the previous 20 years. Some of it is so dated that it really makes no sense at all. Which isn’t exactly a great alternative to the LLMs, and it’s only natural that a lot of people simply turn to these powerful tools. I tend to tell people to ask their favorite LLM to help them solve a crossword. When you ask it to give you words ending on “ing” it’ll give you words that don’t end on “ing” because of how the tokens used work. This tends to be an eye opener for people in regards to how much they trust their LLM. At least until they get refined enough that they can also do these things. Anyway, it’s a good answer.
- bongodongobob 3y agoI'm sorry, but it sounds like you're using ChatGPT3.5. I get very very few invented libraries unless it's a very specific version of an old or rare language. I tried your "ING" example and GPT4 got 100/100. Idk, ymmv, but GPT4 beats the pants off of 3.5.
- samatman 3y agoI use GPT4 and it confabulates constantly. Try it on a language in the 10th-20th range of popularity (let alone lower) and I expect you'll see it as well. I don't see that as often with Python, the only top-5 language I currently work with.
- dimal 3y agoOff topic, but I didn't realize Alan Kay was regularly answering questions on Quora. I can ask "What does Alan Kay think about X?" and get an answer from Alan Kay?!?
- YeGoblynQueenne 3y agoWell, you can try. And if he doesn't answer you can always ask an LLM :|
- stana 3y agoCan we trust it though that is what Alan Kay would say?
- swyx 3y agowhen you're alan kay you can literally just title things in the third person and it'd get read haha
- mempko 3y agoI see a lot of people here are missing his "big deal" which he talks about in the end where he references the "Spaceship Earth" problem. What I believe he is getting at is people are going to use LLMs to build systems at scale to further strip mine society. The "Spaceship Earth" problem is a reference to Limits to Growth. For those who haven't read "Limits to Growth", and the more recent Re-calibration of Limits to Growth, I implore you to do so. https://onlinelibrary.wiley.com/doi/full/10.1111/jiec.13442 https://onlinelibrary.wiley.com/doi/full/10.1111/jiec.13442
- lucianbr 3y agoMost tech seems to be used, at least amongst other things, to further strip mine society. It's the extreme of wealth accumulation, and there's always some people wanting to do that. Microsoft uses the idea of personal computers to get a monopoly on OSes, and get as much rent with that. Google and Amazon use the internet. Apple the iPhone with the walled garden. OPEC uses oil to extract as much wealth as possible. Toyota and VW would also get a monopoly and then raise prices it if they could. Why do you think VW bought so many other car makers? Boeing now that it is too big to fail, focuses on making more money, and less safety and excellence in engineering. Everyone does it. Well, every big business. It's a bad thing for sure, but doesn't seem specific to LLMs, and a solution would not be specific either.
- mempko 3y agoIt's not specific to LLMs, but the point is LLMs will make things more efficient and therefore make 'a bigger straw' to drink the milkshake.
- titzer 3y agoLLMs seem like a new and dangerous tool to scale the bullshit-generation machine to unimaginable levels. Pretty soon it will be impossible to trust any digital information, as every image, recording, video, document, etc can be instantly altered, adjusted, or completely fabricated by ungodly powerful generative AIs.
- ozten 3y ago> That humans also do this all the time is “interesting”, “dangerous” etc., but it is also why trying to move from superstition (this is actually what “reasoning by correlation” amounts to) to more scientific methods is critical for anything like civilization to be created. "reasoning by correlation" as superstition is a brutal insight.
- kurthr 3y agoIt seems like the other part of this is the ability of LLMs to rationalize even answers that are not rational (he says piling BS on BS). That has previously been a very human (and quite distracting/confusing) activity and it's very effective on most people most of the time.
- kergonath 3y ago> "reasoning by correlation" as superstition is a brutal insight. I don’t think he is right on that one, though. Reasoning by correlation is a kind of empiricism, and it can be tested. “These things happen together” or “if I do this, then that happens” can be disproven with statistical analysis even without any understanding of the underlying mechanisms at play their causes. Superstition is beyond that; it is a belief that cannot be proven and that is not based on logic. The problem with superstition is not that there is no causal link between the supposed cause and the effect, it’s that there is not even a correlation.
- halayli 3y agoSuperstition is assuming a causation when you identify a correlation.
- brookst 3y agoOften followed by applying confirmation bias to see correlations when there are none.
- bsaul 3y agoi think this is the part of superstition. Reproductive, effective, correlation is the start of a scientific path to true discoveries. Superstition only begins when the correlation has never truly never been observed, but only repeated from generations to generations without any other kind of justification other than "that's what i heard".
- better_sh 3y agobut what does Ja Rule think?
- amelius 3y agoLLMs are horribly broken. But we somehow like the idea of putting everything behind an API and then call it a solved problem.
- ianbicking 3y agoThis feels like a limited and perhaps naive perspective on LLMs. If you looked at computers as adding machines in the 60s/70s then you'd be missing most of what was interesting about computers. And if you look at LLMs as a question answering service now, you are also missing a lot. It's hard to compare trust of LLMs to other computing, because many of the things that LLMs get wrong and right were previously intractable. You could ask a search engine, but it's certainly no more trustworthy than an LLM, gameable in its own way. The closest might be a knowledge graph or database, which can formally represent some portion of what an LLM represents. To be fair the relational systems can and will give "no answer" when an LLM (like a search engine) always gives some answer. Certainly an issue! But this is all in the realm of coming up with answers in a closed system, hardly the only way LLMs can be used. LLMs can also come up with questions, for instance creating queries for a database. Are these trustworthy? Not entirely, but the closest alternative supportive tool is perhaps some query builder...? I have seen expert humans come up with untrustworthy queries as well... misinterpretation of data is easy and common. That's just one example of how an LLM can be used. If you use an LLM for something that you can directly compare to a non-LLM system, such as speech recognition or intent parsing, it's clear that the LLM is more trustworthy. It can and does do real error correction! That is, you can get higher quality data out of an LLM than you put in. This is not unheard of in computing, but it is uncommon. Internet networking, which Kay refers to, might be an analog... creating reliable connections on top of unreliable connections. What we don't have right now is systematic approaches to computing with LLMs.
- r00fus 3y ago> If you use an LLM for something that you can directly compare to a non-LLM system, such as speech recognition or intent parsing, it's clear that the LLM is more trustworthy Except for the hallucination problem, sure. How do you ensure the answer you get from an LLM is not a hallucination? It sure doesn't.
- pixl97 3y agoHow do you ensure any bits of data? You going to fist fight your uncle at the next family reunion when he promises you pigs can actually fly? There are plenty of bits of data that can be hard references to datasets or actual physics to assign their probability value. And there are other things that can't.
- LAC-Tech 3y agoTypical Alan Kay. Very long, very thought provoking, full of historical references, and barely answered the question at all. :)
- r00fus 3y agoHe did give a TL;DR about halfway through his answer: "Finally, to try to answer the question … (Summary: I don’t think it would be a good idea at all)"
- keybored 3y agoWhat I like about programming (real programming) is that it is dumb and obtuse. You can see why things happen. Because the instruction languages are painfully literal. They will throw their hands up if you omit a step. Or crash. You can understand it. That’s why I dread the AI future.
- aaroninsf 3y agoAlways a welcome and thoughtful opinion, but not correct about the potential for LLM. Waving away their output as "BS" is awfully flip, and IMO is hence one more example of Ximm's Law (that every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.) One might find ambiguity in his criticism though, that LLM alone are insufficient... but, that's what Ximm's Law is saying. It's not very interesting to (as I would say, he does here) take on straw, rather than steel. (A steely defense of LLM is to say that no one is particularly interested in scaling LLM without other improvements, though scaling alone provides improvements; multi-modal, multi-language, long-context, and most of all augmented systems which integrate LLM into systems rather than making them "systems on a chop", are where things look and IMO will be interesting.)
- svieira 3y ago> By “help” I mean that — especially when changes in epistemological points of view from one’s own common sense are required , it can make a huge difference to be near a “special human” whose personality is strong enough to make us rethink what we think we know. This is how I measure Ed-Tech companies. Do they have an awareness that you cannot replace the connection with other human beings that is an essential part of teaching with "facts" or not? If "yes, they have that awareness" how do they mitigate the problem?
- slaymaker1907 3y ago> If we look at human anthropology, we see a species that treats solitary confinement and banishment from society as punishments — we are a society because we can cooperate and trust a little — but when are safely back in society, we start competing like mad (and cheating like mad) as though the society is there to be strip-mined. I really like this quote. We simultaneously value trust and community, yet so many people also treat it as just another resource to turn into money and power. Alan Kay is a real gem.
- PCMPSTR 3y agoThis part of his answer: > A key part of their design was to not allow direct sending of commands — only bits could be sent. This means that (other) software inside each physical computer has the responsibility to interpret the bits, and the power to do (or not do) some action seems, at least on a basic reading, to contradict this famous little argument (or maybe trolling?) he had on HN with Rich Hickey where he seems to be suggesting that one shouldn't just send raw bits, but also a little interpreter along with the data: https://news.ycombinator.com/item?id=11945722 https://news.ycombinator.com/item?id=11945722 Maybe this is an inevitable consequence of always speaking so abstractly/vaguely, but it also makes it difficult to know what exactly he's suggesting the industry, that he is so routinely critical of, should concretely do next.
- discreteevent 3y agoThe main point he was making in that argument is that data is nothing without an interpreter. Sending an interpreter with the days is secondary. Rich hickey was saying that data is meaningful on its own. But that makes no sense.
- deleted 3y ago[deleted]