30 ms·
Last year, when Blake Lemoine was fired by Google for claiming his AI was sentient, a lot of people on this website sensibly noted that the danger of these chat
by yowzadave 4y ago
Last year, when Blake Lemoine was fired by Google for claiming his AI was sentient, a lot of people on this website sensibly noted that the danger of these chatbots was not that they were sentient, but that average people could easily be tricked into thinking that they were.
And now here we are: it seems we tech people are every bit as susceptible to this kind of thinking as the average joes we looked down on.
Actually look at this “language” that GPT-4 has written: it’s just a mishmash of features from the most commonly blogged-about existing programming languages. It has no conceptual insights, no original ideas, no taste that isn’t cribbed from one of the few languages it’s copying from.
There are cases where getting a short distillation of the mean opinion of The Internet is useful. But designing a new programming language is clearly not one of them, mostly because the chatbot is not “designing” a language—it is guessing the most likely response to a question about programming languages.
- habitue 4y agoYeah, I find llms are really useful for taking premises you supply, and trying them out, or exploring inconsistencies in them. It's decent at the synthesis kind of creativity (mashups, remixes, etc) but it isn't smart enough yet to make really big inferential leaps that we would consider more original.
- marviel 4y agoI agree with your point about the dangers of chatbots being mistaken as sentient. And I also recognize the dangers of the self-improving systems like this in general -- that's why I'm not letting it run loose with a LangChainAI Agent script to self-execute a new language (though wouldn't that be fascinating?) While the language it generated needs refinement, the design process is iterative, and it takes time to develop a programming language that is worth using. I encourage you to consider how you would approach designing a programming language. Like you, I would likely start by incorporating my favorite features from languages I love and addressing problems I've encountered in the past, similar to the dialogue GPT and I have in the post. As I mention in the post -- creativity often involves remixing existing ideas, and GPT-4's attempt at generating a programming language is no exception. Notably, it also called out explicitly the constructs it was borrowing from other languages when designing the language. It's worth noting that this was GPT-4's initial attempt at something highly complex, and even if there are "plot holes" -- it's a noteworthy achievement in and of itself.
- krupan 4y agoOk, but take a step back and really look at the whole situation here. So you've created a machine that solves problems in kind of the same way humans do. Yay? What overall problem is a machine like this solving? How is this better than just having a human do the job? It seems to have most of the same drawbacks as humans, and then some on top of that
- justinclift 4y ago> How is this better than just having a human do the job? The cynical take is quite clearly: "Because sales people can sell it" eg wrap some pretty UX around it, tell people it's "nearly human", then have your sales people try and get non-technical CxO types (or consultancies) to cut them a cheque
- marviel 4y agoIt's a hard question to be honest. If you look at the incentives involved, people are very incentivized to create these machines to reduce the amount of toil humans have to endure. I don't disagree about the risks at scale of something like this; as I mention in the post -- I don't get pure warm fuzzies about this at all. But I also don't know how to stop AI progress, because the incentives to keep it going are so real. Government regulation has been brought up multiple times, most recently by Sama himself on twitter. I think it will have to come, but I don't know what flavor it should take.
- ummonk 4y agoI don’t have a massive pool of skilled slave labor I can delegate work to, but I can delegate work to ChatGPT.
- KyeRussell 4y agoWhat!? Look back at the last few hundred years, where the majority of great leaps have been the result of getting a machine to do something that humans did.
- geraneum 4y ago> As I mention in the post -- creativity often involves remixing existing ideas This is carrying the weight of your core argument. It’s not wrong but it’s not completely correct either. Creativity involves remixing but is not all remixing. Is it? Not even all humans are at the same level in this. Number of attempts means nothing if you don’t add features that the LLM desperately needs to be grounded in the reality, or perform logical reasoning beyond RLHF.
- cyclotron3k 4y agoIt seems only yesterday that we were marvelling at the capabilities of ChatGPT - it's ability to parse the sum knowledge of humankind and synthesise not trivial responses to questions we posed. But look how far we've come since those heady and carefree days. We're already able to pour buckets of scorn on its capabilities, and brush them aside as if designing a language (albeit a derivative and uninspired one) is a trivial task that any human could achieve.
- Jensson 4y ago> We're already able to pour buckets of scorn on its capabilities No we don't, we are just trying to put a damper on all the "the singularity is here!" style comments that seems to flood any discussion about GPT.
- KyeRussell 4y agoOh please. This community is filled to the brim with people that are addicted to negativity and contrarianism. LLM criticisms very typically go far far beyond “this isn’t the singularity”, well toward “LLMs do nothing of value, snd certainly can’t I - the esteemed developer - with anything I find hard in my job!”
- Jensson 4y agoNot many say "LLMs do nothing of value". Many comments say LLMs aren't useful to them, but I haven't seen many say that LLMs aren't useful at all.
- UncleEntity 4y agoI’ve seen countless comments stating that it is just predicting the next token, move along, nothing to see here.
- Jensson 4y agoThat is literally what LLMs do though. Those comments try to explain how it works so you better can see how it produced the results you see, they aren't trying to say that the program is worthless just that it isn't the magic some people think it is. These models are trained to produce text snippets that look like text snippets it has seen before, and it has seen all internet. That means it can do a lot of impressive things, but also that it is very dumb in other ways.
- avgcorrection 4y ago> And now here we are: it seems we tech people are every bit as susceptible to this kind of thinking as the average joes we looked down on. More. We’re more suspectible. We are the people who take the Turing Test as an objective test of GAI, after all, even though it only tests how we percieve supposed intelligence.
- zuminator 4y agoThe Turing Test isn't meant to be an objective test of GAI. It's meant to replace a question that might be too vague or philosophically contentious to be answerable, "Can machines think?" with an answerable one, "Can a machine successfully fool a person into thinking it's a person?" So a robot bird that fools real birds arguably has passed the Avian Turing Test. (Also, arguably the Turing Test is by definition subjective.)
- avgcorrection 4y ago> The Turing Test isn't meant to be an objective test of GAI. It's meant to replace a question that might be too vague or philosophically contentious [snip] Correct in that that was what Turing intended. But the vulgar interpretation of it makes it out to be more than it really is. For Pete’s sake: two years ago I even saw a psychologist who suggested that psychologists should take notes from how computer science ostensibly measures “intelligence” by the Turing Test, because it’s so “objective”. That was sad.
- xlii 4y agoWell put. I share the sentiment. I used GPT-3/3.5/4 for multiple tasks and I never found it useful. It just feels… average. It can “answer” many questions correctly but that’s only on the data set it’s been training on. I asked it to solve Elixir issues, write Elisp snippets, figure out why my Rust code was complaining about lifetimes and even went for easy mode and asked for JavaScript code. Only JavaScript (very popular) responses were workable but they still had popular problems included in it (e.g. not clearing out timers or dealing with common edge cases). Elixir “hints” were very interesting since - when out of water - it would produce code assuming that it’s a OOP and that was a rabbit hole it declined to exit. Too technical? Went for fun with family members who have exceptional cooking skills. We tried to figure out a dish variation of an apple pie. Suggestions were heavily criticized as non workable for many reasons (too sugary, consistency wouldn’t be too good, modification change the ingredients but GPT refuse to break of out theme etc.) Colin Meloy made an experiment some time ago and created The Decemberists-like song with it and then actually recorded it. It worked but it was described as below average and bland. I share that vibe. In the end I have this feeling that GPT is on a level of 7 year old with an access to vast library. It uses proper grammar it can copy parts of texts, but one needs to provide it with a great benefit of doubt and energy just so that it’s useful. In contrast to 7 year old, GPT isn’t grateful for that sacrifice and is actually making money of you.
- anonzzzies 4y agoYou probably never had to work with one of the 99% of programmers in the world that are not of your level (I am assuming it is high, otherwise you might just be very scared saying all this, and rightfully so), but me, working in the fortune 1000 company space, worked with many. Most of my colleagues can readily be replaced by gpt-4 today as far as coding and documentation goes. They make the same errors as you noticed gpt make (which I have to fix usually), only gpt does it in seconds vs hours and practically for free. By replacing all these people, I don’t have to build in the overhead of communicating with them (I just copy the jira task in gpt with some context and a few seconds later I have something better than most my colleagues write in 2-3 hours, on a good day), and they are happier doing nothing anyway. The company won’t fire them probably as managers get paid more and promoted more ‘managing’ more people. This is not an exception either; teams like mine exist in every large enterprise. I worked in several of them and this AI makes life so much better. Let’s not pretend it makes certain outsourcing countries not fully obsolete though, to the tune of 10s of millions+ programmers, data entry workers, text producers (bloggers for seo, docs no one reads etc) and more are mostly all already worse than gpt3.5 and 4. Wait for 6 or 7.
- RHSman2 4y agoThe question is when does guessing become a moot point? AlphaGo guessed it’s way to being the best. Can something guess it’s way to evolving into a less guessy thing? I think so. Look at us? I am aware of how it works and used to be ‘it’s ok, it won’t happen’ but now I am actually pretty concerned. We think GAI as the thing but there maybe a reality that doesn’t require full GAI to completely unhinge us. I mean, what is the point? It only plays to the negatives of our society. We don’t need it. It allows for ‘do more’ but what cost does that ‘do more’ come with?
- Jensson 4y ago> AlphaGo guessed it’s way to being the best. No it didn't, AlphaGo was backed by an algorithm that perfectly understood go and used that to train itself to play go better. That is a completely different situation. If you tried to make a go engine that only tried to replicate old moves then you'd have a hard time making it perform valid moves. ChatGPT is like such an engine and we just made it perform valid moves most of the time, and now people say "now that it can make valid chess moves it is only a matter of time until it beats masters!". No just statistically reproducing human moves wont lead to a smart chess engine, so why would you think that would make a smart text engine?
- RHSman2 4y agoYou are too close. AlphaGo was trained with tech. It beat humans. ChatGPT et al are being trained. Everyone is arguing the inner workings of what is in the box. It doesn’t matter. The usage of the box(s) is what will be the big impact. Previously those boxes couldn’t be relied upon so stayed down in the hierarchy of usefulness. Now. They are rising.
- oezi 4y agoBecause it is not just reproducing existing moves. An LLM can come to acquire a model which models the structure of moves (in chess for instance using letter an number indexing of board positions), the checking of valid moves and the valuation of board positions. It seems surprising that training an LLM would result in layers/weights which contain such representations but it seems that this is indeed an efficient representation of information in the same way that neural nets would learn addition by encoding textual integers into floats.
- sparkie 4y ago> It’s just a mishmash of features from the most commonly blogged-about existing programming languages. It has no conceptual insights, no original ideas, no taste that isn’t cribbed from one of the few languages it’s copying from. It will also learn all the bugs and reproduce them. Programmers who think ChatGPT is going to replace their job must have a very low opinion of their own skill set.
- Jensson 4y agoIt will also combine correct things in incorrect ways, so even if everything it is trained on is correct it will still print incorrect things. We see it in this article, both the statement "Indentation-based scoping, similar to Python" and the code with bracers were probably correct in their original contexts, but combined they are no longer correct. LLMs will make such errors no matter what kind of data you train it on, there is no known way to avoid that class of errors.
- sparkie 4y agoYes, and imagine if StackOverflow questions are part of the training material. The "AI" will be learning from unfinished/unworking code samples.
- hkt 4y ago> Programmers who think ChatGPT is going to replace their job must have a very low opinion of their own skill set. As a devops guy I live in fear of anything that can copy and paste from stackoverflow faster than I can.
- amoss 4y agoThe interesting part of the exchange is the contrast between the surface level syntax of the exchange and the semantics below. It reminds me of teaching in a lot of ways. The level of polish on the immediately apparent surface level of the conversation is very high: reasonable tenents, good justification and ability to explain them. But then applying them directly into an example falls down a bit. The language is a bit of a mismash and parts of it might not work semantically. The use of actors looks a bit out of place, and raises a lot of questions about how the memory management and scheduling is working under the hood. Despite the bold aims in the tenents thre is nothing in the language exposed so far that addresses them: why would the language produce more modular code in the large? what features of the syntax would increase the level of abstraction in code written? etc etc Overall this would warrant a (barely) passing grade by modern standards, i.e. it can regurgitate the material succesfully but without demonstrating any deep understanding. I find the result quite interesting as it demonstrates how much of what we judge to be quality is actually fitting a particular pattern of response (w.r.t a specific field of study) rather than exploring the material more deeply. I like the use of the arrows in the function definitions though :) I've seen that somewhere before and will be "appropriating" it for the project that I'm currently working on.
- UncleEntity 4y ago> I like the use of the arrows in the function definitions though :) I've seen that somewhere before… Python type hints probably. I (ab)used them for good effect when I was fiddling around with the spark earley parser that used to be part of the python build system.
- dgb23 4y agoI fully agree with your sentiment. However I want to add that it does the guesswork and the presentation surprisingly well to a degree that it’s effective in certain contexts. The more people think of this as a tool that they can use and master, the more they will benefit from it.
- 28304283409234 4y ago99.9999 percent of humans are no more than copiers of other people's ideas. With no conceptual insight, no original ideas, etc etc. That doesn't mean they're not sentient. :-)
- oska 4y agoThis is (arrogant) misanthropy, and I see far too much of it on forums that contain significant numbers of people who consider themselves more 'enlightened' than the common man (reddit being a prominent example).
- deleted 4y ago[deleted]
- nailer 4y agoThe thing is most humans also have no taste. New programming languages are regularly released with noisy syntax by actual people.
- tpoacher 4y agoWhenever there is a simple error that most laymen fall for, there is always a slightly more sophisticated version of the same problem that experts fall for. --Amos Tversky
- camillomiller 4y agoI would provocatively say that GPT-4 is the stupid sentient being's idea of an intelligent sentient being.
- dragonwriter 4y ago> Actually look at this “language” that GPT-4 has written: it’s just a mishmash of features from the most commonly blogged-about existing programming languages. It has no conceptual insights, no original ideas, no taste that isn’t cribbed from one of the few languages it’s copying from. On the one hand, I agree. On the other hand, I see some variant of this comment on HN about every new language, so...