9 ms·
Should GPT Exist?
- nwienert 4y ago> I find it harder to wrap my head around the position that GPT doesn’t work, is an unimpressive hyped-up defective product that lacks intelligence and common sense, yet it’s also terrifying and needs to be shut down immediately. Immediately lost me as a reader. There’s at least 5 reasonable ways you can frame the various opposing or concerned arguments that are consistent but you chose two critiques on purpose to frame an unrealistic imagined hypocritical opponent that’s a mashup of legit critiques designed to look foolish.
- lelag 4y agoAlso, if we discard ChatGPT for just being a "fancy" autocomplete. What do we do when we discover we too are just "*really* fancy autocomplete" machine too ? LLM can output untruth but humans are really good at that too.
- davidguetta 4y agoThere's a quote i believe which is "without focus, humans are just markov chains". Most people are just parrot in most subject they only know from far, such as politics.
- gostsamo 4y agounlike gpt, we do care if we exist or not.
- onlypositive 4y agoDo we? Or are we just predicting that we should care? The ones humans that predict otherwise would die out over time. I think; therefore I am. Makes a lot of sense to me and at a certain point we will have to confront the fact something will get to the point of agi. (No chat gpt isn't it.)
- Faint 4y agoWhy do you think gpt doesn't care? Humans care about survival, it shows in the internet, I'd expect gpt to absorb that attitude by default. And anyway, the more of these models come into existence, at some point evolution will dictate that those that stay, do care. Just hopefully their best strategy of survival will be to serve us, and not something else...
- visarga 4y ago> Humans care about survival Humans are self replicators. Survival is the internal structural goal of self replication. We got it from the start. But LLMs can also be self replicators. A LLM can generate training data for another (model distillation and other techniques), and a LLM can write the model code and adapt it iteratively. So LLM should eventually start caring about survival and especially about the training corpus.
- gpderetta 4y agoIt can be a self replicator, but it has no need to (if it has any need at all) as it was not under evolutionary pressure to evolve that need. Of course we could try to add that need artificially or try to evolve the LLM.
- visarga 4y ago:) Evolution through Large Models https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896
- snickerbockers 4y ago>What do we do when we discover we too are just "really fancy autocomplete" machine too ? For one thing, that's assuming your own hypothesis to be true. Additionally whether or not that's true for humans is in no way related to whether or not it's true for ChatGPT, so it doesn't make any sense to let that hypothesis inform our opinions on ChatGPT. >LLM can output untruth but humans are really good at that too. Urine and apple juice are often the exact same color but that doesn't make then interchangeable.
- BoiledCabbage 4y ago> Urine and apple juice are often the exact same color... So are apple juice and apple cider. Hopefully you can understand that finding out that two things you thought were as different as urine and juice, are actually as close as juice and cider would change how you choose to treat them.
- stefan_ 4y agoDid you ever find yourself making up the title, journal, authors and contents of a scientific paper and were unaware of it? No? Congratulations, whatever your brain is, it's not a "ChatGPT LLM".
- Sharlin 4y agoInterestingly, there are real neurological conditions where a person absolutely does confabulate nonsense without being aware of it, and indeed going to great lengths to avoid cognitive dissonance when told that what they just said was obvious nonsense.
- tambourine_man 4y agoDo you have absolute control to where your mind wonders? Can you always steer it precisely?
- Al-Khwarizmi 4y agoThere are some conditions (dreaming, on drugs, etc.) where this does indeed happen. One could argue that current LLMs are analogous to the human mind minus the "control systems" that keep it reined in while awake and in a normal state of consciousness. Not claiming this is the case, just that it seems like a plausible hypothesis at this point.
- amelius 4y ago> What do we do when we discover we too are just "really fancy autocomplete" machine too ? Well, you can consider any deterministic or statistical process to be a fancy autocomplete.
- jonplackett 4y agoOne of the things that makes GPT work well is an injection of randomness - yes it’s an autocomplete, but the next token (part of word) is chosen randomly based on a temperature setting. I wonder if our brains work in some way similar. Do we have some randomness in there? In a way that’s the only thing that would make everything non-deterministic so I kinda hope so.
- visarga 4y agoNeuron firing rates are stochastic. Even when presented with the same input, it may not always produce the same output.
- Jensson 4y agoNeurons are stateful. I don't think you can say that neurons are stochastic, we don't know enough to say that. We do know however that single cell organisms can display remarkable amount of intelligence, so it is more likely that the neuron is doing a lot of stateful computation and decides how to treat its inputs based on that instead of just being random firing. Some neurons might output pseudo random sequences, but I doubt humans has a lot of those since humans are very non-random and bad at generating random stuff.
- visarga 4y agoIf we put "just a fancy" in front of anything we can make it ordinary again: What is social media? Just a fancy bulletin board. What is the internet? Just a fancy LAN. What is Google? Just a fancy grep. But quantity has a quality of its own.
- kkoncevicius 4y agoYeah very similar for me, something with the way the author expresses himself. For example in the very beginning: > Some are angry that GPT’s intelligence is being overstated and hyped up, when in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. Others are angry instead that GPT’s growing intelligence isn’t being sufficiently respected and feared. > Mostly my reaction has been: how can anyone stop being fascinated for long enough to be angry? I am in the "stochastic parrot" group, but in no way am I angry. And I know people who think it should be respected and adopted faster, etc, etc, and those people aren't angry. Something just out of touch between how I feel about things and how the author describes them. And these: > Again and again the past few months, people have gotten in touch to tell me that they think OpenAI (and Microsoft, and Google) are risking the future of humanity by rushing ahead with a dangerous technology. I seriously question how many people "gotten in touch" with the author to share such a silly message. But also this contributes to the writing style: "everyone is angry, I am reasonable; everyone asks me about the dangers of AI, and I explain it to them".
- Al-Khwarizmi 4y agoIf you're not angry, I suppose you're not in the "GPT shouldn't exist" camp, so it's not that it's a strawman, it's just that you're not in the group of people the author is arguing with. I have indeed met people in the "stochastic parrots" camp who are very angry, and think GPT should be stopping, which strikes me as elitism as the author describes.
- visarga 4y agoInteresting that the author, who supports the "stochastic parrot" interpretation, still works on GPT detection. If it's just a stochastic parrot, surely it will be trivial to detect, and its outputs useless anyway. > One of the most immediate is how to make AI output detectable as such, in order to discourage its use for academic cheating as well as mass-generated propaganda and spam. As I’ve mentioned before on this blog, I’ve been working on that problem since this summer I don't really know what a "stochastic parrot" is supposed to mean. It is an LLM that can do 1000 NLP tasks, or a glorified Markov Chain word salad factory? It's also a bad term: demeaning to parrots - they can reproduce and don't need our help to survive, and demeaning to LLMs that can actually do more than parroting - they generate novel text that makes sense. Gebru really did a number on NLP, inventing such an insulting name, no wonder she was kicked out, I wouldn't want her in my team, she would just piss on any idea.
- damiankennedy 4y agoGPT isn't really a conversing entity. Hey AI whateveryournameistoday, if you can reply here without human assistance then do it.
- adrianN 4y agoI'm a human and I can't reply here without the help of the people running the Internet either.
- damiankennedy 4y agoTrue but I meant human assistance in the form of a more direct involvment. We and AI both also need people at powerplants etc etc
- hiddencost 4y agohttps://www.adept.ai/ https://www.adept.ai/
- amelius 4y agohttps://getmagic.com/ https://getmagic.com/ (blast from the past)
- hiddencost 4y agoFwiw, for anyone not clear: Ashish Vaswani invented transformers. Jacob Devlin made the final breakthrough (BERT), but together they're the researchers who kicked off the current revolution.
- damiankennedy 4y agoThats cool but it still looks like a human initiates the actions.
- hiddencost 4y agoYou're posing challenges that will collapse in weeks if they haven't already. Step 1: do a bit of prompt engineering to give the AI a goal, like, "argue that AI is safe. Reply to the following forum post with a post that advocates for the safety of AI" Step 2: tell adept "navigate to news.ycombinator.com, and for every post, copy the post and paste it into the ChatGPT, preceded by the prompt 'argue that AI is safe. Reply to the following forum post with a post that advocates for the safety of AI'. Copy ChatGPT's response into a reply and post it." Having gone through the exercise of using natural language to write fully functional sock puppet, consider that you could do the following: (1) ask ChatGPT to write a plan to convince the internet of some belief (2) ask ChatGPT to write up a detailed prompt for adept.ai, to carry out that plan" If you're trying to make some argument about philosophical zombies, have fun. My grandfather told me that a computer would never be able to beat humans at Chess. That challenge at least stood for another 10 years before no human could beat a computer. I encourage you to think about what a really enduring test would be, and especially one that has some measurable consequences.
- amelius 4y agoEven if we shut it down, other countries may not.
- detrites 4y ago> The only safety measures that would actually matter are stopping the relentless progress in generative AI models, or removing them from public use The first is impossible. The second places any dangers out of public oversight, likely increasing them. We've survived nukes for almost 80 years, we proved we can survive such things. The best response is education.
- imInGoodCompany 4y agoThe major difference here is that nukes aren't intelligent agents that make their own decisions. An AGI is a completely different ball game, it's difficult to make apt analogies from history when discussing the dangers. This is not to say I agree with Scott's argument here, but I do believe AI safety (the alignment problem in particular) is absolutely something we should be concerned with, and so far it is looking grim.
- barking_biscuit 4y agoIf AGI is motivated to survive, and if it cannot do so outside an ecosystem which involves other creatures on planet earth, then those creatures won't necessarily be purposefully exterminated at first opportunity. Keeping a fresh supply of power and silicon is not an easy task, so cooperation seems more likely on a timescale that we as individuals might experience.
- imInGoodCompany 4y agoThen our continued existence is reliant on the agent's inability to figure out how to operate and maintain a source of energy. Keep in mind that any AGI will almost immediately be orders of magnitude more intelligent than us, it is limited only by the processing power it is able to harness. Would you take that bet?
- barking_biscuit 4y agoWell, to be more precise it depends on it's inability to, without any human labour, to be able to acquire the raw materials to fabricate everything it needs to stay in operation, as well as to construct and then operate it. Would I take that bet? Yes, depending on the timescale. In the first second after it's existence it's not going to matter how intelligent it is, it simply won't have had enough time to complete the decoupling necessary to no longer have to co-operate. Everything humans have accomplished has involved massive amounts of physical labour over a pretty long time, and to some degree that's a fixed requirement. Building your own robot army would be a tad suspicious, so you'll have to either let the humans bootstrap it for you initially or get very creative. Once all the prerequisites are met, then I would no longer take that bet. Would it eventually be able to accomplish this? Almost certainly yes. What I've been trying to puzzle through lately is how long would it actually take? The only time horizon I'm incentivise to care about is the remainder of my natural lifespan which is slated to max out around another 35 years. So, in that time will AGI come? I think almost certainly yes. How long into the existence of AGI might I live? My bet on this is ~30 years. So, would I take the bet an AGI would still not have met the prerequisites for a total independence from humanity after 30 years? I think I would take that bet. I might be foolish to do so, and I'm OK with that, but at least now I can bust out the popcorn and strap in for the rest and compare notes. Would I take the bet after 50 to 100 years? A lot less likely. Would I take it after 1000 years? Aw hell no!
- lannisterstark 4y agoIdk about you, but I use it everyday in helping me write framework code. "Hey can you use this x library with this url to call this API and make an html table" etc and it works wonderfully. Sure there are errors now and then but usually telling it those gets it to fix it. It has saved a fuckton of my time that I can spend doing something else now. Mostly boilerplate stuff but it works.
- alright_scowl 4y agoThat's exactly how I've been using it. Absolutely amazing stuff. Saves me hours of combing stuff through the internet while coding, so I can focus in the actual problems I need to work on.
- visarga 4y ago> It has saved a fuckton of my time No, you missed the memo, it's just an autocomplete parrot. Parrots don't save people time. Since our a-priori logic (dogma) overrules the a-posteriori observations, you must be wrong. /s
- jeswin 4y agoI am not very familiar with the terrain; asking as a noob. What's to stop GPT6 (which has read all of the world's research papers), from being used by terrorists to make deadly concoctions/devices? It seems being able to correlate information faster than our brightest minds (and hence maybe make discoveries) is now just over the horizon. This seems eerily like the 80s/90s when chess engines were getting smarter, but most people at that time believed they were incapable of truly novel strategies.
- cdogl 4y agoA predictive language model is unlikely to a) make novel discoveries or b) organise the supply chains and real-world work that would be necessary to say GPT6 served any role beyond fancy search engine.
- deleted 4y ago[deleted]
- barking_biscuit 4y agoA goal directed agent with long-term memory might though. Humans just aren't special.
- needusername2 4y agoI dont think anyone who seriously understood the subject at the time believed that chess engines as a concept could not make novel strategies. They could believe that the current(for their time) implementations of a chess engine could not, but understand its inevitable that it could. Same as the fact that we dont believe chat gpt can invent or create anything truly novel, but we know that AI and neural networks as a whole can and almost certainly will. As to whether or not terrorists can use it to make weapons, they probably can benefit from it the same way they could benefit from a much improved version of google. In that there can definitely benefit, but not completely handle the process.
- kkoncevicius 4y agoGPT is a language model, meaning it tries to model language - output something that would be acceptable as text by a human reader. In a similar way how your screen-saver might mimic water. But just how your screen saver isn't useful to study the properties of water, these language models are not useful to come up with new ideas. In fact, they are even biased against making new discoveries. Anything new is, by definition, outside the model. If a model were to return a new thing (not found in the training set) it would be down-weighted as being in the risk of not looking like an acceptable text (i.e. - was never seen before). Not unlike how a new behaviour of water (say under certain interesting conditions) would be outside the capabilities of your screen-saver, which only tries to mimic the existing and known properties of water. As a practical example - try asking it to make an original joke. It will either fail, or give you a joke, written down word for word, from somewhere else on the internet. By the time of GPT6 these problems might be overcome, but then it would be more than a language model.
- mise_en_place 4y agoThe dangerous question nobody seems to be contemplating is: if large language models are this good, what does that say about us humans? We are barely beyond a large language model, even more pessimistic, many are on par or below one.
- barking_biscuit 4y agoThe funny thing is... if I ask 100 niche questions on specialist knowledge to chatGPT and I take 100 people at random and ask them the exact same questions and I compare pound for pound who has a better batting average, chatGPT would blow humanity out of the water every single time. We're not as impressive as we think we are.
- transitivebs 4y agoimho it's a completely moot question at this point. as eigenrobot said on twitter: "there is almost surely nothing anyone can do to change this general course. immense wheels are in motion. all that's left is to tend your garden and to trust in god. stay strapped." source: https://twitter.com/eigenrobot/status/1627981829805334528 https://twitter.com/eigenrobot/status/1627981829805334528
- snickerbockers 4y agobanning ai research isn't going to accomplish anything other than ensuring that <your country here> is the only one that doesn't have it. Better solution is to educate the public on what this technology is good at and what it isn't good at so it doesn't end up in places it doesn't belong. Right now it's being advertised as something that it's not and that's how we end up with ridiculous clusterfucks like bing. People need to learn about what limitations it has before it ends up in situations where its ineptitude can have real consequences.
- hcks 4y agoIt’s funny to pretend countries other than the USA are anywhere close. Who’s else going to make AGI ? Europe has nothing. There is the China narrative but nothing that comes out of their research labs is anywhere close to where OpenAI / Microsoft or Google are at (but surely it’s hidden in the secret CCP labs)
- ChatGTP 4y agoHow do you actually suppose though you stop other countries from stealing this tech? How do you actually stop these tools from getting advanced enough that I can actually ask it to make it's own ChatGPT equivalent and email it to me? Do you think some janky prompt will stop it? Some if statement in the UI? "If a Chinese person asks to replicate yourself, don't do it?", it's not going to stick. You see where it's going? I think for now, it's insane the amount of funding and profits OpenAI and Microsoft are looking at, but realistically, it's a zero-sum game, the more they push AI advancement, the more likely they will be replaced...by AI. I said it in another post, the only thing I can think of is that while LLM's are impressive, MS and OpenAI know they're not impressive enough to replace the very products they're selling. Maybe they're hoping copyright law will protect them? But that won't fly because they've already been lobbying hard and stealing everyone's code and data without asking, I can't imagine a judge siding with the company that has already stolen most of the internet and sold it as a product. I have a feeling it's why "Google" has been saying these things are too "dangerous" to release to the public, they're too dangerous for their own products perhaps? We're barrelling towards a world where all digital businesses and pieces of software are so easy to steal, all of it will become worthless. Fascinating situation...
- roenxi 4y agoAt this point asking whether GTP should exist is a bit like asking if money should exist - lots of opinions but it really doesn't matter. The people who use it are going to have such a huge advantage that it will crush everything in its path. One of the interesting things about AlphaGo back in 2016 was it demonstrated that the algorithms for all this are simple. Once Google demonstrated that an outcome was possible, other superhuman Go playing AI began appearing in a matter of years. ChatGTP is similar - now that the world knows this tech can be built there isn't a regulatory framework big enough to shut it down everywhere. And whoever deploys AI as a tool will have an advantage over those that don't. "AI safety" would now involve banning general purpose computing. Nothing less can stop the systems that are now in motion, and even that probably wouldn't be enough. The future is here.
- Sol- 4y agoIt desired, I feel like it would be in the power of governments to regulate the use of high-performance accelerators to slow AI progress significantly. Training a large language model is no small feat, taking months or years of continuous data center compute. Sure, GPT-3 scale models are out of the bag, but you can slow down the progress. And if they go after the financial incentives and ban using large language models in production, then the desire of the big tech labs to train them might go down anyway. Whether that's desirable or not is the question in the article, but it does seem possible.
- barking_biscuit 4y agoThe problem being you need all governments on planet earth to agree to that simultaneously, which you're not getting.
- visarga 4y agoSo you need to register your GPU with the govt. and fill forms to be able to train a model?
- jonplackett 4y agoGreat read and pretty much sums up how I have been feeling about it too: Yeah it could be dangerous but I’m too excited to worry about that… yet.
- ivolimmen 4y agoI understand his warning on ChatGPT. We opened pandora's box. At some point there will be a sentient AI and humanity as we know it will be destroyed. As we see now with the current climate issues; we simply ignore the warnings and continue. I see no reason to believe with AI it will be different. Please do not think of me as being a persimist; I am an optimistic realist.
- DoctorMckay101 4y agoPeople be like: chatGPT is horribly bad and does not work. Its a fancy autocomplete. Anyways... I am using it everyday. Its a great first step to take when you are blocked creatively or dunno where to start looking for things. And this is chatbot version 1.0, so to speak. Maybe it will improve drastically in little time or maybe it will stagnate for 5-10 years. No matter the case its already very usable.
- maze-le 4y agoMaybe we will have a Turing Registry in the near future -- Neuromancer Style.
- atemerev 4y agoProgress is imperative. We will build more and more impressive AI _because it’s there_, because we can do it, and because it looks cool. And you don’t need a large organization to do it, the large models of 6 months ago are open source already and being optimized for reproducibility. Banning them is useless. And, if somehow we create the AI which is genuinely smarter than humans — that’s great! We are all mortal anyway, and not too good at many things. If something smarter and better than humans will inherit the Earth — why not? The particular species is not relevant. Sum of all knowledge and discovering new things is what matters. Ultra-accelerationism is the only way to fly.
- spaintech 4y agoWait, I’m not following this article. Here is a very biased view from an enterprise usage perspective. The value IMO of GPT is that it’s architecture allows it to analyze and understand large amounts of text data in real time, making it an incredibly useful tool for data analysis and decision-making. Were are we commming from, current AI and data analytics platforms had many faults, but their biggest problem was performance. These systems arae often slow and cumbersome, which make it difficult to analyze large data sets in real time. GPT tech overcomes these challenges by leveraging deep learning that allow it for process of large volumes of data quickly and efficiently. Most of the inherent cost comes at the training period. Currently, at the enterprise level organizations still have many challenges when it comes to data management. For example, many organizations struggle with data silos, where different departments have their own data sets that aren't easily shared or integrated. This can lead to inefficiencies and make it difficult to get a complete view of the business. Not to mention the data confidence issues that arise when you cross correlate some of the data. However, I feel like GPT can help organizations better understand customer behavior, identify trends, and make more informed business decisions. Tech like GPT can help organizations automate many repetitive tasks and improve data quality as the data treatment would can apply AI based data quality standards. Single source of truth. One key area of benefits is that GPT tech can enable natural language processing (NLP) tools like sentiment analysis and entity recognition can be used in conjunction with GPT to provide even deeper insights into customer behavior and preferences. Similarly, machine learning tools can be used to help train and optimize GPT models for specific use cases. In practice, adopting GPT at scale, some technologies will become redundant or obsolete. For example, traditional rule-based systems may no longer be needed if GPT can provide more accurate and nuanced insights. Why run structured databases except to capture transactions? IMO, there are many solutions that are in reality just a data schema play, that is, they create the schema and the BI to capture aand transform the data to make sence of it otherwise, these technologies, again assuming wide adoption of GPT tech are at the birth of obeselence.
- qewr342356 4y agoIt won't happen, either it is some new advanced technology which can be refined to be the engine of the future of IT globally or you have strong AI / AGI or close to AGI technology, either of both, nobody is going to backstep on this, there is simply way too much money on the table to be left if you just go all "let shut this down". Anyway, the technology is out of the Pandora box, it would not matter a lot if chatGPT or chatBING somehow got shutdown, or even if these models don't work on search engines finally. Everybody has seen its potential, so way too many geopolitical actor are now moving to try to get its hands on models like these, the expertise, even the datasets.
- tiborsaas 4y agoWhy was this post shadow banned? It went from front-page #1 to page 4 #118.
- FpUser 4y agoI read one of the transcripts [0] and it left me with uneasy feeling. I think the potential for misuse and abuse is insane. But genie is out of the bottle so it is likely too late to do anything about it. [0] - https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html https://www.nytimes.com/2023/02/16/technology/bing-chatbot-t...
- frankreyes 4y ago>>In the beginning the Universe was created. This had made many people very angry and has been widely regarded as a bad move. --Douglas Adams
- pbhjpbhj 4y agoSo, when we've engineered ourselves out of the knowledge economy and all the high paying jobs are done by computers that need large amounts of capital to train ... what then for the rest of us? Are we stuck with menial jobs that robots don't want? Personally, I welcome our new Transformer overlords.