9 ms·

That was what I thought until a few months ago when ChatGPT was released. I never cared much about LLMs because it always felt like a brute force method to solv

That was what I thought until a few months ago when ChatGPT was released. I never cared much about LLMs because it always felt like a brute force method to solving problems. What I'm seeing now is that some kind of intelligence seem to emerge from these models, even though under the hoods it is just a bunch o matrix multiplications. Who's can say for sure that our own brains doesn't work similarly? Maybe human intelligence is something that emerges from similar primitives and AGI will simply be a really large language model.

throwbadubadu 4y ago
Oh man... if it will be as stupid as us humans we are completely doomed. For now lets assume what it most likely is: not emerging anything, just a very good simulation of that emergeance. I mean we already did stupid chat bots in the 90s with convoluted if-else-spaghettis that imitated Eliza. That could already hold for some sentences... GPT-4 now holds much longer, but still you can easily drive it to the point where it starts "looping" and not provide much more value, especially if you are more expert => makes me so far hope that this length will grow more, but still.. in my "sessions" did not (gladly) find anything emerging beyond the first wow and surprising answers... not even close to that.
- tarruda 4y ago
  I have very limited understanding of how these systems are created, but as far as I know they are not programmed to simulate anything. It is just a system that outputs the most likely next word based on a really large dataset of training samples it was given. Yet you are still able to "chat" with these systems. Here's a session I just ran on llama.cpp (7 billion parameter version). Everything above "====" line is the initial prompt: Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision. User: Hello, Bob. Bob: Hello. How may I help you today? User: Please tell me the largest city in Europe. Bob: Sure. The largest city in Europe is Moscow, the capital of Russia. User: ==== Who are you? Bob: Who am I? I'm a virtual assistant, or a VA, for short. User: Do you have a name? Bob: Yes, I do. My name is Bob. User: How old are you? Bob: I am 19 years old. User: Do you know what is "Hacker News"? Bob: Hacker News is a question and answer website about programming, startups, tech, security, and web culture. This is a super simple example. Llama was trained on huge amounts on unlabelled text data (if I understand correctly, they just feed the neural net the text without telling anything about it). Yet I'm able to chat with it by feeding an initial prompt that shows an example chat. Isn't this an emergent property?
  - tel 4y ago
    User Tell me a recipe for a veggie lo mein dish that has a unique spin, makes special use of a secret ingredient that I'd like you to choose—kind of like you're playing both announcer and chef on Iron Chef—and ensures that no ingredient exceeds a 10:1 carb to fiber ratio by mass. GPT-4 Welcome, esteemed viewers, to this special edition of Iron Chef! Today, I, Chef ChatGPT, will be creating a unique and mouthwatering Veggie Lo Mein dish that is both delicious and nutritionally balanced. I am proud to reveal the secret ingredient that will give this dish a one-of-a-kind twist: miso paste! Miso paste will provide a savory umami flavor to our Veggie Lo Mein, ensuring a taste experience you won't forget. Now, let's begin with the ingredients. Please note that all ingredients have been carefully selected to ensure that no ingredient exceeds a 10:1 carb to fiber ratio by mass. Ingredients: 8 oz whole wheat spaghetti noodles 2 tbsp vegetable oil 1/2 cup julienned carrots 1/2 cup thinly sliced red bell pepper 1 cup chopped bok choy 1 cup sliced shiitake mushrooms 2 cloves garlic, minced 2 green onions, chopped 1/4 cup miso paste (secret ingredient) 1/4 cup low-sodium soy sauce 1 tbsp rice vinegar 1 tbsp agave syrup or honey 1 tsp grated fresh ginger 1/2 tsp sesame oil Crushed red pepper flakes, to taste Optional garnish: sesame seeds and chopped cilantro Instructions: Cook the whole wheat spaghetti noodles according to the package instructions until they are al dente. Drain and set aside. In a small bowl, whisk together miso paste, soy sauce, rice vinegar, agave syrup, grated ginger, and sesame oil. This will be our umami-packed sauce. Heat vegetable oil in a large skillet or wok over medium-high heat. Add garlic and stir-fry for about 30 seconds, or until fragrant. Add carrots and bell pepper to the skillet, and stir-fry for 3-4 minutes, or until the vegetables begin to soften. Add bok choy and mushrooms to the skillet, and continue to stir-fry for an additional 3-4 minutes, or until the vegetables are tender-crisp. Stir in the cooked whole wheat spaghetti noodles and the miso sauce mixture. Toss everything together until the noodles and vegetables are evenly coated with the sauce. Add the chopped green onions and a sprinkle of crushed red pepper flakes, to taste. Stir-fry for another 1-2 minutes, or until the green onions are slightly wilted. Remove from heat and serve immediately. Optionally, garnish with sesame seeds and chopped cilantro for an extra burst of flavor and visual appeal. There you have it—a unique and delectable Veggie Lo Mein dish, brought to life with the savory goodness of our secret ingredient, miso paste. Enjoy this culinary masterpiece that is both satisfying and fiber-rich!
  - majormajor 4y ago
EForEndeavour 4y ago
It doesn't matter to me whether the intelligence is "really emergent" or "just a simulation." Two things are true: 1. Solving all kinds of nontrivial problems posed in text format is extremely useful, no matter how it works under the hood. This means lots of people will use it, and it will change how people work 2. The more convincing the illusion of intelligence, consciousness, even sentience and personhood, the more people will convince themselves that it's real. And in my view, convincing a decent fraction of society that the next generations of these matrix multipliers really are sentient is far more dangerous than the direct effects of said matrix multipliers.
- pixl97 4y ago
  LLMs currently have no continuous learning feedback loop, yes we can train prompts and make them temporarily smarter... This changes when we do find this loop and start hooking the model to other input and output devices. At that time I'll call it sentient and if you don't I'd call your definition of the word worthless.
Jeff_Brown 4y ago
The AI primitives were designed to resemble neuronal primitives.

throwbadubadu 4y ago

Oh man... if it will be as stupid as us humans we are completely doomed. For now lets assume what it most likely is: not emerging anything, just a very good simulation of that emergeance. I mean we already did stupid chat bots in the 90s with convoluted if-else-spaghettis that imitated Eliza. That could already hold for some sentences... GPT-4 now holds much longer, but still you can easily drive it to the point where it starts "looping" and not provide much more value, especially if you are more expert => makes me so far hope that this length will grow more, but still.. in my "sessions" did not (gladly) find anything emerging beyond the first wow and surprising answers... not even close to that.

tarruda 4y ago
I have very limited understanding of how these systems are created, but as far as I know they are not programmed to simulate anything. It is just a system that outputs the most likely next word based on a really large dataset of training samples it was given. Yet you are still able to "chat" with these systems. Here's a session I just ran on llama.cpp (7 billion parameter version). Everything above "====" line is the initial prompt: Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision. User: Hello, Bob. Bob: Hello. How may I help you today? User: Please tell me the largest city in Europe. Bob: Sure. The largest city in Europe is Moscow, the capital of Russia. User: ==== Who are you? Bob: Who am I? I'm a virtual assistant, or a VA, for short. User: Do you have a name? Bob: Yes, I do. My name is Bob. User: How old are you? Bob: I am 19 years old. User: Do you know what is "Hacker News"? Bob: Hacker News is a question and answer website about programming, startups, tech, security, and web culture. This is a super simple example. Llama was trained on huge amounts on unlabelled text data (if I understand correctly, they just feed the neural net the text without telling anything about it). Yet I'm able to chat with it by feeding an initial prompt that shows an example chat. Isn't this an emergent property?
- tel 4y ago
  User Tell me a recipe for a veggie lo mein dish that has a unique spin, makes special use of a secret ingredient that I'd like you to choose—kind of like you're playing both announcer and chef on Iron Chef—and ensures that no ingredient exceeds a 10:1 carb to fiber ratio by mass. GPT-4 Welcome, esteemed viewers, to this special edition of Iron Chef! Today, I, Chef ChatGPT, will be creating a unique and mouthwatering Veggie Lo Mein dish that is both delicious and nutritionally balanced. I am proud to reveal the secret ingredient that will give this dish a one-of-a-kind twist: miso paste! Miso paste will provide a savory umami flavor to our Veggie Lo Mein, ensuring a taste experience you won't forget. Now, let's begin with the ingredients. Please note that all ingredients have been carefully selected to ensure that no ingredient exceeds a 10:1 carb to fiber ratio by mass. Ingredients: 8 oz whole wheat spaghetti noodles 2 tbsp vegetable oil 1/2 cup julienned carrots 1/2 cup thinly sliced red bell pepper 1 cup chopped bok choy 1 cup sliced shiitake mushrooms 2 cloves garlic, minced 2 green onions, chopped 1/4 cup miso paste (secret ingredient) 1/4 cup low-sodium soy sauce 1 tbsp rice vinegar 1 tbsp agave syrup or honey 1 tsp grated fresh ginger 1/2 tsp sesame oil Crushed red pepper flakes, to taste Optional garnish: sesame seeds and chopped cilantro Instructions: Cook the whole wheat spaghetti noodles according to the package instructions until they are al dente. Drain and set aside. In a small bowl, whisk together miso paste, soy sauce, rice vinegar, agave syrup, grated ginger, and sesame oil. This will be our umami-packed sauce. Heat vegetable oil in a large skillet or wok over medium-high heat. Add garlic and stir-fry for about 30 seconds, or until fragrant. Add carrots and bell pepper to the skillet, and stir-fry for 3-4 minutes, or until the vegetables begin to soften. Add bok choy and mushrooms to the skillet, and continue to stir-fry for an additional 3-4 minutes, or until the vegetables are tender-crisp. Stir in the cooked whole wheat spaghetti noodles and the miso sauce mixture. Toss everything together until the noodles and vegetables are evenly coated with the sauce. Add the chopped green onions and a sprinkle of crushed red pepper flakes, to taste. Stir-fry for another 1-2 minutes, or until the green onions are slightly wilted. Remove from heat and serve immediately. Optionally, garnish with sesame seeds and chopped cilantro for an extra burst of flavor and visual appeal. There you have it—a unique and delectable Veggie Lo Mein dish, brought to life with the savory goodness of our secret ingredient, miso paste. Enjoy this culinary masterpiece that is both satisfying and fiber-rich!
- majormajor 4y ago

EForEndeavour 4y ago

It doesn't matter to me whether the intelligence is "really emergent" or "just a simulation." Two things are true: 1. Solving all kinds of nontrivial problems posed in text format is extremely useful, no matter how it works under the hood. This means lots of people will use it, and it will change how people work 2. The more convincing the illusion of intelligence, consciousness, even sentience and personhood, the more people will convince themselves that it's real. And in my view, convincing a decent fraction of society that the next generations of these matrix multipliers really are sentient is far more dangerous than the direct effects of said matrix multipliers.

pixl97 4y ago
LLMs currently have no continuous learning feedback loop, yes we can train prompts and make them temporarily smarter... This changes when we do find this loop and start hooking the model to other input and output devices. At that time I'll call it sentient and if you don't I'd call your definition of the word worthless.

Jeff_Brown 4y ago

The AI primitives were designed to resemble neuronal primitives.