4 ms·

I spent about an hour reading this. I know most people probably won't be able to do the same because of time constraints in their day, but seriously this is the

I spent about an hour reading this. I know most people probably won't be able to do the same because of time constraints in their day, but seriously this is the most awesome post about large language models I've read. Take a second to think about how much time it must have taken to write this, to prepare all of these examples. This post gave me another way to look at something I already partially understood pieces of. It helps me understand how chatgpt can have these moments of greatness. If you think of this like a long lecture you can get a lot out of it. It reminds me of a feynman lecture.

inasio 4y ago
I'm typically a bit wary of Wolfram's posts, but it is very often the case that the content is good enough that I don't mind a bit of self-aggrandizing. This one though I found to be just pure great, super easy and clear intro into the "magic" of LLMs. Thank you.
- GreedClarifies 4y ago
  Best comment! He's clearly brilliant and I read his posts, but usually I have to grit my teeth at least a couple of times due to his self-aggrandizing. But there was almost none of that in this one. Great stuff.
  - PaulDavisThe1st 4y ago
    I'm sure you'd have understood more, though, if you had considered Wolfram Language (TM) and also tried a few examples with Wolfram Alpha. /s
- greenornot 4y ago
  OK, so I started reading the post and initially had the same feeling. May this have been one of those things where he really leaves his pet peeves behind and contributes something? But then his lingo started to creep in. Computational irreducibility, linking to how he invented the concept. Rurial space. Geez man, does everything really have to be a promotion of the cult-like New Kind of Science stuff? Full of links to his own unrelated blog posts? It's sad. I'd really have liked to like the article, he clearly put a lot of effort into it. But I had to stop reading. You can't tell for sure where real explainations end and where his unproven claims start about "knowing" that the universe is just computational and everything just based on some rule from a cellular automaton. And I've gotten conditioned to never start a sentence with "OK, so". It's now giving me the jiffies.
- pedrow 4y ago
  I thought this was an enjoyable and informative article but [0] > As a personal comparison, my total lifetime output ... has been... [78 million words] And, yes, I’ll train a bot from all of that. for non-Wolfram fans - take cover there is a potentially infinite future supply! [0]: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/#the-training-of-chatgpt https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... (paragraph 2)
Upvoter33 4y ago
I had the same reaction. Wow, what a thing to put together. I wonder how many people could produce such a doc, and how long it would take them.
- sixtram 4y ago
  Maybe he produced it with ChatGPT :)
alchemist1e9 4y ago
I was estimating more than 1 hour to read it properly. Since you did it in only 1 hour that’s encouraging me to try tomorrow or another day. I can’t comprehend how ChatGPT can keep such incredible internal state and I’m hoping Wolfram might have some ideas in this post. For instance try the prompt that turns it into a linux terminal and it can closely “hallucinate” approximately the correct changes in state to the host after the commands but long long after the command occurred in the dialogue. To me this shows it extends beyond human language.
bambax 4y ago
Absolutely agree. It's a fantastic piece. Here's my take on it: https://news.ycombinator.com/item?id=34802764 https://news.ycombinator.com/item?id=34802764
FleurBouquet 4y ago
For me, it is not really a time constraint. Wolfram is just on another level. I am not the smartest guy in the room and Wolfram pretty much always is. The coolest thing is chatGPT can actually help me understand this article and bring it down a notch. That to me is what Feynman was great at and Wolfram not so great at. Wolfram "We can think of the construction of computational language—and semantic grammar—as representing a kind of ultimate compression..." chatGPT: The passage you have provided is talking about the idea that the construction of computational language (such as programming languages) and semantic grammar (a way of analyzing the meanings of words and phrases) allows us to compress language into its most basic, essential form. This means that we can talk about the essence of what's possible, without getting bogged down by the many nuances and variations that exist in human language. The author is suggesting that ChatGPT, as an example of a language model, has achieved a similar level of compression and abstraction in its ability to generate language. It can put together language in a semantically meaningful way, without getting hung up on the specific ways that humans might phrase things. In other words, it can focus on the essence of the message being conveyed, rather than being distracted by the many different ways that message could be phrased. This ability to abstract away from the specific details of language and focus on the essence of meaning is seen as a great strength of computational language and of language models like ChatGPT. It allows us to communicate more efficiently and effectively, and to focus on the underlying ideas and concepts being conveyed, rather than getting bogged down in the details of how they are expressed. I get what Wolfram is saying from the chatGPT explanation while there is something about the way Wolfram words things that leaves me confused.

inasio 4y ago

I'm typically a bit wary of Wolfram's posts, but it is very often the case that the content is good enough that I don't mind a bit of self-aggrandizing. This one though I found to be just pure great, super easy and clear intro into the "magic" of LLMs. Thank you.

GreedClarifies 4y ago
Best comment! He's clearly brilliant and I read his posts, but usually I have to grit my teeth at least a couple of times due to his self-aggrandizing. But there was almost none of that in this one. Great stuff.
- PaulDavisThe1st 4y ago
  I'm sure you'd have understood more, though, if you had considered Wolfram Language (TM) and also tried a few examples with Wolfram Alpha. /s
greenornot 4y ago
OK, so I started reading the post and initially had the same feeling. May this have been one of those things where he really leaves his pet peeves behind and contributes something? But then his lingo started to creep in. Computational irreducibility, linking to how he invented the concept. Rurial space. Geez man, does everything really have to be a promotion of the cult-like New Kind of Science stuff? Full of links to his own unrelated blog posts? It's sad. I'd really have liked to like the article, he clearly put a lot of effort into it. But I had to stop reading. You can't tell for sure where real explainations end and where his unproven claims start about "knowing" that the universe is just computational and everything just based on some rule from a cellular automaton. And I've gotten conditioned to never start a sentence with "OK, so". It's now giving me the jiffies.
pedrow 4y ago
I thought this was an enjoyable and informative article but [0] > As a personal comparison, my total lifetime output ... has been... [78 million words] And, yes, I’ll train a bot from all of that. for non-Wolfram fans - take cover there is a potentially infinite future supply! [0]: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/#the-training-of-chatgpt https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... (paragraph 2)

Upvoter33 4y ago

I had the same reaction. Wow, what a thing to put together. I wonder how many people could produce such a doc, and how long it would take them.

sixtram 4y ago
Maybe he produced it with ChatGPT :)

alchemist1e9 4y ago

I was estimating more than 1 hour to read it properly. Since you did it in only 1 hour that’s encouraging me to try tomorrow or another day. I can’t comprehend how ChatGPT can keep such incredible internal state and I’m hoping Wolfram might have some ideas in this post. For instance try the prompt that turns it into a linux terminal and it can closely “hallucinate” approximately the correct changes in state to the host after the commands but long long after the command occurred in the dialogue. To me this shows it extends beyond human language.

bambax 4y ago

Absolutely agree. It's a fantastic piece. Here's my take on it: https://news.ycombinator.com/item?id=34802764 https://news.ycombinator.com/item?id=34802764

FleurBouquet 4y ago

For me, it is not really a time constraint. Wolfram is just on another level. I am not the smartest guy in the room and Wolfram pretty much always is. The coolest thing is chatGPT can actually help me understand this article and bring it down a notch. That to me is what Feynman was great at and Wolfram not so great at. Wolfram "We can think of the construction of computational language—and semantic grammar—as representing a kind of ultimate compression..." chatGPT: The passage you have provided is talking about the idea that the construction of computational language (such as programming languages) and semantic grammar (a way of analyzing the meanings of words and phrases) allows us to compress language into its most basic, essential form. This means that we can talk about the essence of what's possible, without getting bogged down by the many nuances and variations that exist in human language. The author is suggesting that ChatGPT, as an example of a language model, has achieved a similar level of compression and abstraction in its ability to generate language. It can put together language in a semantically meaningful way, without getting hung up on the specific ways that humans might phrase things. In other words, it can focus on the essence of the message being conveyed, rather than being distracted by the many different ways that message could be phrased. This ability to abstract away from the specific details of language and focus on the essence of meaning is seen as a great strength of computational language and of language models like ChatGPT. It allows us to communicate more efficiently and effectively, and to focus on the underlying ideas and concepts being conveyed, rather than getting bogged down in the details of how they are expressed. I get what Wolfram is saying from the chatGPT explanation while there is something about the way Wolfram words things that leaves me confused.