3 ms·

The evolution of comments here on HN are telling. 1 month ago people were saying there was no way AI could do their job and many were telling everyone to calm

The evolution of comments here on HN are telling. 1 month ago people were saying there was no way AI could do their job and many were telling everyone to calm down as nothing would happen. Now one month later this entire thread is about how chatGPT is an integral part of the development process. There are people even saying it replaces junior developers completely. Can anyone venture to guess what the thread will look like in 1 more year? "chatGPT can do the work of an entire team!" And one more year after that? "chatGPT can manage a team!"

lolinder 4y ago
This is about what the comments looked like immediately after ChatGPT came out the first time. Give it a few weeks for the GPT-4 hype to settle and I think we'll be right back to where we were.
- Herval_freire 4y ago
  Unlikely. GPT-4 wasn't as game changing as chatGPT. People are less hyped up and have a more realistic expectation of what they should see out of an LLM. The extreme positive reaction to gpt4 is imo not driven by hype fuel. In fact, if you recall the hype fuel surrounding the initial chatGPT was in both parts positive and negative. There were equal portions of people praising it and people vehemently dismissing it as a stochastic parrot. That's not what I see here.
  - lolinder 4y ago
    There weren't equal parts. I was in the tiny minority saying to chill out—there was a month or so where almost every comment was hyped. It gradually moved to equal parts and then eventually to a majority of realistic perspectives. I agree that it's more measured this time—fewer end of the world predictions—but I still fully expect to see expectations mellow out over the next few weeks.
MagicMoonlight 4y ago
ChatGPT 5 runs its own team of ChatGPT 4s which get weaker hardware because they are lower ranking and don’t deserve the same resources
elicksaur 4y ago
You also have to think about comments sections not being a random sample of the site’s users. Likely, a month ago, anyone was interested in commenting on AI threads, so more skepticism was evident. Now, most people who are AI-skeptic are probably tired of AI news and have said their piece in other threads, so don’t bother. fwiw, I’ve tried to use ChatGPT to make a new, useful Elixir library and it failed miserably. It makes me wonder if the success with JavaScript/Python uses are because the question and answers are extensively in the training data whereas a relatively novel Elixir program is something it can’t do because it’s not truly “reasoning”. It can’t even get basic command line prompts that are straight from the Elixir or Phoenix docs correct, and it cannot solve the error messages that its code produces like others claim experiencing for JavaScript/Python uses. If my hunch about training contamination is correct, we’re going to quickly run into not being able to create novel software if we replace everyone with GPT algos, unless their core functionality vastly changes from excellent pattern matching to true reasoning.
- Herval_freire 4y ago
  No there is reasoning in LLMs. This is actually proven through several scientific papers. I can't say whether it's using reasoning for specific answers. I'm sure many times it just regurgitates something that is known... But saying that LLMs in general can and do reason is categorically true. LLMs do attempt to extract actual mental models of the subject of the text. See here: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559 For your specific case, given the conditions, another explanation that fits as well is that the LLM simply needs more data to build a more complete model for elixir. Once it does that I'm sure it can respond with completely novel elixir code as it already does with python.
  - timbeccue 4y ago
    Is this the type of reasoning that could be used to basic math without errors (without using a dedicated math engine)? I wonder why that type of reasoning seems to struggle. I think chatgpt is interesting in its ability to highlight the subtle variations in a concept we tend to think of as more of a single attribute (e.g. reasoning) because humans tend to do certain types of reasoning tasks at the same relative proficiency as others. No human can write complex code without being able to add large numbers, so they are often lumped into the same skill category.

lolinder 4y ago

This is about what the comments looked like immediately after ChatGPT came out the first time. Give it a few weeks for the GPT-4 hype to settle and I think we'll be right back to where we were.

Herval_freire 4y ago
Unlikely. GPT-4 wasn't as game changing as chatGPT. People are less hyped up and have a more realistic expectation of what they should see out of an LLM. The extreme positive reaction to gpt4 is imo not driven by hype fuel. In fact, if you recall the hype fuel surrounding the initial chatGPT was in both parts positive and negative. There were equal portions of people praising it and people vehemently dismissing it as a stochastic parrot. That's not what I see here.
- lolinder 4y ago
  There weren't equal parts. I was in the tiny minority saying to chill out—there was a month or so where almost every comment was hyped. It gradually moved to equal parts and then eventually to a majority of realistic perspectives. I agree that it's more measured this time—fewer end of the world predictions—but I still fully expect to see expectations mellow out over the next few weeks.

MagicMoonlight 4y ago

ChatGPT 5 runs its own team of ChatGPT 4s which get weaker hardware because they are lower ranking and don’t deserve the same resources

elicksaur 4y ago

You also have to think about comments sections not being a random sample of the site’s users. Likely, a month ago, anyone was interested in commenting on AI threads, so more skepticism was evident. Now, most people who are AI-skeptic are probably tired of AI news and have said their piece in other threads, so don’t bother. fwiw, I’ve tried to use ChatGPT to make a new, useful Elixir library and it failed miserably. It makes me wonder if the success with JavaScript/Python uses are because the question and answers are extensively in the training data whereas a relatively novel Elixir program is something it can’t do because it’s not truly “reasoning”. It can’t even get basic command line prompts that are straight from the Elixir or Phoenix docs correct, and it cannot solve the error messages that its code produces like others claim experiencing for JavaScript/Python uses. If my hunch about training contamination is correct, we’re going to quickly run into not being able to create novel software if we replace everyone with GPT algos, unless their core functionality vastly changes from excellent pattern matching to true reasoning.

Herval_freire 4y ago
No there is reasoning in LLMs. This is actually proven through several scientific papers. I can't say whether it's using reasoning for specific answers. I'm sure many times it just regurgitates something that is known... But saying that LLMs in general can and do reason is categorically true. LLMs do attempt to extract actual mental models of the subject of the text. See here: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559 For your specific case, given the conditions, another explanation that fits as well is that the LLM simply needs more data to build a more complete model for elixir. Once it does that I'm sure it can respond with completely novel elixir code as it already does with python.
- timbeccue 4y ago
  Is this the type of reasoning that could be used to basic math without errors (without using a dedicated math engine)? I wonder why that type of reasoning seems to struggle. I think chatgpt is interesting in its ability to highlight the subtle variations in a concept we tend to think of as more of a single attribute (e.g. reasoning) because humans tend to do certain types of reasoning tasks at the same relative proficiency as others. No human can write complex code without being able to add large numbers, so they are often lumped into the same skill category.