3 ms·

I scanned the paper and it doesn't mention what model they were using within chatgpt. If it was 3.5 turbo, then these results are already meaningless. GPT-4 an

by cjonas 2y ago

I scanned the paper and it doesn't mention what model they were using within chatgpt. If it was 3.5 turbo, then these results are already meaningless. GPT-4 and 4o are much more accurate. I just used GPT-4o to refactor 50 files from react classes to react function components and it did so almost perfectly everytime. Some of these classes were as long as 500 loc.

haolez 2y ago
I'd guess that React code is a lot easier for a LLM, since it's a frequent occurrence in its training dataset and frontend code tends to be repetitive and full of boilerplate. I believe that AI will be a perfect programmer in the future for all niche areas. My point is that frontend will probably be the first niche to be mastered.
- cjonas 2y ago
  I agree, but would say it different: > AI will be a perfect programmer in the future for all NON-niche areas There's going to be a positive/negative feedback loop that makes it hard for new languages and frameworks to gain popularity. And the lack of popularity means lack of training material being generated for the AI to learn. When choosing a tech stack of the future, the ability for AI to pair will be a key consideration.
Hackbraten 2y ago
It says they used GPT-3.5.
- cjonas 2y ago
  Ya if it's GPT-3.5, I'm actually surprised the accuracies were so high! I've been pairing with GPT since 3.5-turbo. I run 20-100 queries a day (have an IDE integration). The improvements for GPT-4 over 3.5 are significant. So far GPT-4o seems like a step-up for most (not all) queries I've run through it. Based on the pricing and speed, my guess is it's a smaller, more optimized model and there are some tradeoffs in that. I'm guessing we'll see a more expensive flagship model from OpenAI this year. But honestly, these details don't really matter... Regardless of the performance and accuracy of the models today, the trend is obvious. AI will be the primary interface for writing all but the most cutting edge code. Two years ago, I thought an AI writing code was 50 years away. Yesterday, I took a picture of an invoice on my phone, and asked GPT to recreate it in HTML and it did so perfectly.
asadotzler 2y ago
Not meaningless when 99% of the people use the free version which apparently has license to lie to them far more than the paid version. What a fucking sick joke, pay up or we lie to you even more.
- mnk47 2y ago
  The new GPT4o model is free. Plus users will get better rate limits and the voice feature, but everyone has access to the best model right now.

by cjonas 2y ago

haolez 2y ago

I'd guess that React code is a lot easier for a LLM, since it's a frequent occurrence in its training dataset and frontend code tends to be repetitive and full of boilerplate. I believe that AI will be a perfect programmer in the future for all niche areas. My point is that frontend will probably be the first niche to be mastered.

cjonas 2y ago
I agree, but would say it different: > AI will be a perfect programmer in the future for all NON-niche areas There's going to be a positive/negative feedback loop that makes it hard for new languages and frameworks to gain popularity. And the lack of popularity means lack of training material being generated for the AI to learn. When choosing a tech stack of the future, the ability for AI to pair will be a key consideration.

Hackbraten 2y ago

It says they used GPT-3.5.

cjonas 2y ago
Ya if it's GPT-3.5, I'm actually surprised the accuracies were so high! I've been pairing with GPT since 3.5-turbo. I run 20-100 queries a day (have an IDE integration). The improvements for GPT-4 over 3.5 are significant. So far GPT-4o seems like a step-up for most (not all) queries I've run through it. Based on the pricing and speed, my guess is it's a smaller, more optimized model and there are some tradeoffs in that. I'm guessing we'll see a more expensive flagship model from OpenAI this year. But honestly, these details don't really matter... Regardless of the performance and accuracy of the models today, the trend is obvious. AI will be the primary interface for writing all but the most cutting edge code. Two years ago, I thought an AI writing code was 50 years away. Yesterday, I took a picture of an invoice on my phone, and asked GPT to recreate it in HTML and it did so perfectly.

asadotzler 2y ago

Not meaningless when 99% of the people use the free version which apparently has license to lie to them far more than the paid version. What a fucking sick joke, pay up or we lie to you even more.

mnk47 2y ago
The new GPT4o model is free. Plus users will get better rate limits and the voice feature, but everyone has access to the best model right now.