4 ms·

The demo video shows it making a game of life. People really should start using examples that don't have literally thousands of step by step tutorials all over

The demo video shows it making a game of life. People really should start using examples that don't have literally thousands of step by step tutorials all over the web. That includes clones of Wordle, Flappy Bird, generic todo lists, etc. With those, I can't tell how well the abilities would generalize to real world projects.

dartos 2y ago
> I can't tell how well the abilities would generalize to real world projects. The secret is that it doesn’t ;) I doubt our current statistics based AI methods ever will. Programming requires precision. Code is an exact specification for how a machine should operate. I don’t see how we can get perfectly precise responses from a nondeterministic process.
- VMG 2y ago
  LLMs are much more deterministic than humans
- realharo 2y ago
  Well, it's not like I'm just going to trust a comment like this either :) I would like to see it in action, and then I'll form an opinion.
- fragmede 2y ago
  Maybe if you're inventing a new compiler that hasn't existed before, but if all you're doing is yet another a CRUD app for a newly undiscovered niche, then it doesn't have to generalize beyond what already exists. The fact that it's nondeterministic is irrelevant, I can write the same function a bunch of different ways just by choosing different names for variables, does that stop the code from working?
- bakuninsbart 2y ago
  The code assistance tools right now are still a bit crude compared to what they will be in a year or two even if capabilities do not increase. I think it will be very useful to have "someone" read along as I code and answer questions I ask, or pose questions on why I do things a certain way. Honestly, even if it is just stuff like "What would be a good variable name for this?" or "What was the map()-syntax like again?", this would already facilitate flow quite a bit. Not having to context switch as much is a real blessing, and I've only reached it in one language in specific contexts. If I can transfer this to many languages and many contexts, my productivity will rise a lot.
  - dartos 2y ago
    So What you’re looking for is basically a proactive context aware Google search. LLMs are great for that! Sometimes they’re good at translating from one language to another. They miss idioms from time to time, but they’re usually pretty good with that task. Not so much for writing software themselves.
irthomasthomas 2y ago
What demos would you like to see? I have my own agent running on its own Linux system. I reckon it's capable of doing most office work in theory, if you have the budget. On my budget it's a lot more constrained, and often needs a human in the loop to achieve more complex goal. But I'd like to canvas for suggestions for tests I can throw at it. If get an interesting request, I will try and record its attempt. But it's expensive - in the order of a $1 a prompt.
- realharo 2y ago
  One random idea: Wordle clone tutorials are of course all over the web (just google make wordle clone and see the results), but what about a browser extension + Discord bot, that lets groups of friends compare their scores for the official Wordle, in Discord. It would automatically post people's results to a Discord server - both the score, and a picture of the full guesses under a spoiler tag. And maybe during playing, the extension would show status like "if you solve it now, you'd be better than 50% of people in the server today". And post a weekly top 3 at the end of each week. Something like it has probably been done before, but nowhere near as much as the examples I mentioned in my previous comment. Might be a bit too big at this cost though.
anonzzzies 2y ago
It doesn’t work for those generally. Even with rag and large context and clever prompting, it is not doing well as you need something that basically is simple enough to get in one go, so you need a programmer/logical thinker to cut things so small that the AI gets it, which this type of -just tell me what to build- is simply not compatible with. The greatest programming minds on earth have issues with composing complex ideas from simple ideas; now we expect the AI to write the simple ideas and then compose them. Or just one-shot them, which is really far beyond what they can do. We have been working on a solution for this for decades with my team and it has nothing to do with AI. We have to solve it for us first and then it will work for AI; or maybe we will never solve it and then AI won’t be able to piece anything together that’s complex and it hasn’t seen before.

dartos 2y ago

> I can't tell how well the abilities would generalize to real world projects. The secret is that it doesn’t ;) I doubt our current statistics based AI methods ever will. Programming requires precision. Code is an exact specification for how a machine should operate. I don’t see how we can get perfectly precise responses from a nondeterministic process.

VMG 2y ago
LLMs are much more deterministic than humans
realharo 2y ago
Well, it's not like I'm just going to trust a comment like this either :) I would like to see it in action, and then I'll form an opinion.
fragmede 2y ago
Maybe if you're inventing a new compiler that hasn't existed before, but if all you're doing is yet another a CRUD app for a newly undiscovered niche, then it doesn't have to generalize beyond what already exists. The fact that it's nondeterministic is irrelevant, I can write the same function a bunch of different ways just by choosing different names for variables, does that stop the code from working?
bakuninsbart 2y ago
The code assistance tools right now are still a bit crude compared to what they will be in a year or two even if capabilities do not increase. I think it will be very useful to have "someone" read along as I code and answer questions I ask, or pose questions on why I do things a certain way. Honestly, even if it is just stuff like "What would be a good variable name for this?" or "What was the map()-syntax like again?", this would already facilitate flow quite a bit. Not having to context switch as much is a real blessing, and I've only reached it in one language in specific contexts. If I can transfer this to many languages and many contexts, my productivity will rise a lot.
- dartos 2y ago
  So What you’re looking for is basically a proactive context aware Google search. LLMs are great for that! Sometimes they’re good at translating from one language to another. They miss idioms from time to time, but they’re usually pretty good with that task. Not so much for writing software themselves.

irthomasthomas 2y ago

What demos would you like to see? I have my own agent running on its own Linux system. I reckon it's capable of doing most office work in theory, if you have the budget. On my budget it's a lot more constrained, and often needs a human in the loop to achieve more complex goal. But I'd like to canvas for suggestions for tests I can throw at it. If get an interesting request, I will try and record its attempt. But it's expensive - in the order of a $1 a prompt.

realharo 2y ago
One random idea: Wordle clone tutorials are of course all over the web (just google make wordle clone and see the results), but what about a browser extension + Discord bot, that lets groups of friends compare their scores for the official Wordle, in Discord. It would automatically post people's results to a Discord server - both the score, and a picture of the full guesses under a spoiler tag. And maybe during playing, the extension would show status like "if you solve it now, you'd be better than 50% of people in the server today". And post a weekly top 3 at the end of each week. Something like it has probably been done before, but nowhere near as much as the examples I mentioned in my previous comment. Might be a bit too big at this cost though.

anonzzzies 2y ago

It doesn’t work for those generally. Even with rag and large context and clever prompting, it is not doing well as you need something that basically is simple enough to get in one go, so you need a programmer/logical thinker to cut things so small that the AI gets it, which this type of -just tell me what to build- is simply not compatible with. The greatest programming minds on earth have issues with composing complex ideas from simple ideas; now we expect the AI to write the simple ideas and then compose them. Or just one-shot them, which is really far beyond what they can do. We have been working on a solution for this for decades with my team and it has nothing to do with AI. We have to solve it for us first and then it will work for AI; or maybe we will never solve it and then AI won’t be able to piece anything together that’s complex and it hasn’t seen before.