5 ms·

ReactAgent: LLM Agent for React Coding

130 points by omarfarooq 3y ago

RayanSord 3y ago
[flagged]
sam-prospect 3y ago
Very cool to see, I can't wait to use something like this especially once someone automates the loop of write -> run code -> check errors -> fix errors for an LLM
iandanforth 3y ago
The critical element of these systems is that it takes you less time to get to production worthy code using the tool than it would without it. You can't just stop a demo at the point where you don't have working code. Anyone with a ChatGPT plus subscription can ask it to generate react components from a mockup, but it doesn't do a great job. You have to show your tool doing a good job, not just flipping through component files.
- weeksie 3y ago
  Yeah, it really didn't seem like there was a lot of "there" there. Maybe I'm missing something in the codebase but they're just piping specs to the gpt-4 api?
- ushakov 3y ago
  What is “a good job” according to your definition? If it can save even 10% time, then the tool is worth the money. And by the way, they never promised anything > ReactAgent is an experimental autonomous agent
  - mdhb 3y ago
    I don’t know how it is even going to achieve that… It’s going to give you a code base that is filled with subtle bugs, no tests and no documentation. You won’t have a good understanding of your own product and you are going to almost certainly end up wasting more time over even a modest time window compared to the feeling of productivity you had at the start.
    - giancarlostoro 3y ago
      I feel like you could build reliable scaffolding tools with zero AI. Code generation tools are underused a lot of the time.
      - mdhb 3y ago
        Correct and from there you can still build the tests and documentation to go with it and actually understand how everything works. This is literally hoping that some linear algebra process is going to magically put all of the right things together in all the right ways while maintaining all of the correct syntax and the underlying logic will make sense. Sometimes I think people forget that it’s just a glorified guessing game of what letter most likely comes next.
    - lgas 3y ago
      It will only give you that codebase if that's what you ask it for. If you ask it for tests it will give you tests. If you ask it for documentation it will give you documentation.
- MuffinFlavored 3y ago
  > doing a good job Doing a good job is the subjective part, no? Especially with opinionated things like programming in general/React
  - givemeethekeys 3y ago
    There are: - Style guidelines - Unit tests - Functional requirements - System requirements If the tool creates commits to pass all of them, then it is objectively doing a good job.
- kodapoda 3y ago
  For me, a critical part of "doing a good job" is how the implementation fits into a larger system. ChatGPT is not very good at that. I tried asking it to build a basic state management component for a TypeScript/React app. It offered a class-based component. I asked to use a closure instead. It offered a closure but skipped some of my other requirements that it has previously added (like types). I asked to add in-memory caching. It added caching but removed something else. I asked to create a context provider based on this component, it created a context provider but skipped some parts from the state management implementation. Basically, it sort of works if you can hand-hold it and pay attention to every detail. But that barely saves me any time. And the code it generates is definitely not production ready and requires refactoring in order to integrate it into an existing code base.
  - lgas 3y ago
    A good trick here is to use the API rather than the web interface and give it access to functions that let it edit files directly. Then you don't have to do the iterations where you ask it for something new and it forgets something old, it just updates the code in place, leaving all the old stuff and changing/adding the new stuff.
    - kodapoda 3y ago
      Interesting. Is there an IDE (ideally, VSCode) extension you could recommend for that? What do you mean by giving access? Is it possible to limit its access only to certain functions/files?
      - vishnumenon 3y ago
        Not the GP, but I've been working on an open platform [0] for integrating OpenAI APIs with other tools (VSCode, JupyterLab, bash, chrome, etc) that you might find interesting, the VSCode integration supports editing specific files / sections etc. Also worth taking a look at Github Copilot Chat[1], it's a bit limited but in certain cases it works well for editing specific parts of files. [0] https://github.com/lightrail-ai/lightrail https://github.com/lightrail-ai/lightrail [1] https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat https://marketplace.visualstudio.com/items?itemName=GitHub.c...
klysm 3y ago
Aren't all of these thin branded clients around chatgpt extremely high risk ventures for both the company and the customers? OpenAI can change their prices, or their terms whenever they please and the crunch _will_ come in the future when they have to make back their money they are currently throwing away to grow.
- omarfarooq 3y ago
  There's competition for alternatives. https://www.promptingguide.ai/models/collection https://www.promptingguide.ai/models/collection Life will find a way.
- HPsquared 3y ago
  They clearly didn't cost much to make. So in terms of business risk it's one of those "high probability, low consequence" type risks. There's upside to be made for as long as it remains viable. Just don't bet the farm on it.
- ushakov 3y ago
  In my opinion, they’re not, because OpenAI makes generalized tools, that lack specificity (still need prompting, splitting, etc.). I’d see it as AI as a Platform. The AI space will get hyper-competitive now with companies like Anthropic and open-source projects like Llama. LLMs will become a commodity
  - omneity 3y ago
    I tend to subscribe to this viewpoint. With the size of the opportunity and the level of competition, and barring some exponential compounding dynamic (possible in this space of all industries), I don’t see foundational model providers like OpenAI able to compete at so many levels of abstraction at once (as in both platform and vertical solutions). What we’re more likely to see is some sort of consolidation and collapse of layers of what used to be a viable business. Companies who are not actively working on differentiation and adding real value will simply start withering away, at the same time giving space to nimbler teams that operate what used to take hundreds of people to manage. tl;dr: No direct competition but consolidation and disruption of current operators.
gardnr 3y ago
The namespace is overloaded. This 2022 paper decsribes a method for looping LLM generations in a specific way to generate more reasonable decisions and outcomes: "ReAct: Synergizing Reasoning and Acting in Language Models" https://react-lm.github.io/ https://react-lm.github.io/ This is the type of agent that AutoGPT uses.
- minimaxir 3y ago
  I had an excellent pun in my blog post about LangChain's problems: > Wars about software complexity and popularity despite its complexity are an eternal recurrence. In the 2010’s, it was with React; in 2023, it’s with ReAct.
  - ushakov 3y ago
    Can you share your blogpost?
    - minimaxir 3y ago
      The Hacker News discussion is here: https://news.ycombinator.com/item?id=36725982 https://news.ycombinator.com/item?id=36725982
ChrisLTD 3y ago
Is there something like this for Swift & SwiftUI development? My day job is React, so I need a lot less help there than with my personal iOS apps.
- mlboss 3y ago
  Just ask ChatGPT4. It is pretty good. Now even better with vision support. Upload a wireframe download code.
worldsayshi 3y ago
Is the demo video intentionally set to double speed or is that a bug/typo?
- benatkin 3y ago
  IDK, but it's a loom video: <iframe width="100%" height="315" src="https://www.loom.com/embed/591fd03b54d04a74a15995815de47c76 https://www.loom.com/embed/591fd03b54d04a74a15995815de47c76" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe> Maybe they checked our gyroscopes and thought based on that we wanted to watch it at double speed. Edit: weird, if I click that link, it plays at normal speed.
blah_why_not 3y ago
[dead]
ssalat 3y ago
You could easily setup the same with https://formula8.ai https://formula8.ai - even more flexible. Just write a React Component Formula.
JCharante 3y ago
Just a heads up, the video playback was 2x by default so I couldn’t understand anything until I looked at the settings. I don’t think it’s a previous setting from me because I’ve never used loom before.
shanghaikid 3y ago
AI powered no-code.
madrox 3y ago
We’ve done experiments with generative UI code, but we keep coming back to the LLM making crappy looking UI, and if the engineer still needs to tweak the code afterward this is just a glorified copilot. Without more specialized models, I believe the next step is an intermediate design language…something an LLM can’t make look bad. Lots of room to innovate here though. Really glad to see people working on this problem.
eylonmiz 3y ago
Hey everyone, happy to be featured here and see your thoughts! I'm the original author, I was curious to find a way to generate production code with LLM, and I did find a path to this, it requires a lot of work and engineering, solving a complex problem by splitting it to multiple smaller ones that an LLM / LLM agent can take. IMO it is doable, but it requires a lot of work and potentially funding, so I left this project to do something fundable in this space :) I wish I could keep working on it full time, if anyone wants to help doing that, or ask anything, reach out!
10xDev 3y ago
Millions of frameworks and hype tools stacked together now regurgitated by a chat bot that can't reason. All to bring you more generic sites that can't run on processors with billions of clock cycles per second. Web "developers" should be embarrassed.

130 points by omarfarooq 3y ago

[flagged]

Very cool to see, I can't wait to use something like this especially once someone automates the loop of write -> run code -> check errors -> fix errors for an LLM

iandanforth 3y ago

The critical element of these systems is that it takes you less time to get to production worthy code using the tool than it would without it. You can't just stop a demo at the point where you don't have working code. Anyone with a ChatGPT plus subscription can ask it to generate react components from a mockup, but it doesn't do a great job. You have to show your tool doing a good job, not just flipping through component files.

weeksie 3y ago
Yeah, it really didn't seem like there was a lot of "there" there. Maybe I'm missing something in the codebase but they're just piping specs to the gpt-4 api?
ushakov 3y ago
What is “a good job” according to your definition? If it can save even 10% time, then the tool is worth the money. And by the way, they never promised anything > ReactAgent is an experimental autonomous agent
- mdhb 3y ago
  I don’t know how it is even going to achieve that… It’s going to give you a code base that is filled with subtle bugs, no tests and no documentation. You won’t have a good understanding of your own product and you are going to almost certainly end up wasting more time over even a modest time window compared to the feeling of productivity you had at the start.
  - giancarlostoro 3y ago
    I feel like you could build reliable scaffolding tools with zero AI. Code generation tools are underused a lot of the time.
    - mdhb 3y ago
      Correct and from there you can still build the tests and documentation to go with it and actually understand how everything works. This is literally hoping that some linear algebra process is going to magically put all of the right things together in all the right ways while maintaining all of the correct syntax and the underlying logic will make sense. Sometimes I think people forget that it’s just a glorified guessing game of what letter most likely comes next.
  - lgas 3y ago
    It will only give you that codebase if that's what you ask it for. If you ask it for tests it will give you tests. If you ask it for documentation it will give you documentation.
MuffinFlavored 3y ago
> doing a good job Doing a good job is the subjective part, no? Especially with opinionated things like programming in general/React
- givemeethekeys 3y ago
  There are: - Style guidelines - Unit tests - Functional requirements - System requirements If the tool creates commits to pass all of them, then it is objectively doing a good job.
kodapoda 3y ago
For me, a critical part of "doing a good job" is how the implementation fits into a larger system. ChatGPT is not very good at that. I tried asking it to build a basic state management component for a TypeScript/React app. It offered a class-based component. I asked to use a closure instead. It offered a closure but skipped some of my other requirements that it has previously added (like types). I asked to add in-memory caching. It added caching but removed something else. I asked to create a context provider based on this component, it created a context provider but skipped some parts from the state management implementation. Basically, it sort of works if you can hand-hold it and pay attention to every detail. But that barely saves me any time. And the code it generates is definitely not production ready and requires refactoring in order to integrate it into an existing code base.
- lgas 3y ago
  A good trick here is to use the API rather than the web interface and give it access to functions that let it edit files directly. Then you don't have to do the iterations where you ask it for something new and it forgets something old, it just updates the code in place, leaving all the old stuff and changing/adding the new stuff.
  - kodapoda 3y ago
    Interesting. Is there an IDE (ideally, VSCode) extension you could recommend for that? What do you mean by giving access? Is it possible to limit its access only to certain functions/files?
    - vishnumenon 3y ago
      Not the GP, but I've been working on an open platform [0] for integrating OpenAI APIs with other tools (VSCode, JupyterLab, bash, chrome, etc) that you might find interesting, the VSCode integration supports editing specific files / sections etc. Also worth taking a look at Github Copilot Chat[1], it's a bit limited but in certain cases it works well for editing specific parts of files. [0] https://github.com/lightrail-ai/lightrail https://github.com/lightrail-ai/lightrail [1] https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat https://marketplace.visualstudio.com/items?itemName=GitHub.c...

klysm 3y ago

Aren't all of these thin branded clients around chatgpt extremely high risk ventures for both the company and the customers? OpenAI can change their prices, or their terms whenever they please and the crunch _will_ come in the future when they have to make back their money they are currently throwing away to grow.

omarfarooq 3y ago
There's competition for alternatives. https://www.promptingguide.ai/models/collection https://www.promptingguide.ai/models/collection Life will find a way.
HPsquared 3y ago
They clearly didn't cost much to make. So in terms of business risk it's one of those "high probability, low consequence" type risks. There's upside to be made for as long as it remains viable. Just don't bet the farm on it.
ushakov 3y ago
In my opinion, they’re not, because OpenAI makes generalized tools, that lack specificity (still need prompting, splitting, etc.). I’d see it as AI as a Platform. The AI space will get hyper-competitive now with companies like Anthropic and open-source projects like Llama. LLMs will become a commodity
- omneity 3y ago
  I tend to subscribe to this viewpoint. With the size of the opportunity and the level of competition, and barring some exponential compounding dynamic (possible in this space of all industries), I don’t see foundational model providers like OpenAI able to compete at so many levels of abstraction at once (as in both platform and vertical solutions). What we’re more likely to see is some sort of consolidation and collapse of layers of what used to be a viable business. Companies who are not actively working on differentiation and adding real value will simply start withering away, at the same time giving space to nimbler teams that operate what used to take hundreds of people to manage. tl;dr: No direct competition but consolidation and disruption of current operators.

gardnr 3y ago

The namespace is overloaded. This 2022 paper decsribes a method for looping LLM generations in a specific way to generate more reasonable decisions and outcomes: "ReAct: Synergizing Reasoning and Acting in Language Models" https://react-lm.github.io/ https://react-lm.github.io/ This is the type of agent that AutoGPT uses.

minimaxir 3y ago
I had an excellent pun in my blog post about LangChain's problems: > Wars about software complexity and popularity despite its complexity are an eternal recurrence. In the 2010’s, it was with React; in 2023, it’s with ReAct.
- ushakov 3y ago
  Can you share your blogpost?
  - minimaxir 3y ago
    The Hacker News discussion is here: https://news.ycombinator.com/item?id=36725982 https://news.ycombinator.com/item?id=36725982

ChrisLTD 3y ago

Is there something like this for Swift & SwiftUI development? My day job is React, so I need a lot less help there than with my personal iOS apps.

mlboss 3y ago
Just ask ChatGPT4. It is pretty good. Now even better with vision support. Upload a wireframe download code.

worldsayshi 3y ago

Is the demo video intentionally set to double speed or is that a bug/typo?

benatkin 3y ago
IDK, but it's a loom video: <iframe width="100%" height="315" src="https://www.loom.com/embed/591fd03b54d04a74a15995815de47c76 https://www.loom.com/embed/591fd03b54d04a74a15995815de47c76" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe> Maybe they checked our gyroscopes and thought based on that we wanted to watch it at double speed. Edit: weird, if I click that link, it plays at normal speed.

blah_why_not 3y ago

[dead]

ssalat 3y ago

You could easily setup the same with https://formula8.ai https://formula8.ai - even more flexible. Just write a React Component Formula.

JCharante 3y ago

Just a heads up, the video playback was 2x by default so I couldn’t understand anything until I looked at the settings. I don’t think it’s a previous setting from me because I’ve never used loom before.

shanghaikid 3y ago

AI powered no-code.

madrox 3y ago

We’ve done experiments with generative UI code, but we keep coming back to the LLM making crappy looking UI, and if the engineer still needs to tweak the code afterward this is just a glorified copilot. Without more specialized models, I believe the next step is an intermediate design language…something an LLM can’t make look bad. Lots of room to innovate here though. Really glad to see people working on this problem.

eylonmiz 3y ago

Hey everyone, happy to be featured here and see your thoughts! I'm the original author, I was curious to find a way to generate production code with LLM, and I did find a path to this, it requires a lot of work and engineering, solving a complex problem by splitting it to multiple smaller ones that an LLM / LLM agent can take. IMO it is doable, but it requires a lot of work and potentially funding, so I left this project to do something fundable in this space :) I wish I could keep working on it full time, if anyone wants to help doing that, or ask anything, reach out!

10xDev 3y ago

Millions of frameworks and hype tools stacked together now regurgitated by a chat bot that can't reason. All to bring you more generic sites that can't run on processors with billions of clock cycles per second. Web "developers" should be embarrassed.