3 ms·

Using the tool in this way is a bit like mining: repeatedly hacking away with a blunt instrument (simple prompt) looking for diamonds (100x speedup out of nowhe

Using the tool in this way is a bit like mining: repeatedly hacking away with a blunt instrument (simple prompt) looking for diamonds (100x speedup out of nowhere). Probably a lot of work will be done in this semi-skilled brute-force sort of way.

exitb 2y ago
Well, in this case it's kind of similar to how people write code. A loop consisting of writing something, reviewing/testing, improving until we're happy enough. Sure, you'll get better results with an LLM when you're more specific, but what's the point then? I don't need AI when I already know what changes to make.
- HPsquared 2y ago
  This method requires a lot less skill, you just need to read it and run it. Reading is much easier than writing!
  - adrianN 2y ago
    For many types of code writing it is much easier than reading it unfortunately.
    - HPsquared 2y ago
      Reading to understand all the subtext and side-effects can be harder than writing, sure. But it won't stop people trying this approach and hammering out code full of those types of subtle bugs. Human developers will be more focused on this type of system integration and diagnostics work. There will be more focus on reading and understanding than the actual writing. It's a bit like working with contractors.
mingus88 2y ago
It looks to me to be exactly what a typical coding interview looks like; the first shot is correct and works, and then the interviewer keeps asking if you can spot any ways to make it better/faster/more efficient If I were a CS student cramming for interviews, I might be dismayed to see that my entire value proposition has been completely automated before I even enter the market.
bodegajed 2y ago
There must be a feedback request mechanism for a "Is this better?" This is doable with RLHF or DPO.
antihero 2y ago
Once you can basically have it run and benchmark the code, and then iterate that overnight, it’s going to be interesting. Automating the feedback loop is key.
- XCSme 2y ago
  Wouldn't there be some safety concerns to letting the AI run overnight with access to run any command? Maybe if it can run sandboxed, with no internet access (but if the LLM is not local, it does require internet access).

exitb 2y ago

Well, in this case it's kind of similar to how people write code. A loop consisting of writing something, reviewing/testing, improving until we're happy enough. Sure, you'll get better results with an LLM when you're more specific, but what's the point then? I don't need AI when I already know what changes to make.

HPsquared 2y ago
This method requires a lot less skill, you just need to read it and run it. Reading is much easier than writing!
- adrianN 2y ago
  For many types of code writing it is much easier than reading it unfortunately.
  - HPsquared 2y ago
    Reading to understand all the subtext and side-effects can be harder than writing, sure. But it won't stop people trying this approach and hammering out code full of those types of subtle bugs. Human developers will be more focused on this type of system integration and diagnostics work. There will be more focus on reading and understanding than the actual writing. It's a bit like working with contractors.

mingus88 2y ago

It looks to me to be exactly what a typical coding interview looks like; the first shot is correct and works, and then the interviewer keeps asking if you can spot any ways to make it better/faster/more efficient If I were a CS student cramming for interviews, I might be dismayed to see that my entire value proposition has been completely automated before I even enter the market.

bodegajed 2y ago

There must be a feedback request mechanism for a "Is this better?" This is doable with RLHF or DPO.

antihero 2y ago

Once you can basically have it run and benchmark the code, and then iterate that overnight, it’s going to be interesting. Automating the feedback loop is key.

XCSme 2y ago
Wouldn't there be some safety concerns to letting the AI run overnight with access to run any command? Maybe if it can run sandboxed, with no internet access (but if the LLM is not local, it does require internet access).