3 ms·

Thank you for supporting an open source ecosystem. A couple of thoughts. 1. It would be nice if we could ingest external resources like web pages. 2. Deep in

Thank you for supporting an open source ecosystem. A couple of thoughts. 1. It would be nice if we could ingest external resources like web pages. 2. Deep integration with the debugger (Limitation mentioned in your docs). Code pilot seems to be very superficial here. I know the issues above are limited by the token context. However, there's not a lot of innovation in these two aspects for code generation.

sestinj 3y ago
The first is already possible today! Using "ContextProviders", you can specify external resources to use as context. For example, we have a GitHub Issues ContextProvider that lets you type '@issue' and then search through open issues in your repo. We haven't yet built a ContextProvider for web pages, but this is a great idea. Theoretically, just type '@<URL>', then the LLM will know about the page. The second sounds quite interesting. I'm curious whether you envision as far as the language model stepping through a program on its own? We had considered working on this at one point in time.
- FloatArtifact 3y ago
  All right. Honestly I could see it both ways depending on the needs of the developer. The language model could step through autonomously or with the developer. Thinking of how this could look like abstractly through a UI. Picture sequential timeline which consists each step the developer or the model has made through the code. A timeline that the developer could fast forward or rewind which has some implications for the LLM. There are three main areas the developer could give LLM inputs. 1. Before debugging The user could give input (Trace back, a question, ingested context (URL) before initializing debugging, instruction steps, autonomously or guided. Autonomously given the context of before debugging, it could even optionally set its own breakpoints. 2. At any step during debugging LLM might influence how the developers steps through the code. 3. Post debugging Now imagine each step through the debugger being logged to which the user could ask questions within the context of the history of the debugger. LLM could have different scoped contexts relative to the timeline of the steps of interest. Imagine the developer steps through 15 points in the code. Developer could ask about step 7 or a range of steps post debugging. LLM could be limited scope to only the previous steps including 7 or a range. Alternatively it could have knowledge of the entire step history. The underline assumption there is limiting context in some cases improves output of LLM.
  - sestinj 3y ago
    This is fantastic thinking. Actually I'm quite excited about the combination of fault-localization techniques and LLMs; taking a traceback, pruning the call-graph, and seeking out bugs. There's a real future in all of this
  - FloatArtifact 3y ago
    Profiling code then optimizing autonomously would be interesting to see as it edits the code.

sestinj 3y ago

The first is already possible today! Using "ContextProviders", you can specify external resources to use as context. For example, we have a GitHub Issues ContextProvider that lets you type '@issue' and then search through open issues in your repo. We haven't yet built a ContextProvider for web pages, but this is a great idea. Theoretically, just type '@<URL>', then the LLM will know about the page. The second sounds quite interesting. I'm curious whether you envision as far as the language model stepping through a program on its own? We had considered working on this at one point in time.

FloatArtifact 3y ago
All right. Honestly I could see it both ways depending on the needs of the developer. The language model could step through autonomously or with the developer. Thinking of how this could look like abstractly through a UI. Picture sequential timeline which consists each step the developer or the model has made through the code. A timeline that the developer could fast forward or rewind which has some implications for the LLM. There are three main areas the developer could give LLM inputs. 1. Before debugging The user could give input (Trace back, a question, ingested context (URL) before initializing debugging, instruction steps, autonomously or guided. Autonomously given the context of before debugging, it could even optionally set its own breakpoints. 2. At any step during debugging LLM might influence how the developers steps through the code. 3. Post debugging Now imagine each step through the debugger being logged to which the user could ask questions within the context of the history of the debugger. LLM could have different scoped contexts relative to the timeline of the steps of interest. Imagine the developer steps through 15 points in the code. Developer could ask about step 7 or a range of steps post debugging. LLM could be limited scope to only the previous steps including 7 or a range. Alternatively it could have knowledge of the entire step history. The underline assumption there is limiting context in some cases improves output of LLM.
- sestinj 3y ago
  This is fantastic thinking. Actually I'm quite excited about the combination of fault-localization techniques and LLMs; taking a traceback, pruning the call-graph, and seeking out bugs. There's a real future in all of this
- FloatArtifact 3y ago
  Profiling code then optimizing autonomously would be interesting to see as it edits the code.