4 ms·

What have been some of your learnings for getting agents to work?

• louiskw 3y ago
  Generate as few tokens as possible, GPT4 is running a few times to generate a single answer and latency quickly becomes the biggest UX issue. We abandoned most of the common thinking around chain of thought reasoning, finding it didn’t help accuracy much whilst increasing response times significantly. Full write up to follow in next week or so.
  • reasonabl_human 3y ago
    Does this mean your queries are all one-shot instead of utilizing techniques like LangChain?
    • louiskw 3y ago
      Exactly, you can see the prompt in this file [0]. I'm not sure how LangChain arrived at their default agent prompt, but you'll almost certainly want to write your own for performance reasons if you put something into production. [0] https://github.com/BloopAI/bloop/blob/main/server/bleep/src/webserver/answer/prompts.rs https://github.com/BloopAI/bloop/blob/main/server/bleep/src/...
      • anotherpaulg 3y ago
        This is great that you got gpt-4 to explore the codebase using an agent approach. I tried this previously with gpt-3.5-turbo and have been meaning to revisit it since I got gpt-4 access. I shared some notes on HN awhile back on a variety of experiments I did with gpt-3.5-turbo. https://news.ycombinator.com/item?id=35441666 https://news.ycombinator.com/item?id=35441666