3 ms·

Sorry noob question - where can I read more about this "agents" paradigm? Is one agent's output directly calling/invoking another agent? Or there's already fixe

by wg0 2y ago

Sorry noob question - where can I read more about this "agents" paradigm? Is one agent's output directly calling/invoking another agent? Or there's already fixed graph of information flow with each agent (I presume some prompt presets/templates like "you are an expert this only respond in that") sorts of? Also, how much success people have or had with automating the E2E tests for their various apps by stringing such agents together themselves EDIT: Typos

CGamesPlay 2y ago
Fundamentally, "Agent" refers to anything that operates in an "observe-act" loop. So in the context of LLMs, an agent sees an observation (like the code base and test output) and produces an action (like a patch), and repeats.
zEddSH 2y ago
> Also, how much success people have or had with automating the E2E tests for their various apps by stringing such agents themselves together? There’s a few startups in the space doing this like QA Tech in Stockholm, and others even in YC (but I forgot the name). I’m skeptical of how successful they’ll be, not just from complex test cases but things like data management and mistakingly affecting other tests. Interesting to follow just in case though, E2E is a pain!
pavi2410 2y ago
I want to learn about agents too!
hcks 2y ago
Don’t waste your time, it’s been around since GPT3, and had no results so far. Also notice how no frontier lab is working on it.
- zby 2y ago
  Letting the LLM to decide what to do is a powerful technique. For example one pass RAG is very limited: https://zzbbyy.substack.com/p/why-iterative-thinking-is-crucial https://zzbbyy.substack.com/p/why-iterative-thinking-is-cruc... To make it iterative you need the cede the control to the LLM.
zby 2y ago
In practice this means function calling - the LLM chooses the function to call (and its parameters). Usually in a loop with a 'finish' function that returns the control to the outside code. You can do that without function calling - as did the original ReAct paper - but then you have to write your own grammar for the communication with the LLM, a parser for it, and also you need to teach the LLM to use that grammar. This is very time consuming.

by wg0 2y ago

CGamesPlay 2y ago

Fundamentally, "Agent" refers to anything that operates in an "observe-act" loop. So in the context of LLMs, an agent sees an observation (like the code base and test output) and produces an action (like a patch), and repeats.

zEddSH 2y ago

> Also, how much success people have or had with automating the E2E tests for their various apps by stringing such agents themselves together? There’s a few startups in the space doing this like QA Tech in Stockholm, and others even in YC (but I forgot the name). I’m skeptical of how successful they’ll be, not just from complex test cases but things like data management and mistakingly affecting other tests. Interesting to follow just in case though, E2E is a pain!

pavi2410 2y ago

I want to learn about agents too!

hcks 2y ago

Don’t waste your time, it’s been around since GPT3, and had no results so far. Also notice how no frontier lab is working on it.

zby 2y ago
Letting the LLM to decide what to do is a powerful technique. For example one pass RAG is very limited: https://zzbbyy.substack.com/p/why-iterative-thinking-is-crucial https://zzbbyy.substack.com/p/why-iterative-thinking-is-cruc... To make it iterative you need the cede the control to the LLM.

zby 2y ago

In practice this means function calling - the LLM chooses the function to call (and its parameters). Usually in a loop with a 'finish' function that returns the control to the outside code. You can do that without function calling - as did the original ReAct paper - but then you have to write your own grammar for the communication with the LLM, a parser for it, and also you need to teach the LLM to use that grammar. This is very time consuming.