7 ms·

Autogen: Enable next-gen large language model applications

163 points by infruset 3y ago

ravix 3y ago
The breakthrough I've had is realizing how important it is to control the conversation between agents. Just like in our work environments and in our relationships, HOW conversations occur largely determines the impact of the conversation. With or without AutoGen We're building a multi-agent postgres data analytics tool. If you're building agentic software, join the conversation: https://youtu.be/4o8tymMQ5GM https://youtu.be/4o8tymMQ5GM
MawKKe 3y ago
Is Microsoft chronically incapable of coming up with original names?
- Jwarder 3y ago
  It isn't too bad; naming stuff is always hard. If the Microsoft marketers knew about it then I would expect to see Azure™ Gen.NET™ Live™.
  - mnky9800n 3y ago
    Did they give up on ONE?
    - albert_e 3y ago
      >> https://learn.microsoft.com/en-us/training/modules/examine-components-of-modern-data-warehouse/5b-fabric https://learn.microsoft.com/en-us/training/modules/examine-c... OneLake is Fabric's lake-centric architecture that provides a single, integrated environment for data professionals and the business to collaborate on data projects. Think of it like OneDrive for data; OneLake combines storage locations across different regions and clouds into a single logical lake, without moving or duplicating data. Data can be stored in any file format in OneLake and can be structured or unstructured. For tabular data, the analytical engines in Fabric will write data in delta format when writing to OneLake. All engines will know how to read this format and treat delta files as tables no matter which engine writes it.
  - meiraleal 3y ago
    > I would expect to see Azure™ Gen.NET™ Live™ Give them 3 major releases.
  - ugh123 3y ago
    you forgot to add 360™
a_bonobo 3y ago
Are these 'safer' than using langchain-based agents that directly execute (arbitrary!) Python code? That was always my main issue with langchain
- andystep 3y ago
  i noticed autogen creates a new docker container each time code is executed by agents (default behaviour, can be turned off), so it's safe as you think docker is safe
  - webappguy 3y ago
    can you elaborate on your docker comment? Not familiar. Tnx
- deleted 3y ago
  [deleted]
h4kor 3y ago
A question for people researching LLMs and their capabilities: Is there any reason to believe that the interaction of multiple agents (using the same model) will yield some emergent property that is beyond the capabilities of the agent model? I'm not working with LLMs, but my intuition is that whatever these multi agent setups come up with could also be achieved by a single agent just talking to itself, as they all are "just guessing" what the most probable next token is.
- lelag 3y ago
  Since a single inference is limited by context length, a multiple agents model is able to process more context at each steps of the reasoning chain, which might improve the overall quality. However, given how easy it is getting to fine tune models, it's likely that multi-agent models will make a lot of sense to split the workload and assign each part to a specialized agent.
  - ca_tech 3y ago
    I think this is right inline with the utility of multi agent models. Whether distributing tasks to specialized agents trained on domain knowledge or collaborating with context aware agents. I think the context is where we are going to find limitations early on especially when models are expected to work on live data. Rather than constantly retraining a model, you leverage a model that is already primed through in-context learning based on previous interactions and relevant data.
  - wokwokwok 3y ago
    > a single inference is limited by context length, Yes. > multiple agents model is able to process more context at each steps of the reasoning chain What? How can a multi agent model have more context at a single step? The single step runs on a single agent. It would literally the same as a single agent? The multi agent approach is simply packaging up different “personas” for single steps; and yes, it is entirely reasonable to assume that given N configurations for an agent (different props, different temp, different models even) you would see emergent behaviour that a single agent wouldn’t. For example, you might have a “creative agent” to scaffold something and a “conservative” agent to fix syntax errors. …but what are you talking about with different context sizes? I think you’re mixing domain terms; context is the input to an LLM. I don’t know what you’re referring to, but multi agent setups make absolutely no difference to the context size.
m3kw9 3y ago
Having conversations amongst agents is it like treating each agent as your traditional nodes? Maybe in the future there would be millions of nodes(agents) conversing and maybe this is how next gen AGI will form
- lagrange77 3y ago
  next gen AGI?
- ShamelessC 3y ago
  Hot take.
gavi 3y ago
Anyone trying this - Please note the python package is called pyautogen
anais9 3y ago
Have been working with this and very impressed so far - it’s a step ahead of LangChain agents and seems to be receiving more attention/development than LangChain was interested in committing to agents. FWIW the “group research” and “chess” examples from the notebooks folder in their repo have been the best for explaining the utility of this tech to others - the meme generator does a good job showing functions stripped down but misses a lot of the important bits
- simonw 3y ago
  Here's that group research notebook: https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat_research.ipynb https://github.com/microsoft/autogen/blob/main/notebook/agen... And the chess one: https://github.com/microsoft/autogen/blob/main/notebook/agentchat_chess.ipynb https://github.com/microsoft/autogen/blob/main/notebook/agen...
SunghoYahng 3y ago
Unless I'm missing something, how is this library different from prompting a single chatbot: "Write a dialog in which A, B, and C, each playing a different role, have a conversation and do something D"?
- staticman2 3y ago
  Maybe it depends on the model but I find you'll get a different result if you say "write a dialog in which, A, B, and C talk about D" versus "read what A said and reply as B". The latter will result in each participant talking longer.
  - webappguy 3y ago
    Not sure talking longer is the goal. More so, the focus and separation of each facilitates a interplay and dynamic with which an attention window on a segmented linear response (be A, B and C) cannot be individually represent nearly as rhobustly (the main inference is the primary focus). Would love to hear some other opinions chime in here.
    - webappguy 3y ago
      additionally, each agent can be a model using its own RAG or training data as well
- leobg 3y ago
  You can have the character description more front and center, if that makes any sense. So instead of diluting attention across three separate character descriptions, your model will see just the chat log and the single description of the persona it should respond from. This may or may not make a difference.
deleted 3y ago
[deleted]
ugh123 3y ago
Matthew Berman has a good series on AutoGen with tutorials and demos: https://www.youtube.com/watch?v=10FCv-gCKug https://www.youtube.com/watch?v=10FCv-gCKug However from his examples (and his own admission) it seems that AutoGen isn't benefitting from full GPT4-level performance even tho he's pointed it directly at OpenAPI GPT4 (and other LLMs). The back and forth between the agents does not produce great results even tho similar prompts pumped directly into ChatGPT seem to give better results. Anyone know whats going on?
- Tostino 3y ago
  Temperature being set differently is one culprit. There are a few hyper parameters that can be tweaked to get some pretty different output.
  - ProofHouse 3y ago
    This is a top potential cause for sure. The variability can change drastically with temperature differences
zerop 3y ago
Use cases for multi agents?
- webappguy 3y ago
  software development, for 1. Nearly any use
digitcatphd 3y ago
It doesn’t help you inherently solve the problem per se, but what it does allow you to do that is distinctive is keep the human and the loop that can assist the agents to solve problems. To some degree it can also keep problems in the logic chain from snowballing, and causing the overall objective to fail because there’s invalid logic in the sequence
dang 3y ago
A bunch of single-comment related threads. Others? AutoGen: A Multi-Agent Framework for Streamlining Task Customization - https://news.ycombinator.com/item?id=37855314 https://news.ycombinator.com/item?id=37855314 - Oct 2023 (1 comment) Microsoft's AutoGen – Guide to code execution by LLMs - https://news.ycombinator.com/item?id=37822809 https://news.ycombinator.com/item?id=37822809 - Oct 2023 (1 comment) Making memes with Autogen AI (open source LLM agent framework) [video] - https://news.ycombinator.com/item?id=37750897 https://news.ycombinator.com/item?id=37750897 - Oct 2023 (1 comment) AutoGen: Enabling next-generation large language model applications - https://news.ycombinator.com/item?id=37647404 https://news.ycombinator.com/item?id=37647404 - Sept 2023 (1 comment) AutoGen: Enabling Next-Gen GPT-X Applications - https://news.ycombinator.com/item?id=37220686 https://news.ycombinator.com/item?id=37220686 - Aug 2023 (1 comment)
TaylorAlexander 3y ago
This just reminds me: I have been wondering, if you get multiple instances of GPT-4 talking to each other, each seeded with a different personality prompt, do they have interesting conversations? I suspect it would devolve in to nonsense quickly, but I’ve never seen any chat log of two GPT instances talking. Does anyone have a reference for this? Thanks.
- webappguy 3y ago
  easy enough to test, copy and paste the responses after initial prompt
  - TaylorAlexander 3y ago
    Right but someone with API access could do this much more easily. I don’t really want to sit there copying and pasting back and forth. I’d rather write two or three starting prompts and have a few agents do all the work of talking.
    - bigfudge 3y ago
      I can't find the script now, but my kids and I did this had the Queen Elizabeth 1st talking to a pirate about his life on the high seas. It was quite fun. I wouldn't want to read a historical novel written that way tho...
- MiSeRyDeee 3y ago
  Check https://github.com/OpenBMB/ChatDev https://github.com/OpenBMB/ChatDev out, they simulate personas in a company and build products by simulating the interactions.
- tuchsen 3y ago
  I did a DSL to facilitate this at https://prlang.com https://prlang.com. I've had some success setting up agents to "act" out scenes, where each plays a different part, but it was kinda limited in that conversations would kinda de cohere into nonsense after a bit.
- 3y ago
webappguy 3y ago
AutoGen is great, but have you heard of GeniA?

163 points by infruset 3y ago

The breakthrough I've had is realizing how important it is to control the conversation between agents. Just like in our work environments and in our relationships, HOW conversations occur largely determines the impact of the conversation. With or without AutoGen We're building a multi-agent postgres data analytics tool. If you're building agentic software, join the conversation: https://youtu.be/4o8tymMQ5GM https://youtu.be/4o8tymMQ5GM

MawKKe 3y ago

Is Microsoft chronically incapable of coming up with original names?

Jwarder 3y ago
It isn't too bad; naming stuff is always hard. If the Microsoft marketers knew about it then I would expect to see Azure™ Gen.NET™ Live™.
- mnky9800n 3y ago
  Did they give up on ONE?
  - albert_e 3y ago
    >> https://learn.microsoft.com/en-us/training/modules/examine-components-of-modern-data-warehouse/5b-fabric https://learn.microsoft.com/en-us/training/modules/examine-c... OneLake is Fabric's lake-centric architecture that provides a single, integrated environment for data professionals and the business to collaborate on data projects. Think of it like OneDrive for data; OneLake combines storage locations across different regions and clouds into a single logical lake, without moving or duplicating data. Data can be stored in any file format in OneLake and can be structured or unstructured. For tabular data, the analytical engines in Fabric will write data in delta format when writing to OneLake. All engines will know how to read this format and treat delta files as tables no matter which engine writes it.
- meiraleal 3y ago
  > I would expect to see Azure™ Gen.NET™ Live™ Give them 3 major releases.
- ugh123 3y ago
  you forgot to add 360™

a_bonobo 3y ago

Are these 'safer' than using langchain-based agents that directly execute (arbitrary!) Python code? That was always my main issue with langchain

andystep 3y ago
i noticed autogen creates a new docker container each time code is executed by agents (default behaviour, can be turned off), so it's safe as you think docker is safe
- webappguy 3y ago
  can you elaborate on your docker comment? Not familiar. Tnx
deleted 3y ago
[deleted]

h4kor 3y ago

A question for people researching LLMs and their capabilities: Is there any reason to believe that the interaction of multiple agents (using the same model) will yield some emergent property that is beyond the capabilities of the agent model? I'm not working with LLMs, but my intuition is that whatever these multi agent setups come up with could also be achieved by a single agent just talking to itself, as they all are "just guessing" what the most probable next token is.

lelag 3y ago
Since a single inference is limited by context length, a multiple agents model is able to process more context at each steps of the reasoning chain, which might improve the overall quality. However, given how easy it is getting to fine tune models, it's likely that multi-agent models will make a lot of sense to split the workload and assign each part to a specialized agent.
- ca_tech 3y ago
  I think this is right inline with the utility of multi agent models. Whether distributing tasks to specialized agents trained on domain knowledge or collaborating with context aware agents. I think the context is where we are going to find limitations early on especially when models are expected to work on live data. Rather than constantly retraining a model, you leverage a model that is already primed through in-context learning based on previous interactions and relevant data.
- wokwokwok 3y ago
  > a single inference is limited by context length, Yes. > multiple agents model is able to process more context at each steps of the reasoning chain What? How can a multi agent model have more context at a single step? The single step runs on a single agent. It would literally the same as a single agent? The multi agent approach is simply packaging up different “personas” for single steps; and yes, it is entirely reasonable to assume that given N configurations for an agent (different props, different temp, different models even) you would see emergent behaviour that a single agent wouldn’t. For example, you might have a “creative agent” to scaffold something and a “conservative” agent to fix syntax errors. …but what are you talking about with different context sizes? I think you’re mixing domain terms; context is the input to an LLM. I don’t know what you’re referring to, but multi agent setups make absolutely no difference to the context size.

m3kw9 3y ago

Having conversations amongst agents is it like treating each agent as your traditional nodes? Maybe in the future there would be millions of nodes(agents) conversing and maybe this is how next gen AGI will form

lagrange77 3y ago
next gen AGI?
ShamelessC 3y ago
Hot take.

gavi 3y ago

Anyone trying this - Please note the python package is called pyautogen

anais9 3y ago

Have been working with this and very impressed so far - it’s a step ahead of LangChain agents and seems to be receiving more attention/development than LangChain was interested in committing to agents. FWIW the “group research” and “chess” examples from the notebooks folder in their repo have been the best for explaining the utility of this tech to others - the meme generator does a good job showing functions stripped down but misses a lot of the important bits

simonw 3y ago
Here's that group research notebook: https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat_research.ipynb https://github.com/microsoft/autogen/blob/main/notebook/agen... And the chess one: https://github.com/microsoft/autogen/blob/main/notebook/agentchat_chess.ipynb https://github.com/microsoft/autogen/blob/main/notebook/agen...

SunghoYahng 3y ago

Unless I'm missing something, how is this library different from prompting a single chatbot: "Write a dialog in which A, B, and C, each playing a different role, have a conversation and do something D"?

staticman2 3y ago
Maybe it depends on the model but I find you'll get a different result if you say "write a dialog in which, A, B, and C talk about D" versus "read what A said and reply as B". The latter will result in each participant talking longer.
- webappguy 3y ago
  Not sure talking longer is the goal. More so, the focus and separation of each facilitates a interplay and dynamic with which an attention window on a segmented linear response (be A, B and C) cannot be individually represent nearly as rhobustly (the main inference is the primary focus). Would love to hear some other opinions chime in here.
  - webappguy 3y ago
    additionally, each agent can be a model using its own RAG or training data as well
leobg 3y ago
You can have the character description more front and center, if that makes any sense. So instead of diluting attention across three separate character descriptions, your model will see just the chat log and the single description of the persona it should respond from. This may or may not make a difference.

deleted 3y ago

[deleted]

ugh123 3y ago

Matthew Berman has a good series on AutoGen with tutorials and demos: https://www.youtube.com/watch?v=10FCv-gCKug https://www.youtube.com/watch?v=10FCv-gCKug However from his examples (and his own admission) it seems that AutoGen isn't benefitting from full GPT4-level performance even tho he's pointed it directly at OpenAPI GPT4 (and other LLMs). The back and forth between the agents does not produce great results even tho similar prompts pumped directly into ChatGPT seem to give better results. Anyone know whats going on?

Tostino 3y ago
Temperature being set differently is one culprit. There are a few hyper parameters that can be tweaked to get some pretty different output.
- ProofHouse 3y ago
  This is a top potential cause for sure. The variability can change drastically with temperature differences

zerop 3y ago

Use cases for multi agents?

webappguy 3y ago
software development, for 1. Nearly any use

digitcatphd 3y ago

It doesn’t help you inherently solve the problem per se, but what it does allow you to do that is distinctive is keep the human and the loop that can assist the agents to solve problems. To some degree it can also keep problems in the logic chain from snowballing, and causing the overall objective to fail because there’s invalid logic in the sequence

dang 3y ago

A bunch of single-comment related threads. Others? AutoGen: A Multi-Agent Framework for Streamlining Task Customization - https://news.ycombinator.com/item?id=37855314 https://news.ycombinator.com/item?id=37855314 - Oct 2023 (1 comment) Microsoft's AutoGen – Guide to code execution by LLMs - https://news.ycombinator.com/item?id=37822809 https://news.ycombinator.com/item?id=37822809 - Oct 2023 (1 comment) Making memes with Autogen AI (open source LLM agent framework) [video] - https://news.ycombinator.com/item?id=37750897 https://news.ycombinator.com/item?id=37750897 - Oct 2023 (1 comment) AutoGen: Enabling next-generation large language model applications - https://news.ycombinator.com/item?id=37647404 https://news.ycombinator.com/item?id=37647404 - Sept 2023 (1 comment) AutoGen: Enabling Next-Gen GPT-X Applications - https://news.ycombinator.com/item?id=37220686 https://news.ycombinator.com/item?id=37220686 - Aug 2023 (1 comment)

TaylorAlexander 3y ago

This just reminds me: I have been wondering, if you get multiple instances of GPT-4 talking to each other, each seeded with a different personality prompt, do they have interesting conversations? I suspect it would devolve in to nonsense quickly, but I’ve never seen any chat log of two GPT instances talking. Does anyone have a reference for this? Thanks.

webappguy 3y ago
easy enough to test, copy and paste the responses after initial prompt
- TaylorAlexander 3y ago
  Right but someone with API access could do this much more easily. I don’t really want to sit there copying and pasting back and forth. I’d rather write two or three starting prompts and have a few agents do all the work of talking.
  - bigfudge 3y ago
    I can't find the script now, but my kids and I did this had the Queen Elizabeth 1st talking to a pirate about his life on the high seas. It was quite fun. I wouldn't want to read a historical novel written that way tho...
MiSeRyDeee 3y ago
Check https://github.com/OpenBMB/ChatDev https://github.com/OpenBMB/ChatDev out, they simulate personas in a company and build products by simulating the interactions.
tuchsen 3y ago
I did a DSL to facilitate this at https://prlang.com https://prlang.com. I've had some success setting up agents to "act" out scenes, where each plays a different part, but it was kinda limited in that conversations would kinda de cohere into nonsense after a bit.
3y ago

webappguy 3y ago

AutoGen is great, but have you heard of GeniA?