3 ms·
In my (few hours of) testing, Auto-GPT was quite unreliable. If you'll pardon the expression, it suffers from severe ADHD: procrastinates, overthinks, gets dist
by andai 4y ago
In my (few hours of) testing, Auto-GPT was quite unreliable. If you'll pardon the expression, it suffers from severe ADHD: procrastinates, overthinks, gets distracted.
I think this is due to the main loop being a GPT feedback loop. Each loop it has a small chance of something going wrong (or a large chance, depending on the query), so as it loops repeatedly, the chance of failure approaches 100%.
My idea was to replace the core loop, instead of a GPT feedback loop just make it a few lines of Python.
Now the thing actually does what it says it's going to do, "thinking lag" is eliminated, and API usage is reduced 80%.
I turned (parts of) Auto-GPT into a tiny Python library, specialized for internet research.
GPT-3 and GPT-4 are able to use this library to write Python programs that do useful work.
This way they can "crystallize" their plans in code, to ensure that they will run.
Here is the interface:
def search(query, max_results=8) -> List[Dict]: pass # uses duckduckgo
def load(url) -> str | None: pass # uses requests and beautifulsoup
def summarize(text, task) -> str | None: pass # uses gpt-3
def save(filename, text) -> bool: pass
See the comments below the gist for the GPT-3 and 4 versions of main.py (20-30 lines for an internet research agent!).
https://gist.github.com/avelican/2d4e718954593e3df9e0e5ee6751f470 https://gist.github.com/avelican/2d4e718954593e3df9e0e5ee675...
Note: It's currently optimized for my main use-case, which is internet research. So it's not an Auto-GPT in any sense. But it does one thing, and does it fairly well.
P.S. The Holy Grail would be a system where the user enters a query, and the system translates it into Python on top of Auto-GPT's library, and runs that. I haven't tried that yet, and I'm a little afraid to...
- andai 4y agoIn using it, I found that my summaries / queries would often result in "The text did not include (thing you asked for)." because the 2nd half of the page text would be unrelated to the main content (and the text is split into small chunks for processing to fit in the context window). My solution to this (not yet integrated) is to use a faster / cheaper model to process all the text first and see if it's actually relevant or not, before running the summarization / task prompt with the main model. I realized that this is essentially a semantic search engine for text. Using the same principle you can feed a very large text file in, ask it a question, and it finds the page that answers that question. This would be useful as a layer "beneath" the internet research agent, that it could use to sort through all the noise and answer the question. https://gist.github.com/avelican/58958a2cf2b7e9f9f555ab94549dfe2f https://gist.github.com/avelican/58958a2cf2b7e9f9f555ab94549... It gives a lot of false positives, I'm not sure if this is a limitation of GPT-3 (have not tried GPT-4 for this yet, a bit too expensive for searching books) or a limitation of my implementation. It's probably slower and more expensive than just using a vector db, I haven't tried those yet.
- alreadydone 4y agoYou could give this a try: https://www.aomni.com/ https://www.aomni.com/
- andai 4y agoUpdate: Using the embeddings API is an order of magnitude cheaper for filtering text.