4 ms·

Show HN: Python package for interfacing with ChatGPT with minimized complexity

129 points by minimaxir 3y ago

H8crilA 3y ago
Didn't find this in the README: does it handle network/OpenAI errors via retries? In particular the very common "model is overloaded at the moment". It's fine if not, I can imagine you want to keep it as simple as possible and outsource it to the library user.
- minimaxir 3y ago
  I was mixed about putting it in the base package but you can get the same behavior with the tenacity package.
  - H8crilA 3y ago
    Yeah, I understand. Maybe it would be good to add a comment on it in the README, since most people will hit this problem.
  - mikeravkine 3y ago
    Errors are so common with these endpoints, not having retry out of the box just makes it harder to hack on which I think is against the spirit?
    - Klathmon 3y ago
      On the other hand, automatic retires can be suprising, especially when they don't completely understand the returned error and retry on errors they shouldn't. I just wasted a couple hours the other week due to langchainjs defaulting to retrying errors which were ultimately caused by timeouts and would never complete.
_private 3y ago
Great work! I'd love to change this to use the new function logic in chatgpt
- H8crilA 3y ago
  The project has an implementation for the function logic: https://github.com/minimaxir/simpleaichat#functions https://github.com/minimaxir/simpleaichat#functions
nextworddev 3y ago
I hope you keep investing in this project- it’s a crowded field but I do think existing projects like Langchain and LLamaindex feel very bloated and beyond a refactor - sometimes we need fresh takes like this and start over.
- minimaxir 3y ago
  I am: the reason I made simpleaichat is because I have a need for more controllable work with AI generation, and it was easier to create a package from scratch than to figure out how to hack LangChain.
senko 3y ago
Nice! Totally agree with the project goals, it seems too many other packages are created by people who are researchers (or enthusiasts) first and software developers second, and it shows. I see you're using Pydantic. I've recently been playing with using pydantic to implement chatgpt functions, making it a bit easier to define functions (tools) with more control over the attributes, like this: class SearchWeb(pydantic.BaseModel): """ Docstring description to help GPT figure out what this does, like functions in your library. """ query: str = pydantic.Field(description="More info so GPT understands how to use this param") def handle(self): # my wrapper will call this to implement the tool after the arguments are parsed # at this point you can be sure self.query is correct and has passed any validation you might have It's definitely more verbose than the function definitions you have now, but you get schema definition for free, and is more strict about option parsing. It also makes it easy to throw errors back at GPT if it hallucinated some parameters incorrectly. ...aaaanyways, great work there, I'll be following the progress!
- senko 3y ago
  Ah, I see I've been ninja'd by about 17 hours: https://github.com/minimaxir/simpleaichat/releases/tag/v0.2.0 https://github.com/minimaxir/simpleaichat/releases/tag/v0.2.... :-D
  - minimaxir 3y ago
    I've spent a lot of time experimenting with schema: https://github.com/minimaxir/simpleaichat/blob/main/examples/notebooks/schema_ttrpg.ipynb https://github.com/minimaxir/simpleaichat/blob/main/examples...
- kordlessagain 3y ago
  There was another post today about using Pydantic for function enabled completions: https://github.com/jxnl/openai_function_call https://github.com/jxnl/openai_function_call I whipped up an example doing something similar last Friday using a decorator, inspect, ast and __doc__ usage: https://gist.github.com/kordless/7d306b0646bf0b56c44ebca2b8e96678 https://gist.github.com/kordless/7d306b0646bf0b56c44ebca2b8e.... The example pulls top results from Algoia's HN search and then chains them into another prompt for GPT-X. The blog post is here: https://www.featurebase.com/blog/function-integration-in-openais-completions https://www.featurebase.com/blog/function-integration-in-ope... Currently integrating this approach into PythonGPT[1], which will build a function on the fly, extract the method info, then call the code in exec(). I would label it "very dangerous"... [1] https://github.com/FeatureBaseDB/PythonGPT https://github.com/FeatureBaseDB/PythonGPT
gautiert 3y ago
Nice work! Is there some JavaScript/Typescript alternatives to Langchain?
- screye 3y ago
  Langchain ? - https://js.langchain.com/docs/ https://js.langchain.com/docs/
bravura 3y ago
Since this is async, can it automatically handle provided rate limits and batch queries appropriately? Seems like everyone has to roll their own on this and it’s much nicer to have smooth tooling for it.
- minimaxir 3y ago
  The underlying library for both sync and async is httpx (https://www.python-httpx.org/ https://www.python-httpx.org/) which may be limited from the HTTP Client perspective but it may be possible to add rate limiting at a Session level.
MuffinFlavored 3y ago
Is interfacting with the GPT-4 API acting as if you are the ChatGPT UI against terms of service? I was under the impression it was and they had a bunch of methods to protect against this. They want you using ChatGPT for $20/mo and then paying for GPT-4 API usage separately per token from what I can tell.
- minimaxir 3y ago
  You have to have an approved GPT-4 API key to use model="gpt-4". Nothing in this package circumvents it, which is what the free GPT-4 apps were doing.
  - MuffinFlavored 3y ago
    I don't have a GPT-4 API key, but I can use GPT-4 in the OpenAI official ChatGPT UI, right?
    - minimaxir 3y ago
      The ChatGPT UI is not the API.
      - MuffinFlavored 3y ago
        Does the ChatGPT UI use the GPT-4 API? Why can I use the GPT-4 API through the ChatGPT UI, but not any other way?
rane 3y ago
Are we anywhere close to being able to give a whole codebase to GPT-4 and ask various things about it?
- killthebuddha 3y ago
  This of course depends entirely on how big the codebase is. But the short answer is "yes".
  - smabie 3y ago
    Do you have any links / tools for this?
    - killthebuddha 3y ago
      Here's a couple (I apologize if some of these are already obvious to you): - https://python.langchain.com/docs/use_cases/code/code-analysis-deeplake - https://twitter.com/jerryjliu0/status/1636728577524916225?s=20 (GPT repo loader) FWIW there's tons of tools you could use to do this. GitHub Copilot is pretty close to being a comprehensive solution as well.
    - zora_goron 3y ago
      I wrote a simple ChatGPT plugin that exposes files within a local directory on my computer to ChatGPT [0]. It's been quite helpful for me, both for working with codebases as well as other material (supports docx and pdf as well). It does require plugin developer access to run, though. [0] https://github.com/samrawal/chatgpt-localfiles https://github.com/samrawal/chatgpt-localfiles
- anotherpaulg 3y ago
  My open source CLI tool can do this, as well as make changes to the code. Here's a chat transcript that shows how you can use it to explore and then modify a public github repo for a js game: https://aider.chat/examples/2048-game.html https://aider.chat/examples/2048-game.html Here a link to aider with more info: https://github.com/paul-gauthier/aider https://github.com/paul-gauthier/aider
jimmySixDOF 3y ago
Any chance your roadmap might include access to some local run open source models kind of like an oobabooga but with all your extra tooling baked in ?
m3kw9 3y ago
Why not just use ChatGPT website to chat if you dont want to handle any complexity.

129 points by minimaxir 3y ago

Didn't find this in the README: does it handle network/OpenAI errors via retries? In particular the very common "model is overloaded at the moment". It's fine if not, I can imagine you want to keep it as simple as possible and outsource it to the library user.

minimaxir 3y ago
I was mixed about putting it in the base package but you can get the same behavior with the tenacity package.
- H8crilA 3y ago
  Yeah, I understand. Maybe it would be good to add a comment on it in the README, since most people will hit this problem.
- mikeravkine 3y ago
  Errors are so common with these endpoints, not having retry out of the box just makes it harder to hack on which I think is against the spirit?
  - Klathmon 3y ago
    On the other hand, automatic retires can be suprising, especially when they don't completely understand the returned error and retry on errors they shouldn't. I just wasted a couple hours the other week due to langchainjs defaulting to retrying errors which were ultimately caused by timeouts and would never complete.

_private 3y ago

Great work! I'd love to change this to use the new function logic in chatgpt

H8crilA 3y ago
The project has an implementation for the function logic: https://github.com/minimaxir/simpleaichat#functions https://github.com/minimaxir/simpleaichat#functions

nextworddev 3y ago

I hope you keep investing in this project- it’s a crowded field but I do think existing projects like Langchain and LLamaindex feel very bloated and beyond a refactor - sometimes we need fresh takes like this and start over.

minimaxir 3y ago
I am: the reason I made simpleaichat is because I have a need for more controllable work with AI generation, and it was easier to create a package from scratch than to figure out how to hack LangChain.

senko 3y ago

Nice! Totally agree with the project goals, it seems too many other packages are created by people who are researchers (or enthusiasts) first and software developers second, and it shows. I see you're using Pydantic. I've recently been playing with using pydantic to implement chatgpt functions, making it a bit easier to define functions (tools) with more control over the attributes, like this: class SearchWeb(pydantic.BaseModel): """ Docstring description to help GPT figure out what this does, like functions in your library. """ query: str = pydantic.Field(description="More info so GPT understands how to use this param") def handle(self): # my wrapper will call this to implement the tool after the arguments are parsed # at this point you can be sure self.query is correct and has passed any validation you might have It's definitely more verbose than the function definitions you have now, but you get schema definition for free, and is more strict about option parsing. It also makes it easy to throw errors back at GPT if it hallucinated some parameters incorrectly. ...aaaanyways, great work there, I'll be following the progress!

senko 3y ago
Ah, I see I've been ninja'd by about 17 hours: https://github.com/minimaxir/simpleaichat/releases/tag/v0.2.0 https://github.com/minimaxir/simpleaichat/releases/tag/v0.2.... :-D
- minimaxir 3y ago
  I've spent a lot of time experimenting with schema: https://github.com/minimaxir/simpleaichat/blob/main/examples/notebooks/schema_ttrpg.ipynb https://github.com/minimaxir/simpleaichat/blob/main/examples...
kordlessagain 3y ago
There was another post today about using Pydantic for function enabled completions: https://github.com/jxnl/openai_function_call https://github.com/jxnl/openai_function_call I whipped up an example doing something similar last Friday using a decorator, inspect, ast and __doc__ usage: https://gist.github.com/kordless/7d306b0646bf0b56c44ebca2b8e96678 https://gist.github.com/kordless/7d306b0646bf0b56c44ebca2b8e.... The example pulls top results from Algoia's HN search and then chains them into another prompt for GPT-X. The blog post is here: https://www.featurebase.com/blog/function-integration-in-openais-completions https://www.featurebase.com/blog/function-integration-in-ope... Currently integrating this approach into PythonGPT[1], which will build a function on the fly, extract the method info, then call the code in exec(). I would label it "very dangerous"... [1] https://github.com/FeatureBaseDB/PythonGPT https://github.com/FeatureBaseDB/PythonGPT

gautiert 3y ago

Nice work! Is there some JavaScript/Typescript alternatives to Langchain?

screye 3y ago
Langchain ? - https://js.langchain.com/docs/ https://js.langchain.com/docs/

bravura 3y ago

Since this is async, can it automatically handle provided rate limits and batch queries appropriately? Seems like everyone has to roll their own on this and it’s much nicer to have smooth tooling for it.

minimaxir 3y ago
The underlying library for both sync and async is httpx (https://www.python-httpx.org/ https://www.python-httpx.org/) which may be limited from the HTTP Client perspective but it may be possible to add rate limiting at a Session level.

MuffinFlavored 3y ago

Is interfacting with the GPT-4 API acting as if you are the ChatGPT UI against terms of service? I was under the impression it was and they had a bunch of methods to protect against this. They want you using ChatGPT for $20/mo and then paying for GPT-4 API usage separately per token from what I can tell.

minimaxir 3y ago
You have to have an approved GPT-4 API key to use model="gpt-4". Nothing in this package circumvents it, which is what the free GPT-4 apps were doing.
- MuffinFlavored 3y ago
  I don't have a GPT-4 API key, but I can use GPT-4 in the OpenAI official ChatGPT UI, right?
  - minimaxir 3y ago
    The ChatGPT UI is not the API.
    - MuffinFlavored 3y ago
      Does the ChatGPT UI use the GPT-4 API? Why can I use the GPT-4 API through the ChatGPT UI, but not any other way?

rane 3y ago

Are we anywhere close to being able to give a whole codebase to GPT-4 and ask various things about it?

killthebuddha 3y ago
This of course depends entirely on how big the codebase is. But the short answer is "yes".
- smabie 3y ago
  Do you have any links / tools for this?
  - killthebuddha 3y ago
    Here's a couple (I apologize if some of these are already obvious to you): - https://python.langchain.com/docs/use_cases/code/code-analysis-deeplake - https://twitter.com/jerryjliu0/status/1636728577524916225?s=20 (GPT repo loader) FWIW there's tons of tools you could use to do this. GitHub Copilot is pretty close to being a comprehensive solution as well.
  - zora_goron 3y ago
    I wrote a simple ChatGPT plugin that exposes files within a local directory on my computer to ChatGPT [0]. It's been quite helpful for me, both for working with codebases as well as other material (supports docx and pdf as well). It does require plugin developer access to run, though. [0] https://github.com/samrawal/chatgpt-localfiles https://github.com/samrawal/chatgpt-localfiles
anotherpaulg 3y ago
My open source CLI tool can do this, as well as make changes to the code. Here's a chat transcript that shows how you can use it to explore and then modify a public github repo for a js game: https://aider.chat/examples/2048-game.html https://aider.chat/examples/2048-game.html Here a link to aider with more info: https://github.com/paul-gauthier/aider https://github.com/paul-gauthier/aider

jimmySixDOF 3y ago

Any chance your roadmap might include access to some local run open source models kind of like an oobabooga but with all your extra tooling baked in ?

m3kw9 3y ago

Why not just use ChatGPT website to chat if you dont want to handle any complexity.