5 ms·

>a personal AI assistant to act as a second brain, and help me prioritize and remember things with human intuition Everyone wants this, but this is not the pro

by ratg13 4y ago

>a personal AI assistant to act as a second brain, and help me prioritize and remember things with human intuition Everyone wants this, but this is not the product. The current AI offerings are information in --> information out. It is not meant to keep state long term, and it is not meant to be your friend. It is meant to answer questions with the information available to it. You can even see in the example screenshot they showcase the fact that it is not designed to be asked follow-up questions.

JanSt 4y ago
You can use Langchain / Plugins to get around that issue
hn_throwaway_99 4y ago
That's just the case for this bot, but is obviously not the case for ChatGPT. I have a long running conversation with ChatGPT that I use to keep track of a verbal to-do list. I tell it my items with categories (e.g. work, personal, etc.) and estimated times, and then it outputs my complete task list, grouped by category. I then just tell it when I add tasks or complete tasks, and it continually keeps track of and outputs my current outstanding task list. I've been using this for weeks now, and since it's all in a single conversation ChatGPT can keep track of the entire state over time. I don't have access to plugins yet but it would be trivial to implement a personal AI assistant with ChatGPT if it could, for example, look up flight times and prices.
- ravenstine 4y ago
  Oh dang, that sounds awesome. So a ChatGPT conversation doesn't have a historical limit? I guess I assumed it would start having to forget things at a certain point.
  - hesdeadjim 4y ago
    As far as I’m aware it will definitely forget things as the history grows. You can “remind” it about things to keep them relevant, and I think a cool product built like OP’s project would take that into consideration for long running tasks.
  - zwily 4y ago
    Yes, there’s a limit. Right now it’s 4000 tokens (gpt4 has a 32k model but i don’t think it’s available yet, at least with chatgpt). Once you near the limit, chatgpt will start dropping previous messages to stay under the limit. I don’t know what their algorithm is for deciding what to drop. It could be as simple as dropping the oldest stuff. Or maybe taking a long message and replacing it with a summary. But at a certain point, the conversation becomes “lossy”.
  - rolisz 4y ago
    If it periodically outputs the latest state of the todo list, it won't have the problem of forgetting things from a long time ago.
    - hn_throwaway_99 4y ago
      FWIW that's exactly what it does (updates the current state of the whole todo list each time). After two weeks of use I've never had it make a mistake in keeping track of my total list of tasks. By default (I mean I didn't ask it to do this) it outputs the total estimated time for each category of tasks, and that summation has been wrong (which wasn't that surprising to me, as "LLMs aren't great at math" is a known issue), but even then I just tell it "Can you double check the category totals?" and it fixes them.
  - haukurb 4y ago
    They might make it summarize intermittently behind the scenes when the context grows too large. Which would be how it has a gist of the stuff that was mentioned way earlier even thought it doesn't know the specifics.
- m3kw9 4y ago
  And what happens when your window runs out? It prunes the oldest convo which you will need to keep track. Likely it could be fine, but if there was a task from a long time ago, that could get wiped out
  - pixl97 4y ago
    I'm assuming at this point you'd need a means of pruning expired information from your list, and then importing that shorter list into a new session. I guess the question is, can we get ChatGPT to make that new list reliably?
  - aqme28 4y ago
    You could store whatever personal data you want in embeddings and use the api to refer to those. Can't do it on vanilla ChatGPT with the website though, I don't think.
    - majormajor 4y ago
      Am I missing something or is that the opposite of how embeddings work? https://platform.openai.com/docs/guides/chat https://platform.openai.com/docs/guides/chat - this API endpoint for completions doesn't take embeddings, just messages. Their API docs for embeddings also don't talk about using them to get outside of the context size limit; instead, the way I've used it and seen others is "create embeddings from documents to enable fast search for relevant documents to populate in context" which still requires a separate data store.
  - hn_throwaway_99 4y ago
    Every time I add or complete a task, ChatGPT outputs the latest complete copy of my task list (I only had to ask it to do this the first time, then it just did that automatically). So, in other words, it always updates the latest "live state" in its most recent message. If I hit the limit on context window it would just take me ~30 seconds to open up a new window with my original prompt and the latest iteration of my task list if I so desired. Here's where I posted a snippet of this convo a couple weeks ago: https://news.ycombinator.com/item?id=35390644 https://news.ycombinator.com/item?id=35390644
- joseph_grobbles 4y ago
  [dead]
- ratg13 4y ago
  ChatGPT can keep some state for you, but there is a limit to the amount of tokens you can keep going in an instance. It’s enough to keep a todo list going, it’s not enough to make it your friend / coworker If you built what you were describing right now, either the flight questions would push out your todo list, or you would need to build something to keep state yourself.

by ratg13 4y ago

JanSt 4y ago

You can use Langchain / Plugins to get around that issue

hn_throwaway_99 4y ago

That's just the case for this bot, but is obviously not the case for ChatGPT. I have a long running conversation with ChatGPT that I use to keep track of a verbal to-do list. I tell it my items with categories (e.g. work, personal, etc.) and estimated times, and then it outputs my complete task list, grouped by category. I then just tell it when I add tasks or complete tasks, and it continually keeps track of and outputs my current outstanding task list. I've been using this for weeks now, and since it's all in a single conversation ChatGPT can keep track of the entire state over time. I don't have access to plugins yet but it would be trivial to implement a personal AI assistant with ChatGPT if it could, for example, look up flight times and prices.

ravenstine 4y ago
Oh dang, that sounds awesome. So a ChatGPT conversation doesn't have a historical limit? I guess I assumed it would start having to forget things at a certain point.
- hesdeadjim 4y ago
  As far as I’m aware it will definitely forget things as the history grows. You can “remind” it about things to keep them relevant, and I think a cool product built like OP’s project would take that into consideration for long running tasks.
- zwily 4y ago
  Yes, there’s a limit. Right now it’s 4000 tokens (gpt4 has a 32k model but i don’t think it’s available yet, at least with chatgpt). Once you near the limit, chatgpt will start dropping previous messages to stay under the limit. I don’t know what their algorithm is for deciding what to drop. It could be as simple as dropping the oldest stuff. Or maybe taking a long message and replacing it with a summary. But at a certain point, the conversation becomes “lossy”.
- rolisz 4y ago
  If it periodically outputs the latest state of the todo list, it won't have the problem of forgetting things from a long time ago.
  - hn_throwaway_99 4y ago
    FWIW that's exactly what it does (updates the current state of the whole todo list each time). After two weeks of use I've never had it make a mistake in keeping track of my total list of tasks. By default (I mean I didn't ask it to do this) it outputs the total estimated time for each category of tasks, and that summation has been wrong (which wasn't that surprising to me, as "LLMs aren't great at math" is a known issue), but even then I just tell it "Can you double check the category totals?" and it fixes them.
- haukurb 4y ago
  They might make it summarize intermittently behind the scenes when the context grows too large. Which would be how it has a gist of the stuff that was mentioned way earlier even thought it doesn't know the specifics.
m3kw9 4y ago
And what happens when your window runs out? It prunes the oldest convo which you will need to keep track. Likely it could be fine, but if there was a task from a long time ago, that could get wiped out
- pixl97 4y ago
  I'm assuming at this point you'd need a means of pruning expired information from your list, and then importing that shorter list into a new session. I guess the question is, can we get ChatGPT to make that new list reliably?
- aqme28 4y ago
  You could store whatever personal data you want in embeddings and use the api to refer to those. Can't do it on vanilla ChatGPT with the website though, I don't think.
  - majormajor 4y ago
    Am I missing something or is that the opposite of how embeddings work? https://platform.openai.com/docs/guides/chat https://platform.openai.com/docs/guides/chat - this API endpoint for completions doesn't take embeddings, just messages. Their API docs for embeddings also don't talk about using them to get outside of the context size limit; instead, the way I've used it and seen others is "create embeddings from documents to enable fast search for relevant documents to populate in context" which still requires a separate data store.
- hn_throwaway_99 4y ago
  Every time I add or complete a task, ChatGPT outputs the latest complete copy of my task list (I only had to ask it to do this the first time, then it just did that automatically). So, in other words, it always updates the latest "live state" in its most recent message. If I hit the limit on context window it would just take me ~30 seconds to open up a new window with my original prompt and the latest iteration of my task list if I so desired. Here's where I posted a snippet of this convo a couple weeks ago: https://news.ycombinator.com/item?id=35390644 https://news.ycombinator.com/item?id=35390644
joseph_grobbles 4y ago
[dead]
ratg13 4y ago
ChatGPT can keep some state for you, but there is a limit to the amount of tokens you can keep going in an instance. It’s enough to keep a todo list going, it’s not enough to make it your friend / coworker If you built what you were describing right now, either the flight questions would push out your todo list, or you would need to build something to keep state yourself.