5 ms·

I spent about two weeks having "morning meetings" with my AI life coach, which was essentially just a GPT-3 prompt that I continually tuned, and fed (summaries

I spent about two weeks having "morning meetings" with my AI life coach, which was essentially just a GPT-3 prompt that I continually tuned, and fed (summaries of) the previous days' conversations. There were major advantages but a few things missing. It was probably most useful as "rubber duck" technique. Forcing myself to articulate all of the things I needed/wanted to get done that day was itself extremely useful. Sometimes the agent would help me by identifying the highest-priority next action, but usually it was just recognizing what I thought was highest priority from implicit context. This can still be psychologically valuable, as a lot of procrastination can be caused by the logjam of not being sure which thing to focus on. The main missing ingredient, which caused me to ultimately stop the practice, was that it didn't really remember past conversations. I would feed past conversations to it and tell them to summarize the key points, then feed those summaries in as starting context, but this workflow was not sustainable. First of all, the summarization lost too much important nuance. Second and more importantly, even that summarization context block became larger than GPT-3's context window within a few days. This lack of persistent context destroyed the sense that I was talking to a real person, someone who could reliably recall information about a project that I last worked on 10 days ago and apply that context to the current conversation. I suspect we are not far away from both of these issues being mostly solved. The trend is obviously going in the direction of LLMs with different types of memory and/or much larger context windows.

mandliya 4y ago
I am curious if you tried something like langchain[1] it would solve the problem of not remembering previous conversations. In one of the examples people have used an entire company notion database on top of GPT-3 to answer questions specifically from that database. PS: not tried myself. [1] https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain
- swyx 4y ago
  if i understand correctly it doesnt really remember all previous conversations, it summarizes them to pack more into the limited context window right? thats kind of just kicking the can down the road vs solving the problem of actual long term memory
- Xeophon 4y ago
  Isn’t this the same as replying to ChatGPT in the same conversation?
  - frabcus 4y ago
    It more let’s the bot can query a database (using a DSL you teach it) when it wants to. So for example if it couldn’t remember an anecdote from the clients past, I guess it could search the history? eg ask to search for “Sarah" and then use all stories about her as part of a prompt and iteratively rerun itself. I think humans would still beat it at synthesising patterns drawn from disparate sessions. But it sounds doable to code…
hooande 4y ago
I'm working on something similar right now. best bet for something as open ended as therapy would be to use embeddings to compare your most recent post to past posts, and then use text snippets or a summary of similar posts as part of the prompt. granted you won't really be able to do this with ChatGPT, as you'd need to add another layer on top to do the similarity comparison and archiving of previous posts. but making an app to do this with the gpt api should be fairly straightforward.
moneywoes 4y ago
What prompts did you use?
- mordymoop 4y ago
  "This is an annotated transcript of a conversation featuring {made up AI life coach name}, one of the most sought-after life coaches of all time. {name} has advised and worked with luminaries such as {long list of people that I personally admire and think are successful}. {coach} has a Ph.D. in {list of accomplishments and roles that I would find personally useful in mentor}. The following transcript covers his pro bono work with {my name}, a promising student of {coach}'s who consented to make his life coaching sessions public. {coach} further supplied commentary on the reasoning behind his remarks." Sometimes during the interactions I would guide the model to create a "commentary" block from the coach, often coaxing it into putting a commentary block before the response. Something like, "(Commentary: mordymoop is still making progress on his goals, but his productivity is starting to slip. I'm going to help him get back on track in this session.)" This is my own way of implementing "let's think step by step" into a conversational context. The model is allowed to think out loud about the kinds of things that a wise mentor figure might say to me, and lay out its reasoning transparently, and then give a more well-thought-out reply.

mandliya 4y ago

I am curious if you tried something like langchain[1] it would solve the problem of not remembering previous conversations. In one of the examples people have used an entire company notion database on top of GPT-3 to answer questions specifically from that database. PS: not tried myself. [1] https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain

swyx 4y ago
if i understand correctly it doesnt really remember all previous conversations, it summarizes them to pack more into the limited context window right? thats kind of just kicking the can down the road vs solving the problem of actual long term memory
Xeophon 4y ago
Isn’t this the same as replying to ChatGPT in the same conversation?
- frabcus 4y ago
  It more let’s the bot can query a database (using a DSL you teach it) when it wants to. So for example if it couldn’t remember an anecdote from the clients past, I guess it could search the history? eg ask to search for “Sarah" and then use all stories about her as part of a prompt and iteratively rerun itself. I think humans would still beat it at synthesising patterns drawn from disparate sessions. But it sounds doable to code…

hooande 4y ago

I'm working on something similar right now. best bet for something as open ended as therapy would be to use embeddings to compare your most recent post to past posts, and then use text snippets or a summary of similar posts as part of the prompt. granted you won't really be able to do this with ChatGPT, as you'd need to add another layer on top to do the similarity comparison and archiving of previous posts. but making an app to do this with the gpt api should be fairly straightforward.

moneywoes 4y ago

What prompts did you use?

mordymoop 4y ago
"This is an annotated transcript of a conversation featuring {made up AI life coach name}, one of the most sought-after life coaches of all time. {name} has advised and worked with luminaries such as {long list of people that I personally admire and think are successful}. {coach} has a Ph.D. in {list of accomplishments and roles that I would find personally useful in mentor}. The following transcript covers his pro bono work with {my name}, a promising student of {coach}'s who consented to make his life coaching sessions public. {coach} further supplied commentary on the reasoning behind his remarks." Sometimes during the interactions I would guide the model to create a "commentary" block from the coach, often coaxing it into putting a commentary block before the response. Something like, "(Commentary: mordymoop is still making progress on his goals, but his productivity is starting to slip. I'm going to help him get back on track in this session.)" This is my own way of implementing "let's think step by step" into a conversational context. The model is allowed to think out loud about the kinds of things that a wise mentor figure might say to me, and lay out its reasoning transparently, and then give a more well-thought-out reply.