4 ms·

Was thinking of building something very similar - upload your technical docs - and create a chat bot help system. One of the tricky things to overcome is the l

Was thinking of building something very similar - upload your technical docs - and create a chat bot help system. One of the tricky things to overcome is the length limits of the prompt you can feed into things like GPT-3. There are some suggestions on the OpenAI website on how to overcome this. The main one seems to be to filter the content using embeddings and then only feed "relevant" sections into the prompt. Would be interested to know what your approach is.

chandan_maruthi 4y ago
I was able to break down the content into smaller prompts/completions. And remove some recurring and low-information data. Your suggestion of using embeddings to identify and feed relevant completions is pretty good. I'll try that soon.
- iamflimflam1 4y ago
  Here’s the example notebook that shows how to do this. https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb https://github.com/openai/openai-cookbook/blob/main/examples...
- NicoJuicy 4y ago
  What's the main difference between using https://trainmy.ai/ https://trainmy.ai/ vs. with your app. Since I'm planning to use that for where i work, for the same use-case.
  - iamflimflam1 4y ago
    From what I was thinking of bulding: Packaging it up as a nice widget that can be embedded on your site easily. So you just upload your docs and have a snippet of JavaScript for your website that gives you a help bot.
    - chandan_maruthi 4y ago
      Agree, a snippet that you add to the application that trains every day and gets better. I am thinking we can ingest data every day/week as documentation updates.
      - iamflimflam1 4y ago
        Would be good to have a chat - sounds like we are thinking along similar lines. My email address is in my profile.
  - chandan_maruthi 4y ago
    Hi, I took a quick look at trainmyai. I would say we are focused on this specific usecase. Making it easy for users to interact with help and support docs and knowledge. We will keep improving and getting better in this specific use case. You get custom/fine-tuned models with Twig that only you have access to. You also get to train on private/behind-login information that is available to only your users. Think of it as use-case-specific solutions vs generic solution.
amrb 4y ago
I've been working on a hobby project to make all the site, videos and conversations I've seen searchable (aka external memory). While building it I used this to prepare the data and get around the prompt limits https://github.com/jerryjliu/gpt_index https://github.com/jerryjliu/gpt_index
- visarga 4y ago
  Is there a list of GPT+search projects? I've been seeing them popping up more and more.
  - saliagato 4y ago
    this is where the future is
  - prashantsengar 4y ago
    Someone can create a chat bot which is fed HN comments and it tells you which GPT+search bot to use for your usecase
- mritchie712 4y ago
  > https://github.com/jerryjliu/gpt_index https://github.com/jerryjliu/gpt_index Could you share anything (e.g. how many rows of data and tokens in each row) around how much it cost you to use GPT Index? It looks interesting, but it seems it'd be expensive.
  - the_real_sparky 4y ago
    Looks like it’s making extensive use of embeddings. The embeddings and vector comparisons are dirt cheap. The summarizing / answering functionality when the LLMs get involved is going to be the expensive part.
cm2187 4y ago
Wouldn't your energy be better spent doing a better hierarchisation of your doc so users can navigate to the authoritative anwser easily and intuitively? Rather than relying on some black box commands to get the information that should be what people look for but maybe not?
- the_real_sparky 4y ago
  I have been responsible for maintaining many thousands of pages of documentation with a dozen+ staff, and can say that at least for my industry there is often no hierarchy that fits every use case. Then consider that your documentation references dozens or hundreds of other pieces of external industry documentation, and now you have a real mess. That’s before we even get into the ERP system and all the relevant material specifications and such that’s stored away in there. I do think LLMs will eventually be the solution to this long-standing problem. Even if you could ask questions and it would do a general summary for it and then point you in the right directions for more details we’d be way ahead.

chandan_maruthi 4y ago

I was able to break down the content into smaller prompts/completions. And remove some recurring and low-information data. Your suggestion of using embeddings to identify and feed relevant completions is pretty good. I'll try that soon.

iamflimflam1 4y ago
Here’s the example notebook that shows how to do this. https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb https://github.com/openai/openai-cookbook/blob/main/examples...
NicoJuicy 4y ago
What's the main difference between using https://trainmy.ai/ https://trainmy.ai/ vs. with your app. Since I'm planning to use that for where i work, for the same use-case.
- iamflimflam1 4y ago
  From what I was thinking of bulding: Packaging it up as a nice widget that can be embedded on your site easily. So you just upload your docs and have a snippet of JavaScript for your website that gives you a help bot.
  - chandan_maruthi 4y ago
    Agree, a snippet that you add to the application that trains every day and gets better. I am thinking we can ingest data every day/week as documentation updates.
    - iamflimflam1 4y ago
      Would be good to have a chat - sounds like we are thinking along similar lines. My email address is in my profile.
- chandan_maruthi 4y ago
  Hi, I took a quick look at trainmyai. I would say we are focused on this specific usecase. Making it easy for users to interact with help and support docs and knowledge. We will keep improving and getting better in this specific use case. You get custom/fine-tuned models with Twig that only you have access to. You also get to train on private/behind-login information that is available to only your users. Think of it as use-case-specific solutions vs generic solution.

amrb 4y ago

I've been working on a hobby project to make all the site, videos and conversations I've seen searchable (aka external memory). While building it I used this to prepare the data and get around the prompt limits https://github.com/jerryjliu/gpt_index https://github.com/jerryjliu/gpt_index

visarga 4y ago
Is there a list of GPT+search projects? I've been seeing them popping up more and more.
- saliagato 4y ago
  this is where the future is
- prashantsengar 4y ago
  Someone can create a chat bot which is fed HN comments and it tells you which GPT+search bot to use for your usecase
mritchie712 4y ago
> https://github.com/jerryjliu/gpt_index https://github.com/jerryjliu/gpt_index Could you share anything (e.g. how many rows of data and tokens in each row) around how much it cost you to use GPT Index? It looks interesting, but it seems it'd be expensive.
- the_real_sparky 4y ago
  Looks like it’s making extensive use of embeddings. The embeddings and vector comparisons are dirt cheap. The summarizing / answering functionality when the LLMs get involved is going to be the expensive part.

cm2187 4y ago

Wouldn't your energy be better spent doing a better hierarchisation of your doc so users can navigate to the authoritative anwser easily and intuitively? Rather than relying on some black box commands to get the information that should be what people look for but maybe not?

the_real_sparky 4y ago
I have been responsible for maintaining many thousands of pages of documentation with a dozen+ staff, and can say that at least for my industry there is often no hierarchy that fits every use case. Then consider that your documentation references dozens or hundreds of other pieces of external industry documentation, and now you have a real mess. That’s before we even get into the ERP system and all the relevant material specifications and such that’s stored away in there. I do think LLMs will eventually be the solution to this long-standing problem. Even if you could ask questions and it would do a general summary for it and then point you in the right directions for more details we’d be way ahead.