7 ms·

Build ChatGPT like chatbots on your website

130 points by nigamanth 4y ago

m3affan 4y ago
On that note, have there been any chatgpt like open-source projects, that are on the same or similar level?
- nadermx 4y ago
  I known of bloom ai, https://huggingface.co/bigscience/bloom https://huggingface.co/bigscience/bloom it has 1 billion more parameters. But it is a completion ai, not a query and response ai. I wonder if it can be tweaked
- bioemerl 4y ago
  The strongest one right now is a project called koboldai. However, instructing models like chat GPT are not open source yet, so it only runs stuff that writes books for you. The problem with running a chat GPT sized system at home is that you need like 15 graphics cards to do it, and very few people have the equipment to manage something like that. On a 24 gig graphics card you can run maybe 13 billion parameters, and stuff like chat GPT gets up above 200 billion. I'm trying to set up a server at home with a bunch of old Tesla 40 series cards which have 24 gigs of vram and cost $200 each. A server that is a 2U super micro GPU server can hold six cards. At the lovely power consumption of 1800 watts, about what your wall can deliver, you can hit about 96 gigs of VRAM for one to $2,000. If you really wanted to go crazy you can get the v100 card, which are about $1,000 each, with 32 gigs . Trouble is, Nvidia starting to switch all these cards to their fuck you regular old business practice of making the connectors owned by Nvidia and changing the connector you need every single generation, so it's getting harder and harder to get these cards on the used market.
  - hanniabu 4y ago
    > However, instructing models like chat GPT are not open source yet Closed source AI built by OpenAI, another oxymoron just like the Patriot Act and countless others meant to subvert public goodwill
    - ksniwmidjd 4y ago
      [dead]
- MilStdJunkie 4y ago
  Working in the defense industry, it's extremely difficult to see how we're going to capitalize on these systems. It's a damn shame, because 95% of our document requirements are very-nearly-boilerplate, a great application for these early AI systems. I know the image processing AI things are coming along pretty well, but in some ways that's an easier problem. The problem for us is multilevel stovepipes. The biggest and grandest is ITAR, which restricts the physical path that data can take. Recently there was a tweak in draft that allowed for the data to take a path outside the physical USA, with the guarantee that the endpoints are encrypted. Not generally implemented though. The second is what I would call datarest or Data Restrictions, which includes the whole data classification system of DoD, DoE, and others. If each model is only able to pull from its bucket, it's going to be a bad model. The third is the proprietary problem. Since there are very few organizations competing - often they arrange for one to be the sole "winner", the ultimate smoke filled backroom - they make frameworks that generally work for just a single org. XML in Lockheed is not XML for Boeing, but replace "XML" with "anything that's normally standards-based". That's another layer of stovepipes. DoD will have to provide the framework and big ass models for this stuff to work, but that's going to be a hell of a job, and will need serious political horsepower to keep it from being kidnapped by LockBoNorthRay.
- simonw 4y ago
  Open-Assistant - https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant - is an interesting open source project I found this morning. As others have pointed out, running a truly large language model like GPT-3 isn't (yet) feasible on your own hardware - you need a LOT of powerful GPUs racked up in order to run inference. https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals is a really interesting project here: it works a bit like bittorrent, allowing you to join a larger network of people who share time on their GPUs, enabling execution of models that can't fit on a single member's hardware.
antirez 4y ago
An alternative to using old school NLP, is to use GPT itself for the first pipeline as well, with a prompt like: I've the following resources with data. power_troubleshooting.txt contains information for customers that have issues powering on the device, (and so forth in the next lines... with other resources). This is the user question: ..., please reply with what is the resource I should access. Then you get the file and create a second prompt. Based on the following information: ..., answer this question: ... question ... A slower and more powerful way involves showing GPT different parts of potentially relevant text (for instance 3 each time) and ask it to score from 0 to 10 the level of usefulness of the resource in order to reply to the question. And make it select what resource to use. But this requires a lot of back and forth.
- jfkimmes 4y ago
  Just be aware that your pipeline prompt should not contain any secrets and you should expect that users will be able to subvert your pipeline prompt! I think the most popular name for these attacks is currently 'prompt injection'.
  - cma 4y ago
    It may also make binding commitments to your customers as your agent.
simonw 4y ago
The challenge with this kind of system is always the bit that figures out the most relevant text from the corpus to bake together into the prompt. It's interesting to see that this example takes the simplest approach possible, and it seems to provide pretty decent results: > Third, each word of the list cleaned up above is searched inside the information paragraph. When a word is found, the whole sentence that includes it is extracted. All the sentences found for each and all of the relevant words are put together into a paragraph that is then fed to GPT-3 for few-shot learning. This is stripping punctuation and stopwords and then doing a straight string match to find the relevant sentences to include in the prompt! A lot of people - myself included - have been trying semantic search using embeddings to solve this. I wrote about my approach here: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer... My version dumps in entire truncated blog entries, but from this piece I'm thinking that breaking down to much smaller snippets (maybe even at the sentence level) is worth investigating further.
- jerpint 4y ago
  I’ve been playing around with this and have gotten decent results simply using openAI embeddings. I’ll probably be making some kind of post showing results soon
  - simonw 4y ago
    Yeah I'm happy with the results I got from embeddings so far. The areas I want to explore there are: 1. What's the ideal size of text to embed? I'm doing whole blog entries right now but I'm confident I can get better results if I divide them up into smaller chunks first - I'm just not sure how best to do that. 2. There's a trick called Hypothetical Document Embeddings (HyDE) where you ask GPT-3 to invent an answer to the user's question, embed THAT fictional answer, then use that embedding to find relevant documents in your corpus. https://arxiv.org/abs/2212.10496 https://arxiv.org/abs/2212.10496
    - jerpint 4y ago
      Wow thanks for sharing that’s such a clever hack
deleted 4y ago
[deleted]
SmileyJames 4y ago
Chat with Cassandra: https://www.cbmdigital.co.uk/contact https://www.cbmdigital.co.uk/contact our friendly assistant
- SmileyJames 4y ago
  Glad people are having fun with this, Cassandra has told me about some interesting project ideas discussed with her. Thanks HN
bborn 4y ago
I'm doing a similar thing but with a web-based platform that lets you build chatbots (AI Agents) in your browser: https://agent-hq.io https://agent-hq.io
CSSer 4y ago
This seems terrifying for Google Search and similar products. If I can cram the majority of my static, rarely changing information and proverbial (not literally, of course) consciousness into a better search format (a model) than Google et al can, why should I bother building out the rest of my website or sharing that model with Google et al? This is especially true if it's conversational enough for most people to chat with it casually. It seems the obvious answer is that people still need to be able to find me and you can't easily backlink the contents of a model. Google can create an interface or standard for this à la bots talking to bots, but the compute cost is just fundamentally higher for everyone involved. Maybe it's worth it for the end-user's sake? Anyway, a search query can be shorter than the question(s) it's going to take to get that information out of a model too. And as for Google, OpenAI or similar scraping the entire internet and creating a model like ChatGPT, sure, that works now, but how are people going to feel about that now that the cat's out of the bag? It seems the knee-jerk reaction to this is to more highly scrutinize what you publicly make available for scraping, especially since I have no idea what level of accuracy a model like this is going to possess in terms of representing my information. As a closing example, I have a friend who runs one of the most popular NPM packages available. He doesn't billboard his name all over the project, but it's public information that can be discovered trivially by a human with a search engine for various reasons (on govt. websites no less). Essentially, he's a de facto, albeit shy, public figure. I asked ChatGPT various questions about the library and it nailed the answers. Next I asked ChatGPT various formulations of who wrote or maintains the project. It gave us a random, wildly incorrect first name and said no other public information is available about him. To be honest, I'm really ambivalent about this because of all sorts of different reasons centered around the above topics. It seems there's some tension here. For those of us willing to embrace this, we may want to maintain technical stewardship. However, those changes may fundamentally change the fabric of discoverability on the web. Please let me know if I'm misunderstanding the technology or you believe I'm jumping to any conclusions here. Thanks!
- moneywoes 4y ago
  The main reason I think would be discovery. How would users find your site otherwise?
- 4y ago
iamflimflam1 4y ago
If you want a simple command line chat bot, I made this simple example: https://github.com/atomic14/command_line_chatbot https://github.com/atomic14/command_line_chatbot
hamasho 4y ago
(Probably) related discussion. Natural language is the lazy user interface (2 days ago) https://news.ycombinator.com/item?id=34549378 https://news.ycombinator.com/item?id=34549378 Chatbot is often a useless, unintuitive UX for solving problems. We already know how most websites work, so it's easier to navigate to the intended resources with a few clicks rather than typing uncertain questions.
- jMyles 4y ago
  ...yet we seem to be coalescing on agreement that it is at least "a" (if not "the") lazy user interface. Lazy user interfaces are difficult to create and have value. That doesn't mean they supplant motivated user interfaces; they're for different purposes.
- IshKebab 4y ago
  You could say the same about search engines but for some reason they seem to be quite popular!
- joe_the_user 4y ago
  Yeah, the thing about my interactions with ChatGPT is that as it has apparently been tuned since release, it's output has come to more and more resemble a decent FAQ on whatever topic I'm asking about. It's useful - but it's only better if a given website lacks such a good faq or equivalent. Another thing to consider is even for the sites that have real humans standing by to chat, the chat can be useless when the company basically doesn't want to their low-level any more leeway than the site's normal forms/application allows. I mean, I could imagine kinds of AI chatbot that could be very useful - a bot that could talk someone through the process of medium-level auto repair. But this seems to be also something well beyond the ability of current systems.
butz 4y ago
How to approach building ChatGPT like chatbot for business application, to help users who do not like reading documentation? How much more precise the documentation should be, so chatbot would be actually useful? Is it possible to teach chatbot by having sessions with users who are experts of the application, so bot could gather required information from them?
- yreg 4y ago
  I'm skeptical of this, but one could also feed it support tickets and responses, slack conversations involving product support, etc.
- jrvarela56 4y ago
  Checkout this example: https://blog.langchain.dev/langchain-chat/ https://blog.langchain.dev/langchain-chat/ It's a chatbot based on the LangChain docs, a Python library for interacting with LLMs.
noduerme 4y ago
Maybe naively, I was hoping this would involve a way of hosting a specially trained model oneself. If this pre-feeding of a corpus needs to be done every time the bot is "launched", that seems like a lot of extra tokens to pay for. I question the practicality of getting people to input their own API keys, which seems to be the only purpose of the PHP wrapper. On the other hand, passing the costs on to people (the "intermediate solution"[1]) would only make sense if the value added by the several-shot training was really significant, e.g. a very large body of domain-specific knowledge. Which again becomes impractical to feed in at the start of every session. [1] https://towardsdatascience.com/custom-informed-gpt-3-models-for-your-website-with-very-simple-code-47134b25620b https://towardsdatascience.com/custom-informed-gpt-3-models-...
- kenjackson 4y ago
  This is exactly my question. I want to be able to give GPT a large body of domain specific information and have it use this information in the same way it uses the information it already has. I've tried creating a fine tuned divinci model, but it didn't work great, and honestly I'm not sure I really trained it right -- I've yet to someone give a good example of it. For example, I'd love to see someone do an example where they add in all the information from this past NFL football season and then have the bot be able to discuss this past season as well as any other searson.
  - james-revisoai 4y ago
    Yeah that's it exactly, and a major thing people run into. Finetuning won't add "knowledge", it's changing the bias likelihood going forward for each tokens generation(with context to tokens around it in your training dataset). Fundamentally it's building on top of the "associations" the internal mechanism of the model has learned at different parts of itself - attention/self-attention - during the original training. Finetuning changes things superficially on the outside of the box. It changes things in a way that do not alter the fundamentals, unless you literally abduct weights deliberately (some good posts on lesswrong about this where they remove concepts like fire/water by finding with SVD where they are). If you think of it like a building, training makes the entire building, and finetuning has no access to the ground floor/basement/foundations of the building to change those lower, important parts, except in terms of how the floors are presented as you go upwards. Some things link the lift will always be in the same place[fundamental orderings of words in general], but you can use a different floorplan[change the vocab distribution+etc by finetuning]. Can you try uploading that data in PDF format (or youtube videos) to Fragen.co.uk and let me know what you think? It should reasonably by able to discuss the current season well once you provide enough data (but it will have reduced ability to discuss previous seasons). That's a tool which uses a similar approach to OP but with some mechanics to share and order knowledge smartly according to the question (e.g. replacing "it" with the right nouns, bringing in predicate facts relevant to the question). The answers have checkmarks next to statements it is confident about, and you could reasonably expect performance like base GPT-3 on NFL football season of 2019 if you did this with sufficient data (aka. 1000+ pages/game reports). If you have a youtube video of one game, you should be able to test it quickly with that by asking things mentioned in the games video, and the answers should not be wrong. It will reject questions it can't answer well.
mshake2 4y ago
We need a transferable GPT, like how CNN models have been trained on basic shapes and patterns, and can then be fine-tuned to an application. A transferable GPT wouldn't know the entire internet's worth of knowledge, but it would know to predict generalized structures. Maybe those structures could have placeholders that could be filled with specific knowledge.
gr4yb34rd 4y ago
i am just happy to be back on the timeline where we type in what we want the computer to do rather than going on a widget hunt.
Princesingh 4y ago
[dead]

130 points by nigamanth 4y ago

m3affan 4y ago

On that note, have there been any chatgpt like open-source projects, that are on the same or similar level?

nadermx 4y ago
I known of bloom ai, https://huggingface.co/bigscience/bloom https://huggingface.co/bigscience/bloom it has 1 billion more parameters. But it is a completion ai, not a query and response ai. I wonder if it can be tweaked
bioemerl 4y ago
The strongest one right now is a project called koboldai. However, instructing models like chat GPT are not open source yet, so it only runs stuff that writes books for you. The problem with running a chat GPT sized system at home is that you need like 15 graphics cards to do it, and very few people have the equipment to manage something like that. On a 24 gig graphics card you can run maybe 13 billion parameters, and stuff like chat GPT gets up above 200 billion. I'm trying to set up a server at home with a bunch of old Tesla 40 series cards which have 24 gigs of vram and cost $200 each. A server that is a 2U super micro GPU server can hold six cards. At the lovely power consumption of 1800 watts, about what your wall can deliver, you can hit about 96 gigs of VRAM for one to $2,000. If you really wanted to go crazy you can get the v100 card, which are about $1,000 each, with 32 gigs . Trouble is, Nvidia starting to switch all these cards to their fuck you regular old business practice of making the connectors owned by Nvidia and changing the connector you need every single generation, so it's getting harder and harder to get these cards on the used market.
- hanniabu 4y ago
  > However, instructing models like chat GPT are not open source yet Closed source AI built by OpenAI, another oxymoron just like the Patriot Act and countless others meant to subvert public goodwill
  - ksniwmidjd 4y ago
    [dead]
MilStdJunkie 4y ago
Working in the defense industry, it's extremely difficult to see how we're going to capitalize on these systems. It's a damn shame, because 95% of our document requirements are very-nearly-boilerplate, a great application for these early AI systems. I know the image processing AI things are coming along pretty well, but in some ways that's an easier problem. The problem for us is multilevel stovepipes. The biggest and grandest is ITAR, which restricts the physical path that data can take. Recently there was a tweak in draft that allowed for the data to take a path outside the physical USA, with the guarantee that the endpoints are encrypted. Not generally implemented though. The second is what I would call datarest or Data Restrictions, which includes the whole data classification system of DoD, DoE, and others. If each model is only able to pull from its bucket, it's going to be a bad model. The third is the proprietary problem. Since there are very few organizations competing - often they arrange for one to be the sole "winner", the ultimate smoke filled backroom - they make frameworks that generally work for just a single org. XML in Lockheed is not XML for Boeing, but replace "XML" with "anything that's normally standards-based". That's another layer of stovepipes. DoD will have to provide the framework and big ass models for this stuff to work, but that's going to be a hell of a job, and will need serious political horsepower to keep it from being kidnapped by LockBoNorthRay.
simonw 4y ago
Open-Assistant - https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant - is an interesting open source project I found this morning. As others have pointed out, running a truly large language model like GPT-3 isn't (yet) feasible on your own hardware - you need a LOT of powerful GPUs racked up in order to run inference. https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals is a really interesting project here: it works a bit like bittorrent, allowing you to join a larger network of people who share time on their GPUs, enabling execution of models that can't fit on a single member's hardware.

antirez 4y ago

An alternative to using old school NLP, is to use GPT itself for the first pipeline as well, with a prompt like: I've the following resources with data. power_troubleshooting.txt contains information for customers that have issues powering on the device, (and so forth in the next lines... with other resources). This is the user question: ..., please reply with what is the resource I should access. Then you get the file and create a second prompt. Based on the following information: ..., answer this question: ... question ... A slower and more powerful way involves showing GPT different parts of potentially relevant text (for instance 3 each time) and ask it to score from 0 to 10 the level of usefulness of the resource in order to reply to the question. And make it select what resource to use. But this requires a lot of back and forth.

jfkimmes 4y ago
Just be aware that your pipeline prompt should not contain any secrets and you should expect that users will be able to subvert your pipeline prompt! I think the most popular name for these attacks is currently 'prompt injection'.
- cma 4y ago
  It may also make binding commitments to your customers as your agent.

simonw 4y ago

The challenge with this kind of system is always the bit that figures out the most relevant text from the corpus to bake together into the prompt. It's interesting to see that this example takes the simplest approach possible, and it seems to provide pretty decent results: > Third, each word of the list cleaned up above is searched inside the information paragraph. When a word is found, the whole sentence that includes it is extracted. All the sentences found for each and all of the relevant words are put together into a paragraph that is then fed to GPT-3 for few-shot learning. This is stripping punctuation and stopwords and then doing a straight string match to find the relevant sentences to include in the prompt! A lot of people - myself included - have been trying semantic search using embeddings to solve this. I wrote about my approach here: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer... My version dumps in entire truncated blog entries, but from this piece I'm thinking that breaking down to much smaller snippets (maybe even at the sentence level) is worth investigating further.

jerpint 4y ago
I’ve been playing around with this and have gotten decent results simply using openAI embeddings. I’ll probably be making some kind of post showing results soon
- simonw 4y ago
  Yeah I'm happy with the results I got from embeddings so far. The areas I want to explore there are: 1. What's the ideal size of text to embed? I'm doing whole blog entries right now but I'm confident I can get better results if I divide them up into smaller chunks first - I'm just not sure how best to do that. 2. There's a trick called Hypothetical Document Embeddings (HyDE) where you ask GPT-3 to invent an answer to the user's question, embed THAT fictional answer, then use that embedding to find relevant documents in your corpus. https://arxiv.org/abs/2212.10496 https://arxiv.org/abs/2212.10496
  - jerpint 4y ago
    Wow thanks for sharing that’s such a clever hack

deleted 4y ago

[deleted]

SmileyJames 4y ago

Chat with Cassandra: https://www.cbmdigital.co.uk/contact https://www.cbmdigital.co.uk/contact our friendly assistant

SmileyJames 4y ago
Glad people are having fun with this, Cassandra has told me about some interesting project ideas discussed with her. Thanks HN

bborn 4y ago

I'm doing a similar thing but with a web-based platform that lets you build chatbots (AI Agents) in your browser: https://agent-hq.io https://agent-hq.io

CSSer 4y ago

This seems terrifying for Google Search and similar products. If I can cram the majority of my static, rarely changing information and proverbial (not literally, of course) consciousness into a better search format (a model) than Google et al can, why should I bother building out the rest of my website or sharing that model with Google et al? This is especially true if it's conversational enough for most people to chat with it casually. It seems the obvious answer is that people still need to be able to find me and you can't easily backlink the contents of a model. Google can create an interface or standard for this à la bots talking to bots, but the compute cost is just fundamentally higher for everyone involved. Maybe it's worth it for the end-user's sake? Anyway, a search query can be shorter than the question(s) it's going to take to get that information out of a model too. And as for Google, OpenAI or similar scraping the entire internet and creating a model like ChatGPT, sure, that works now, but how are people going to feel about that now that the cat's out of the bag? It seems the knee-jerk reaction to this is to more highly scrutinize what you publicly make available for scraping, especially since I have no idea what level of accuracy a model like this is going to possess in terms of representing my information. As a closing example, I have a friend who runs one of the most popular NPM packages available. He doesn't billboard his name all over the project, but it's public information that can be discovered trivially by a human with a search engine for various reasons (on govt. websites no less). Essentially, he's a de facto, albeit shy, public figure. I asked ChatGPT various questions about the library and it nailed the answers. Next I asked ChatGPT various formulations of who wrote or maintains the project. It gave us a random, wildly incorrect first name and said no other public information is available about him. To be honest, I'm really ambivalent about this because of all sorts of different reasons centered around the above topics. It seems there's some tension here. For those of us willing to embrace this, we may want to maintain technical stewardship. However, those changes may fundamentally change the fabric of discoverability on the web. Please let me know if I'm misunderstanding the technology or you believe I'm jumping to any conclusions here. Thanks!

moneywoes 4y ago
The main reason I think would be discovery. How would users find your site otherwise?
4y ago

iamflimflam1 4y ago

If you want a simple command line chat bot, I made this simple example: https://github.com/atomic14/command_line_chatbot https://github.com/atomic14/command_line_chatbot

hamasho 4y ago

(Probably) related discussion. Natural language is the lazy user interface (2 days ago) https://news.ycombinator.com/item?id=34549378 https://news.ycombinator.com/item?id=34549378 Chatbot is often a useless, unintuitive UX for solving problems. We already know how most websites work, so it's easier to navigate to the intended resources with a few clicks rather than typing uncertain questions.

jMyles 4y ago
...yet we seem to be coalescing on agreement that it is at least "a" (if not "the") lazy user interface. Lazy user interfaces are difficult to create and have value. That doesn't mean they supplant motivated user interfaces; they're for different purposes.
IshKebab 4y ago
You could say the same about search engines but for some reason they seem to be quite popular!
joe_the_user 4y ago
Yeah, the thing about my interactions with ChatGPT is that as it has apparently been tuned since release, it's output has come to more and more resemble a decent FAQ on whatever topic I'm asking about. It's useful - but it's only better if a given website lacks such a good faq or equivalent. Another thing to consider is even for the sites that have real humans standing by to chat, the chat can be useless when the company basically doesn't want to their low-level any more leeway than the site's normal forms/application allows. I mean, I could imagine kinds of AI chatbot that could be very useful - a bot that could talk someone through the process of medium-level auto repair. But this seems to be also something well beyond the ability of current systems.

butz 4y ago

How to approach building ChatGPT like chatbot for business application, to help users who do not like reading documentation? How much more precise the documentation should be, so chatbot would be actually useful? Is it possible to teach chatbot by having sessions with users who are experts of the application, so bot could gather required information from them?

yreg 4y ago
I'm skeptical of this, but one could also feed it support tickets and responses, slack conversations involving product support, etc.
jrvarela56 4y ago
Checkout this example: https://blog.langchain.dev/langchain-chat/ https://blog.langchain.dev/langchain-chat/ It's a chatbot based on the LangChain docs, a Python library for interacting with LLMs.

noduerme 4y ago

Maybe naively, I was hoping this would involve a way of hosting a specially trained model oneself. If this pre-feeding of a corpus needs to be done every time the bot is "launched", that seems like a lot of extra tokens to pay for. I question the practicality of getting people to input their own API keys, which seems to be the only purpose of the PHP wrapper. On the other hand, passing the costs on to people (the "intermediate solution"[1]) would only make sense if the value added by the several-shot training was really significant, e.g. a very large body of domain-specific knowledge. Which again becomes impractical to feed in at the start of every session. [1] https://towardsdatascience.com/custom-informed-gpt-3-models-for-your-website-with-very-simple-code-47134b25620b https://towardsdatascience.com/custom-informed-gpt-3-models-...

kenjackson 4y ago
This is exactly my question. I want to be able to give GPT a large body of domain specific information and have it use this information in the same way it uses the information it already has. I've tried creating a fine tuned divinci model, but it didn't work great, and honestly I'm not sure I really trained it right -- I've yet to someone give a good example of it. For example, I'd love to see someone do an example where they add in all the information from this past NFL football season and then have the bot be able to discuss this past season as well as any other searson.
- james-revisoai 4y ago
  Yeah that's it exactly, and a major thing people run into. Finetuning won't add "knowledge", it's changing the bias likelihood going forward for each tokens generation(with context to tokens around it in your training dataset). Fundamentally it's building on top of the "associations" the internal mechanism of the model has learned at different parts of itself - attention/self-attention - during the original training. Finetuning changes things superficially on the outside of the box. It changes things in a way that do not alter the fundamentals, unless you literally abduct weights deliberately (some good posts on lesswrong about this where they remove concepts like fire/water by finding with SVD where they are). If you think of it like a building, training makes the entire building, and finetuning has no access to the ground floor/basement/foundations of the building to change those lower, important parts, except in terms of how the floors are presented as you go upwards. Some things link the lift will always be in the same place[fundamental orderings of words in general], but you can use a different floorplan[change the vocab distribution+etc by finetuning]. Can you try uploading that data in PDF format (or youtube videos) to Fragen.co.uk and let me know what you think? It should reasonably by able to discuss the current season well once you provide enough data (but it will have reduced ability to discuss previous seasons). That's a tool which uses a similar approach to OP but with some mechanics to share and order knowledge smartly according to the question (e.g. replacing "it" with the right nouns, bringing in predicate facts relevant to the question). The answers have checkmarks next to statements it is confident about, and you could reasonably expect performance like base GPT-3 on NFL football season of 2019 if you did this with sufficient data (aka. 1000+ pages/game reports). If you have a youtube video of one game, you should be able to test it quickly with that by asking things mentioned in the games video, and the answers should not be wrong. It will reject questions it can't answer well.

mshake2 4y ago

We need a transferable GPT, like how CNN models have been trained on basic shapes and patterns, and can then be fine-tuned to an application. A transferable GPT wouldn't know the entire internet's worth of knowledge, but it would know to predict generalized structures. Maybe those structures could have placeholders that could be filled with specific knowledge.

gr4yb34rd 4y ago

i am just happy to be back on the timeline where we type in what we want the computer to do rather than going on a widget hunt.

Princesingh 4y ago

[dead]