3 ms·

The links aren't loading for me, but there is a difference between the output when the AI is trained vs having it google something for you, no? Having the abili

The links aren't loading for me, but there is a difference between the output when the AI is trained vs having it google something for you, no? Having the ability to google something for you vs just making up an answer or being unable to answer is definitely a step in the right direction, but isn't the same as the AI incorporating the new data into it's model on an ongoing basis as a way of continuous improvement.

scarface_74 2y ago
I don’t know why the links aren’t working correctly. The idea is that if an LLM can now search the web and analyze data, it will be more up to date than training it. Another unrelated improvement with the newer versions of ChatGPT is that while LLMs are notoriously bad at math, they are pretty good at writing Python. ChatGPT can write Python code to solve a problem, run it and give you an answer based on the code. You can also verify the code yourself. I used ChatGPT to model financial scenarios and the answers were correct in 4x. I verified the generated code it was using. But here is what it answered when I asked ChatGPT “What’s going on in Atlanta this weekend”? > This weekend in Atlanta, there are several exciting events happening: 1. Lakewood Antiques Market (May 31 - June 2): A large market in Cumming featuring antiques and collectibles . 2. Bark Meow-Ket (June 1): A craft and pet vendor market at The Lawn at Uptown Atlanta, with food and pet adoptions . (and a few other events) But a better example and where I forced it to look on the web is “Compare Amex’s current transfer partners to Chase’s partners” For context, when you earn credit card points with Amex and Chase, you can transfer them to airline and hotel loyalty programs. It searches the web (and provides the links it used to find the information) and then answers with: ——- ### American Express (Amex) Transfer Partners Amex has a robust list of transfer partners, spanning both airlines and hotels. As of 2024, Amex Membership Rewards points can be transferred to the following partners: #### Airlines: 1. Aer Lingus 2. Aeromexico … (A list of the airlines) #### Hotels: 1. Choice Privileges 2. Hilton Honors 3. Marriott Bonvoy ### Chase Transfer Partners Chase Ultimate Rewards points can be transferred to a variety of airline and hotel partners. Here are the current partners for 2024: #### Airlines: 1. Aer Lingus (AerClub) 2. Air Canada (Aeroplan) (a list of airlines) #### Hotels: 1. Hyatt (World of Hyatt) 2. IHG (IHG One Rewards) 3. Marriott Bonvoy ### Comparison Both Amex and Chase offer a strong selection of transfer partners, but there are some differences: - *Unique to Amex*: ANA, Avianca, Cathay Pacific, Delta, Etihad, Hawaiian, and Qatar. - *Unique to Chase*: Southwest Airlines, United Airlines, and Hyatt. - *Common Partners*: Aer Lingus, Air Canada, British Airways, Emirates, Flying Blue (Air France/KLM), Iberia Plus, JetBlue, Singapore Airlines, Virgin Atlantic, and Marriott Bonvoy. Amex tends to have a larger selection of airline partners, especially those in the SkyTeam and oneworld alliances. Chase, on the other hand, includes Southwest and United, which are popular with domestic travelers in the U.S., and Hyatt, which is highly regarded among hotel loyalty programs (a list of web citations)
skissane 2y ago
Idea I had: given user question, run model twice – once with web search assistance, once without. Then, continuously fine-tune the model in the background on every question (or a random sample of them) to prefer the web-search-augmented response to the non-augmented response. Maybe update the model every day (or even once every few hours) based on that fine-tuning. I wonder how that idea would perform in practice.
- scarface_74 2y ago
  Why though? In practice, humans don’t try to remember everything when they can use the internet and search for it.
  - skissane 2y ago
    For questions about current events it could be very helpful. I mean, suppose {Biden|Trump} wins the 2024 US Presidential election. You want the model to know who the winner is the next day, as part of its background knowledge, not wait 6 months for a new version to be released whose background knowledge includes that fact. It is right that humans don’t try to remember everything, but if one only did this for a random sample of conversations, you wouldn’t remember everything - you’d probably remember key developments because they’d come up a lot (in many questions hence high probability of being sampled). You’d likely also remember some obscure trivia just because it got sampled (just like humans do too) Another option would be to have the model evaluate each conversation and have it decide whether to include it in the fine-tune (e.g. did the web search reveal important new developments the model didn’t know about?) One could also do some kind of clustering to find common gaps in the model’s background knowledge and use that to guide sampling for fine-tuning
    - scarface_74 2y ago
      What I am saying that today with ChatGPT 4x, if it detects that its model doesn’t have the current information, it will search on the web and summarize the results from different websites. One example I have was “what’s going on in Atlanta over the next two weeks”. I had another example I just posted where I asked it “who are Amex’s and Chase’s current transfer partners and which ones do they have in common”. It searched the web and summarized the information like you would expect.

scarface_74 2y ago

I don’t know why the links aren’t working correctly. The idea is that if an LLM can now search the web and analyze data, it will be more up to date than training it. Another unrelated improvement with the newer versions of ChatGPT is that while LLMs are notoriously bad at math, they are pretty good at writing Python. ChatGPT can write Python code to solve a problem, run it and give you an answer based on the code. You can also verify the code yourself. I used ChatGPT to model financial scenarios and the answers were correct in 4x. I verified the generated code it was using. But here is what it answered when I asked ChatGPT “What’s going on in Atlanta this weekend”? > This weekend in Atlanta, there are several exciting events happening: 1. Lakewood Antiques Market (May 31 - June 2): A large market in Cumming featuring antiques and collectibles . 2. Bark Meow-Ket (June 1): A craft and pet vendor market at The Lawn at Uptown Atlanta, with food and pet adoptions . (and a few other events) But a better example and where I forced it to look on the web is “Compare Amex’s current transfer partners to Chase’s partners” For context, when you earn credit card points with Amex and Chase, you can transfer them to airline and hotel loyalty programs. It searches the web (and provides the links it used to find the information) and then answers with: ——- ### American Express (Amex) Transfer Partners Amex has a robust list of transfer partners, spanning both airlines and hotels. As of 2024, Amex Membership Rewards points can be transferred to the following partners: #### Airlines: 1. Aer Lingus 2. Aeromexico … (A list of the airlines) #### Hotels: 1. Choice Privileges 2. Hilton Honors 3. Marriott Bonvoy ### Chase Transfer Partners Chase Ultimate Rewards points can be transferred to a variety of airline and hotel partners. Here are the current partners for 2024: #### Airlines: 1. Aer Lingus (AerClub) 2. Air Canada (Aeroplan) (a list of airlines) #### Hotels: 1. Hyatt (World of Hyatt) 2. IHG (IHG One Rewards) 3. Marriott Bonvoy ### Comparison Both Amex and Chase offer a strong selection of transfer partners, but there are some differences: - *Unique to Amex*: ANA, Avianca, Cathay Pacific, Delta, Etihad, Hawaiian, and Qatar. - *Unique to Chase*: Southwest Airlines, United Airlines, and Hyatt. - *Common Partners*: Aer Lingus, Air Canada, British Airways, Emirates, Flying Blue (Air France/KLM), Iberia Plus, JetBlue, Singapore Airlines, Virgin Atlantic, and Marriott Bonvoy. Amex tends to have a larger selection of airline partners, especially those in the SkyTeam and oneworld alliances. Chase, on the other hand, includes Southwest and United, which are popular with domestic travelers in the U.S., and Hyatt, which is highly regarded among hotel loyalty programs (a list of web citations)

skissane 2y ago

Idea I had: given user question, run model twice – once with web search assistance, once without. Then, continuously fine-tune the model in the background on every question (or a random sample of them) to prefer the web-search-augmented response to the non-augmented response. Maybe update the model every day (or even once every few hours) based on that fine-tuning. I wonder how that idea would perform in practice.

scarface_74 2y ago
Why though? In practice, humans don’t try to remember everything when they can use the internet and search for it.
- skissane 2y ago
  For questions about current events it could be very helpful. I mean, suppose {Biden|Trump} wins the 2024 US Presidential election. You want the model to know who the winner is the next day, as part of its background knowledge, not wait 6 months for a new version to be released whose background knowledge includes that fact. It is right that humans don’t try to remember everything, but if one only did this for a random sample of conversations, you wouldn’t remember everything - you’d probably remember key developments because they’d come up a lot (in many questions hence high probability of being sampled). You’d likely also remember some obscure trivia just because it got sampled (just like humans do too) Another option would be to have the model evaluate each conversation and have it decide whether to include it in the fine-tune (e.g. did the web search reveal important new developments the model didn’t know about?) One could also do some kind of clustering to find common gaps in the model’s background knowledge and use that to guide sampling for fine-tuning
  - scarface_74 2y ago
    What I am saying that today with ChatGPT 4x, if it detects that its model doesn’t have the current information, it will search on the web and summarize the results from different websites. One example I have was “what’s going on in Atlanta over the next two weeks”. I had another example I just posted where I asked it “who are Amex’s and Chase’s current transfer partners and which ones do they have in common”. It searched the web and summarized the information like you would expect.