4 ms·

This reads like a standard analysis done by the “points guy” every year. I suspect this is more or less scraped from his nevof those articles.

scarface74 4y ago
So it scraped it based on my own point valuations?
- foobazgt 4y ago
  Yes! Well, scrape is a slight exaggeration, but it's more than possible that most of the relevant data came from points guy analysis. I'd suggest reading https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... to understand why just changing a few values in your input wouldn't throw an LLM off. It's not matching on exact words but rather embeddings (think like synonyms, but stronger).
  - scarface74 4y ago
    I’ve been able to throw almost any random pre algebra problem at it and it got it right. But how is this any different than how the average high schooler studies for the SAT? You study enough problems and you recognize similarities?
    - danans 4y ago
      Algebra is by definition a language, and a very simple one at that that whose rules can be summarized in a few pages [1]. That's exactly the domain that ChatGPT excels at the most: languages for which tons of examples are available. Just like programming languages. It falls on its face with things that involve non-linguistic facts that require knowledge to answer, my current favorite being driving directions. It will just make up completely fictitious roads and turns if you ask it for directions for point A to point B. 1. http://faculty.ung.edu/mgoodroe/PriorCourses/Math_0999_General/The%20Formal%20Rules%20of%20Algebra.pdf http://faculty.ung.edu/mgoodroe/PriorCourses/Math_0999_Gener...
      - scarface74 4y ago
        Again, how is that different from humans? You throw me in a random city I won’t be able to give you directions. But that’s an easy problem to solve from a technology standpoint. If ChatGPT recognizes the same types of domains that dumb assistants can do, delegate the answer to an API.
        danans 4y ago
        > If ChatGPT recognizes the same types of domains that dumb assistants can do, delegate the answer to an API This is backward from an architectural standpoint. LLMs are a very expensive way to do intent detection, and a very inexact way to delegate to an API. The more sensible way is to first try the standard search engine approaches to detecting intent (which often use smaller language models) and delegating to knowledge based services, and if that doesn't return good result, delegate to the LLM if the task is suited to that.

scarface74 4y ago

So it scraped it based on my own point valuations?

foobazgt 4y ago
Yes! Well, scrape is a slight exaggeration, but it's more than possible that most of the relevant data came from points guy analysis. I'd suggest reading https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... to understand why just changing a few values in your input wouldn't throw an LLM off. It's not matching on exact words but rather embeddings (think like synonyms, but stronger).
- scarface74 4y ago
  I’ve been able to throw almost any random pre algebra problem at it and it got it right. But how is this any different than how the average high schooler studies for the SAT? You study enough problems and you recognize similarities?
  - danans 4y ago
    Algebra is by definition a language, and a very simple one at that that whose rules can be summarized in a few pages [1]. That's exactly the domain that ChatGPT excels at the most: languages for which tons of examples are available. Just like programming languages. It falls on its face with things that involve non-linguistic facts that require knowledge to answer, my current favorite being driving directions. It will just make up completely fictitious roads and turns if you ask it for directions for point A to point B. 1. http://faculty.ung.edu/mgoodroe/PriorCourses/Math_0999_General/The%20Formal%20Rules%20of%20Algebra.pdf http://faculty.ung.edu/mgoodroe/PriorCourses/Math_0999_Gener...
    - scarface74 4y ago
      Again, how is that different from humans? You throw me in a random city I won’t be able to give you directions. But that’s an easy problem to solve from a technology standpoint. If ChatGPT recognizes the same types of domains that dumb assistants can do, delegate the answer to an API.
      - danans 4y ago
        > If ChatGPT recognizes the same types of domains that dumb assistants can do, delegate the answer to an API This is backward from an architectural standpoint. LLMs are a very expensive way to do intent detection, and a very inexact way to delegate to an API. The more sensible way is to first try the standard search engine approaches to detecting intent (which often use smaller language models) and delegating to knowledge based services, and if that doesn't return good result, delegate to the LLM if the task is suited to that.