4 ms·

> by next month we will have some previously-never-seen model/algorithm/chip/tech that will run locally on our phones and computers. At that point is game over

> by next month we will have some previously-never-seen model/algorithm/chip/tech that will run locally on our phones and computers. At that point is game over This comment is so wildly inaccurate and hype, and yet so common in the GPT craze on HN It is not happening on your phone and computers in the near term for 2 reasons: 1. Entropy, the models are LARGER, NOT SMALLER. The amazing performance come from the fact that they are BIG. This is opposite of Moore's law, which itself is ending. 2. Companies want to make money on their models, vending a client is clearly not the direction that companies are going. In fact greedy companies wanted everything to happen on the server end even without any real customer need, e.g. Jira.

nico 4y ago
1. You might not be up to date. This was posted on HN 4 days ago: https://news.ycombinator.com/item?id=35281026 https://news.ycombinator.com/item?id=35281026 Check it out. They are claiming 100x faster training and inference with smaller models, performing almost with the same accuracy as GPT. People are already running models on their phones. Some people have been posting here and on Twitter how they can run Alpaca (Llama + GPT fine tuning), on their Pixel phones. 2. The cat is out of the bag. People are creating and releasing stuff faster than companies can keep up with. Have you seen what ChatGPT is already able to do with plug-ins? You can give it control of a Linux machine and tell it “create a basic CRUD app and run it on a docker container”, and it will do it. That means soon AI is going to be training itself, or at least with minimum input/direction from humans.
- aleksiy123 4y ago
  Do you have any links to chatgpt with Linux. I want to try
  - nico 4y ago
    They used ChatGPT’s code interpreter plug-in. Here’s a good write up about that plug-in and an overview of a lot of the things you can do: https://andrewmayneblog.wordpress.com/2023/03/23/chatgpt-code-interpreter-magic/ https://andrewmayneblog.wordpress.com/2023/03/23/chatgpt-cod... If you don’t have access to plug-ins yet, you could do it yourself by using the GPT api, writing a prompt that tells GPT they are interacting with a terminal and to only output terminal commands, then pass the output to a terminal, and passing back the output to GPT in a loop. I can’t find the exact link to the docker app video/article. Lately it’s really hard to keep up. The search feature here on HN kinda sucks on mobile and Google is hopelessly behind with how fast things are coming out.
    - aleksiy123 4y ago
      Awesome thanks
- 6gvONxR4sf7o 4y ago
  That project isn’t close to the SoTA models. It performs well, but not close to GPT-4 for example. “Almost the same performance as gpt” always ends up meaning it performs like one of the smaller GPT models.
  - nico 4y ago
    Sure but it just came out. And at the rate things are going, it will likely be just as good as GPT-3 or 4 soon. Also what is “close”? The benchmarks I saw showed around 80% accuracy compared to GPT.
    - Aperocky 4y ago
      > it will likely be just as good as GPT-3 or 4 soon. It just came out, but in 4 months of development this vehicle should be able to do 2000 miles per gallon instead of 20 miles per gallon. GPT is not magic, it's performance is linear with its size.
      - nico 4y ago
        It is not about GPT, it’s a whole revolution that is happening. New models are coming out almost every day. Including some that have almost same performance as GPT but are 100x faster to train and are smaller (look at this:https://news.ycombinator.com/item?id=35281026 https://news.ycombinator.com/item?id=35281026). You can believe or deny whatever you want. The technology is moving way faster than we can keep up with and the speed is accelerating.
    - 6gvONxR4sf7o 4y ago
      I'm lazy, so this isn't exhaustive. The first benchmark in the RWKV README is LAMBADA, where it got about 39% accuracy. SOTA is 90%, so that's about 6x more errors than SOTA. 4.5x the error rate of the first versions of GPT-3 in 2020. I didn't find LAMBADA benchmarks for GPT3.5 or GPT4, but from the GPT4 technical report, it generally makes about 2-3x fewer mistakes than GPT3.5 or prior SOTA on various benchmarks, so the difference is even more extreme. It's safe to say RWKV is not close to SOTA. Meanwhile GPT4 still really sucks at a lot of things. So by the time open source commodity hardware models do reach GPT4 level, the SOTA will be that much further ahead. The tech will be ubiquitous some day, but it's not right around the corner.

nico 4y ago

1. You might not be up to date. This was posted on HN 4 days ago: https://news.ycombinator.com/item?id=35281026 https://news.ycombinator.com/item?id=35281026 Check it out. They are claiming 100x faster training and inference with smaller models, performing almost with the same accuracy as GPT. People are already running models on their phones. Some people have been posting here and on Twitter how they can run Alpaca (Llama + GPT fine tuning), on their Pixel phones. 2. The cat is out of the bag. People are creating and releasing stuff faster than companies can keep up with. Have you seen what ChatGPT is already able to do with plug-ins? You can give it control of a Linux machine and tell it “create a basic CRUD app and run it on a docker container”, and it will do it. That means soon AI is going to be training itself, or at least with minimum input/direction from humans.

aleksiy123 4y ago
Do you have any links to chatgpt with Linux. I want to try
- nico 4y ago
  They used ChatGPT’s code interpreter plug-in. Here’s a good write up about that plug-in and an overview of a lot of the things you can do: https://andrewmayneblog.wordpress.com/2023/03/23/chatgpt-code-interpreter-magic/ https://andrewmayneblog.wordpress.com/2023/03/23/chatgpt-cod... If you don’t have access to plug-ins yet, you could do it yourself by using the GPT api, writing a prompt that tells GPT they are interacting with a terminal and to only output terminal commands, then pass the output to a terminal, and passing back the output to GPT in a loop. I can’t find the exact link to the docker app video/article. Lately it’s really hard to keep up. The search feature here on HN kinda sucks on mobile and Google is hopelessly behind with how fast things are coming out.
  - aleksiy123 4y ago
    Awesome thanks
6gvONxR4sf7o 4y ago
That project isn’t close to the SoTA models. It performs well, but not close to GPT-4 for example. “Almost the same performance as gpt” always ends up meaning it performs like one of the smaller GPT models.
- nico 4y ago
  Sure but it just came out. And at the rate things are going, it will likely be just as good as GPT-3 or 4 soon. Also what is “close”? The benchmarks I saw showed around 80% accuracy compared to GPT.
  - Aperocky 4y ago
    > it will likely be just as good as GPT-3 or 4 soon. It just came out, but in 4 months of development this vehicle should be able to do 2000 miles per gallon instead of 20 miles per gallon. GPT is not magic, it's performance is linear with its size.
    - nico 4y ago
      It is not about GPT, it’s a whole revolution that is happening. New models are coming out almost every day. Including some that have almost same performance as GPT but are 100x faster to train and are smaller (look at this:https://news.ycombinator.com/item?id=35281026 https://news.ycombinator.com/item?id=35281026). You can believe or deny whatever you want. The technology is moving way faster than we can keep up with and the speed is accelerating.
  - 6gvONxR4sf7o 4y ago
    I'm lazy, so this isn't exhaustive. The first benchmark in the RWKV README is LAMBADA, where it got about 39% accuracy. SOTA is 90%, so that's about 6x more errors than SOTA. 4.5x the error rate of the first versions of GPT-3 in 2020. I didn't find LAMBADA benchmarks for GPT3.5 or GPT4, but from the GPT4 technical report, it generally makes about 2-3x fewer mistakes than GPT3.5 or prior SOTA on various benchmarks, so the difference is even more extreme. It's safe to say RWKV is not close to SOTA. Meanwhile GPT4 still really sucks at a lot of things. So by the time open source commodity hardware models do reach GPT4 level, the SOTA will be that much further ahead. The tech will be ubiquitous some day, but it's not right around the corner.