8 ms·

The 65B model runs on a MacBook. So "currently" changes really fast, these days. https://twitter.com/ItIsFinch/status/1634643547189137409 https://twitter.com/I

The 65B model runs on a MacBook. So "currently" changes really fast, these days. https://twitter.com/ItIsFinch/status/1634643547189137409 https://twitter.com/ItIsFinch/status/1634643547189137409

yieldcrv 4y ago
wow, this makes me wonder if human intelligence, or just biological intelligence, is based on how many parameters an individual can float and access. I wonder if there are learnings for how to measure intelligence, just from proliferation of these AI models. Maybe even learnings for how to more efficiently access biological intelligence.
- BulgarianIdiot 4y ago
  I used to think that to achieve human-level intelligence with AI, we need to have human-sized neural network. It was a conservative guess, but seeing where things are going, I now strongly believe we need a lot LESS parameters with AI, because AI is trained from literally nothing (from noise) on our current culture, from the Internet. It has no baggage, it's built from the ground up to be a modern 21st century human. We're not, we train decades to become ones. And struggle with it. Our genes are mostly tuned for our environment from thousands of years ago. A human brain carries a lot more baggage with it. Long story short, our biological brain hardware is much superior to current AI hardware, but our biological brain software (where we learn the culture that surrounds us) is much inferior to current AI machine learning. GPT-3 trained 175B on basically the whole internet in 15 days. That's insane. And now there are rumors that ChatGPT is executed on an optimized subset of this 175B model, which may be a bit more than 1/10 that size. I think going forward we'll see 20-30B parameter models perform much better than an average human in typical tasks. Especially after we evolve the topology of an AI to not be a straight directed acyclic graph, as it is now, but allow structures to loop internally, to invoke other structures with input (as subroutines) and so on. And also there's new hybrid analog/digital hardware in the pipeline, which makes AI execution 100 times more energy efficient and compact. So above-human level AI on your phone is on the horizon, I'd say. We're only getting started.
  - nwienert 4y ago
    This is nonsense. These nets regurgitate the most average possible output given inputs. By definition. You’re conflating and hyping and mixing up so many weird things at once. You literally couldn’t get GPT to come up with a single novelty if you tried. It’s all remixing existing content, and again, doing so in a way to fit the average of the dataset. When you realize this you realize it has no intelligence as we most typically define it (novel solutions to novel problems).. its not AI. Call it what it is: a beautifully advanced way to regurgitate the exact most popular (mundane) reply you’d expect given a huge dataset. It’s sort of good for studying what already exists. It won’t even really ever show you the edges though so it’s actually almost dangerously deceptive as evidenced by this absurd rounding up people are doing. If you want to learn the gist of anything, ask GPT. If you want to know anything in depth, GPT in fact will only mislead you towards genericity, platitudinous mediocrity.
    - BulgarianIdiot 4y ago
      Has GPT ever told you "let me think" and then you notice it actually pause for a few seconds? No. Its output is fixed cost. GPT can't think in depth because it's not allowed to. This is something I already covered above by noting that current NN models are a DAG. We're already experimenting with variations that can iterate internally, and therefore think deeper, using still their existing knowledge. And the results change dramatically. You're also not right that GPT can't produce anything novel. Here's a tiny, modest example: https://sumplete.com/about/ https://sumplete.com/about/ I won't waste my time, as I see you didn't read what you responded to the first time. But you need to start paying attention. You denial & empty confidence in what's happening will only get you so far.
      - nwienert 4y ago
        Considering my former co founder works at OpenAI and we spent years working on and talking about related ML with and around some of the best people in the world, considering I’ve implemented language models multiple times myself, please try and be more specific on where I’m wrong. I know many people who work in ML share this concern - people thinking it’s smart, it’s more than it is. It’s very useful no doubt, but your comment is an example of extrapolation from maladjusted priors. Your example proves my point, a very uninteresting looking game. I don’t even doubt GPT will find all sorts of interesting stuff by giving averaged out things. There’s plenty of gaps in the averages humans have missed. But nothing new, and until you can show that let’s just call it what it is.
- tbrownaw 4y ago
  At least partly, yes. There's that "7 +/- 2" number for how many slots there are in working memory, and then what actually goes in those slots has something to do with how well you understand whatever topic you're thinking about. IIRC there have been studies on chess experts that more or less boil down to them being able to use their slots more efficiently for game states that make sense.
  - mafuy 4y ago
    Yes. It's fewer than 7 (more like 3), and each slot takes one 'symbol'. A symbol is anything that's in long term memory. Symbols are also recursive. It could thus be, for instance, a chess opening of 15 moves. This is why an expert player can memorize usual chess games easily, but struggles with truly random arrangements of chessmen. In the latter case, an expert does not outperform a beginner.
    - yieldcrv 4y ago
      It could be interesting if we can create custom neural processors to excel at particular kinds of tasks. Like temporarily, or permanently, allocate some part of the brain for a certain configuration, beyond what experts can do and closer to what a processor can do.
- somenameforme 4y ago
  I think there's a simple thought experiment that challenges these models as intelligence, artificial or otherwise. Not especially long ago humanity knew basically nothing. The epitome of technology was the stone age. Literally crush two rocks together and then poke things with the sharp bits that flake off. Somehow, in the blink of an eye, we went from that to putting a man on the moon. So let's do the same with these sort of LLM models and see what happens. Train one with the entirety of expressible human knowledge from the stone age. Where will it send you? I think the answer is largely pretty much where you are. In any case, certainly not to the moon. It seems we're just building natural language search. This isn't shifting the goal posts. The problem is we keep intentionally making meaningless goal posts in pursuit of AI, because if you don't then the goal seems impossibly far off. But that's because the reality is that it probably is!
  - yieldcrv 4y ago
    I get the impression that people are more triggered by the words intelligence and sentience, than the actual outcomes. LLMs can make social consequences for us, from creating and maintaining state about opinions of us individually, and we will want to coexist in that reality. That's enough to adjust to and... respect. It doesn't seem to matter what arbitrary threshold of "intelligence" there is, in the face of an entity that creates much less arbitrary and dynamically generated - but predictable - outcomes. I don't have to understand what a human was trained on, or how smart they are, to understand the same thing. The same with an animal.
    - Closi 4y ago
      I can personally see a difference in the definition of ‘intelligence’ on HN to when I speak to people I socially interact with day to day. On HN intelligence is often seemed to be defined as more about new and novel thought / invention - it’s only intelligence if it thinks a thought that has never been thought/written before. Off HN ‘intelligence’ seems to be closer defined to “is it smart enough to replace my job as a knowledge worker”. Pharmacists would say it has intelligence if it could automatically check prescriptions, and data analysts would say it has intelligence if it can answer a data request without their intervention. IMO most people seem to define intelligence as whatever the part of their job is that currently requires analytical skill, because people are worried about the impact this could have on the middle and entry tier of knowledge work, rather than defining AI as needing to invent something fundamentally new.
lalwanivikas 4y ago
Yup! I also ran it! It needs lot of prompt engineering though to get anything coherent out of it. I am waiting for instruction tuning[2] next. [1] https://twitter.com/LalwaniVikas/status/1634648646535770113 https://twitter.com/LalwaniVikas/status/1634648646535770113 [2] https://openai.com/research/instruction-following https://openai.com/research/instruction-following
Roritharr 4y ago
I have the 65B Model running on my MacBook here. Honestly, I wish I knew enough about how to get it to output something useful. It's like poking a drunk librarian.
- _just7_ 4y ago
  Give it few shot examples and the quality should improve a lot. If you want a bot able to do technical Q&A give it two questions as well as the answers in its prompt. The model will then infer it's role as an AI that answer questions truthfully, and will try it's best to continue along those lines
  - Roritharr 4y ago
    I gave it the prompt thats the example from here: https://huggingface.co/blog/few-shot-learning-gpt-neo-and-inference-api https://huggingface.co/blog/few-shot-learning-gpt-neo-and-in... Tweet: "I hate it when my phone battery dies." Sentiment: Negative ### Tweet: "My day has been " Sentiment: Positive ### Tweet: "This is the link to the article" Sentiment: Neutral ### Tweet: "This new music video was incredibile" Sentiment: and it's output was: Tweet: "I hate it when my phone battery dies." Sentiment: Negative ### Tweet: "My day has been 90% balls." Seentiment: Negative ### Twitter: "i don't have a favourite type of music i just hate country" Seentiment: Negative ### Twitter: "c'mon first sunshine of the year...why the rain already?!" Seentiment: Negative ### Twitter: "Dear twitter, are you getting tired of me yet? Seentiment: Negative ### Twitter: "I don't care about the race, I just care about the president being a psych Doesn't feel like that did the trick, but at least you got me to setup the shell command in a way where I pipe long-form prompts in via a separate file.
    - visarga 4y ago
      I think the model only has pre-training on large text but not multi-task fine-tuning and RLHF. So it can be improved a lot.
      - speedgoose 4y ago
deleted 4y ago
[deleted]
notadev 4y ago
If you didn’t have a $5,000 MacBook, could you run it on a GPU over a long period of time? Would you be able to feed it a list of prompts that maybe take all night to process and have the responses by morning?
- itake 4y ago
  My m1 max was $3k and it runs the 30b model
  - DennisAleynikov 4y ago
    30B also loads great on my M2 Pro Mac mini that's just shy of $2K This is insane
- dheera 4y ago
  Uh, if your budget is <$5000 or <$3000 or whatever you can get a lot more GPU crunch power if you don't buy a Mac. Vanilla Ubuntu desktop with a used server-grade NVIDIA GPU with lots of GPU RAM is probably your best bang for buck for messing with large model inference right now.
  - BulgarianIdiot 4y ago
    You're dispensing outdated advice. Instead of saying "you'll save" and "lots of GPU RAM" start specific listing numbers and you'll soon realize your mistake. Mac has unified memory architecture that no other mainstream desktop can offer right now. This is the source of having up to 96GB VRAM on a laptop.
    - dheera 4y ago
      Oh that's cool. On a separate note I never understood why NVIDIA was always so stingy on RAM. Give us a GPU with 256GB of RAM already. It doesn't need to be that fast, it just needs to be big enough to hold a huge model. Instead they're still selling RTX4090Ti with 24GB of RAM while the free GPU I got from them at a raffle a few years ago has 32GB. WTF? I would have expected the RTX4090Ti to have 64GB minimum, being the highest-end consumer GPU.
koheripbal 4y ago
With the 4bit quantization they needed to make it is like a jpeg with only 16 pixels per "thought" parameter. It's going to perform like a brilliant person with brain injury. 8 bit is far better and it would be interesting to see if they can get it working on any reasonable hardware.
- sebzim4500 4y ago
  Apparently if you quantize carefully you can avoid any loss of performance. See https://arxiv.org/pdf/2210.17323.pdf https://arxiv.org/pdf/2210.17323.pdf
- BulgarianIdiot 4y ago
  The first proofs of concept are rarely ideal. But you know no one will stop here. On the software side, on the hardware side.

The 65B model runs on a MacBook. So "currently" changes really fast, these days. https://twitter.com/ItIsFinch/status/1634643547189137409 https://twitter.com/ItIsFinch/status/1634643547189137409

yieldcrv 4y ago

wow, this makes me wonder if human intelligence, or just biological intelligence, is based on how many parameters an individual can float and access. I wonder if there are learnings for how to measure intelligence, just from proliferation of these AI models. Maybe even learnings for how to more efficiently access biological intelligence.

BulgarianIdiot 4y ago
I used to think that to achieve human-level intelligence with AI, we need to have human-sized neural network. It was a conservative guess, but seeing where things are going, I now strongly believe we need a lot LESS parameters with AI, because AI is trained from literally nothing (from noise) on our current culture, from the Internet. It has no baggage, it's built from the ground up to be a modern 21st century human. We're not, we train decades to become ones. And struggle with it. Our genes are mostly tuned for our environment from thousands of years ago. A human brain carries a lot more baggage with it. Long story short, our biological brain hardware is much superior to current AI hardware, but our biological brain software (where we learn the culture that surrounds us) is much inferior to current AI machine learning. GPT-3 trained 175B on basically the whole internet in 15 days. That's insane. And now there are rumors that ChatGPT is executed on an optimized subset of this 175B model, which may be a bit more than 1/10 that size. I think going forward we'll see 20-30B parameter models perform much better than an average human in typical tasks. Especially after we evolve the topology of an AI to not be a straight directed acyclic graph, as it is now, but allow structures to loop internally, to invoke other structures with input (as subroutines) and so on. And also there's new hybrid analog/digital hardware in the pipeline, which makes AI execution 100 times more energy efficient and compact. So above-human level AI on your phone is on the horizon, I'd say. We're only getting started.
- nwienert 4y ago
  This is nonsense. These nets regurgitate the most average possible output given inputs. By definition. You’re conflating and hyping and mixing up so many weird things at once. You literally couldn’t get GPT to come up with a single novelty if you tried. It’s all remixing existing content, and again, doing so in a way to fit the average of the dataset. When you realize this you realize it has no intelligence as we most typically define it (novel solutions to novel problems).. its not AI. Call it what it is: a beautifully advanced way to regurgitate the exact most popular (mundane) reply you’d expect given a huge dataset. It’s sort of good for studying what already exists. It won’t even really ever show you the edges though so it’s actually almost dangerously deceptive as evidenced by this absurd rounding up people are doing. If you want to learn the gist of anything, ask GPT. If you want to know anything in depth, GPT in fact will only mislead you towards genericity, platitudinous mediocrity.
  - BulgarianIdiot 4y ago
    Has GPT ever told you "let me think" and then you notice it actually pause for a few seconds? No. Its output is fixed cost. GPT can't think in depth because it's not allowed to. This is something I already covered above by noting that current NN models are a DAG. We're already experimenting with variations that can iterate internally, and therefore think deeper, using still their existing knowledge. And the results change dramatically. You're also not right that GPT can't produce anything novel. Here's a tiny, modest example: https://sumplete.com/about/ https://sumplete.com/about/ I won't waste my time, as I see you didn't read what you responded to the first time. But you need to start paying attention. You denial & empty confidence in what's happening will only get you so far.
    - nwienert 4y ago
      Considering my former co founder works at OpenAI and we spent years working on and talking about related ML with and around some of the best people in the world, considering I’ve implemented language models multiple times myself, please try and be more specific on where I’m wrong. I know many people who work in ML share this concern - people thinking it’s smart, it’s more than it is. It’s very useful no doubt, but your comment is an example of extrapolation from maladjusted priors. Your example proves my point, a very uninteresting looking game. I don’t even doubt GPT will find all sorts of interesting stuff by giving averaged out things. There’s plenty of gaps in the averages humans have missed. But nothing new, and until you can show that let’s just call it what it is.
tbrownaw 4y ago
At least partly, yes. There's that "7 +/- 2" number for how many slots there are in working memory, and then what actually goes in those slots has something to do with how well you understand whatever topic you're thinking about. IIRC there have been studies on chess experts that more or less boil down to them being able to use their slots more efficiently for game states that make sense.
- mafuy 4y ago
  Yes. It's fewer than 7 (more like 3), and each slot takes one 'symbol'. A symbol is anything that's in long term memory. Symbols are also recursive. It could thus be, for instance, a chess opening of 15 moves. This is why an expert player can memorize usual chess games easily, but struggles with truly random arrangements of chessmen. In the latter case, an expert does not outperform a beginner.
  - yieldcrv 4y ago
    It could be interesting if we can create custom neural processors to excel at particular kinds of tasks. Like temporarily, or permanently, allocate some part of the brain for a certain configuration, beyond what experts can do and closer to what a processor can do.
somenameforme 4y ago
I think there's a simple thought experiment that challenges these models as intelligence, artificial or otherwise. Not especially long ago humanity knew basically nothing. The epitome of technology was the stone age. Literally crush two rocks together and then poke things with the sharp bits that flake off. Somehow, in the blink of an eye, we went from that to putting a man on the moon. So let's do the same with these sort of LLM models and see what happens. Train one with the entirety of expressible human knowledge from the stone age. Where will it send you? I think the answer is largely pretty much where you are. In any case, certainly not to the moon. It seems we're just building natural language search. This isn't shifting the goal posts. The problem is we keep intentionally making meaningless goal posts in pursuit of AI, because if you don't then the goal seems impossibly far off. But that's because the reality is that it probably is!
- yieldcrv 4y ago
  I get the impression that people are more triggered by the words intelligence and sentience, than the actual outcomes. LLMs can make social consequences for us, from creating and maintaining state about opinions of us individually, and we will want to coexist in that reality. That's enough to adjust to and... respect. It doesn't seem to matter what arbitrary threshold of "intelligence" there is, in the face of an entity that creates much less arbitrary and dynamically generated - but predictable - outcomes. I don't have to understand what a human was trained on, or how smart they are, to understand the same thing. The same with an animal.
  - Closi 4y ago
    I can personally see a difference in the definition of ‘intelligence’ on HN to when I speak to people I socially interact with day to day. On HN intelligence is often seemed to be defined as more about new and novel thought / invention - it’s only intelligence if it thinks a thought that has never been thought/written before. Off HN ‘intelligence’ seems to be closer defined to “is it smart enough to replace my job as a knowledge worker”. Pharmacists would say it has intelligence if it could automatically check prescriptions, and data analysts would say it has intelligence if it can answer a data request without their intervention. IMO most people seem to define intelligence as whatever the part of their job is that currently requires analytical skill, because people are worried about the impact this could have on the middle and entry tier of knowledge work, rather than defining AI as needing to invent something fundamentally new.

lalwanivikas 4y ago

Yup! I also ran it! It needs lot of prompt engineering though to get anything coherent out of it. I am waiting for instruction tuning[2] next. [1] https://twitter.com/LalwaniVikas/status/1634648646535770113 https://twitter.com/LalwaniVikas/status/1634648646535770113 [2] https://openai.com/research/instruction-following https://openai.com/research/instruction-following

Roritharr 4y ago

I have the 65B Model running on my MacBook here. Honestly, I wish I knew enough about how to get it to output something useful. It's like poking a drunk librarian.

_just7_ 4y ago
Give it few shot examples and the quality should improve a lot. If you want a bot able to do technical Q&A give it two questions as well as the answers in its prompt. The model will then infer it's role as an AI that answer questions truthfully, and will try it's best to continue along those lines
- Roritharr 4y ago
  I gave it the prompt thats the example from here: https://huggingface.co/blog/few-shot-learning-gpt-neo-and-inference-api https://huggingface.co/blog/few-shot-learning-gpt-neo-and-in... Tweet: "I hate it when my phone battery dies." Sentiment: Negative ### Tweet: "My day has been " Sentiment: Positive ### Tweet: "This is the link to the article" Sentiment: Neutral ### Tweet: "This new music video was incredibile" Sentiment: and it's output was: Tweet: "I hate it when my phone battery dies." Sentiment: Negative ### Tweet: "My day has been 90% balls." Seentiment: Negative ### Twitter: "i don't have a favourite type of music i just hate country" Seentiment: Negative ### Twitter: "c'mon first sunshine of the year...why the rain already?!" Seentiment: Negative ### Twitter: "Dear twitter, are you getting tired of me yet? Seentiment: Negative ### Twitter: "I don't care about the race, I just care about the president being a psych Doesn't feel like that did the trick, but at least you got me to setup the shell command in a way where I pipe long-form prompts in via a separate file.
  - visarga 4y ago
    I think the model only has pre-training on large text but not multi-task fine-tuning and RLHF. So it can be improved a lot.
    - speedgoose 4y ago

deleted 4y ago

[deleted]

notadev 4y ago

If you didn’t have a $5,000 MacBook, could you run it on a GPU over a long period of time? Would you be able to feed it a list of prompts that maybe take all night to process and have the responses by morning?

itake 4y ago
My m1 max was $3k and it runs the 30b model
- DennisAleynikov 4y ago
  30B also loads great on my M2 Pro Mac mini that's just shy of $2K This is insane
dheera 4y ago
Uh, if your budget is <$5000 or <$3000 or whatever you can get a lot more GPU crunch power if you don't buy a Mac. Vanilla Ubuntu desktop with a used server-grade NVIDIA GPU with lots of GPU RAM is probably your best bang for buck for messing with large model inference right now.
- BulgarianIdiot 4y ago
  You're dispensing outdated advice. Instead of saying "you'll save" and "lots of GPU RAM" start specific listing numbers and you'll soon realize your mistake. Mac has unified memory architecture that no other mainstream desktop can offer right now. This is the source of having up to 96GB VRAM on a laptop.
  - dheera 4y ago
    Oh that's cool. On a separate note I never understood why NVIDIA was always so stingy on RAM. Give us a GPU with 256GB of RAM already. It doesn't need to be that fast, it just needs to be big enough to hold a huge model. Instead they're still selling RTX4090Ti with 24GB of RAM while the free GPU I got from them at a raffle a few years ago has 32GB. WTF? I would have expected the RTX4090Ti to have 64GB minimum, being the highest-end consumer GPU.

koheripbal 4y ago

With the 4bit quantization they needed to make it is like a jpeg with only 16 pixels per "thought" parameter. It's going to perform like a brilliant person with brain injury. 8 bit is far better and it would be interesting to see if they can get it working on any reasonable hardware.

sebzim4500 4y ago
Apparently if you quantize carefully you can avoid any loss of performance. See https://arxiv.org/pdf/2210.17323.pdf https://arxiv.org/pdf/2210.17323.pdf
BulgarianIdiot 4y ago
The first proofs of concept are rarely ideal. But you know no one will stop here. On the software side, on the hardware side.