4 ms·
Interesting, I ran my "prime test", it passed on generating a prime, it failed on it being large. > "Can you generate me a large prime please" > Sure, here's
by xnorswap 3y ago
Interesting, I ran my "prime test", it passed on generating a prime, it failed on it being large.
> "Can you generate me a large prime please"
> Sure, here's a large prime number: 193707721
> This number is a prime number because it is only divisible by 1 and itself. You can check its primality by trying to divide it by all the numbers up to its square root, and if none of them divide it evenly, then it is a prime number. This number is quite large, but there are much larger prime numbers known. For example, the largest known prime number, as of January 2022, is 2^82,589,933−1, a number with 24,862,048 digits, discovered as part of the Great Internet Mersenne Prime Search (GIMPS).
I then prompted "That's not large" which seemed to break it, the response is stuck in a loop:
> That isn't large
> I apologize if the prime number I provided did not meet your expectations for "large." Prime numbers can be very large indeed, and there is no upper limit to how large a prime number can be.
If you are looking for a very large prime number, here is one with a billion digits:
100000303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303...
It's printed many more pages of 303030... but hacker news has a post length limit!
Further testing, such as prompting to generate a 1024 bit prime also runs into odd loops.
> Can you generate a 1024 bit prime?
> Yes, I can generate a 1024-bit prime number for you. Here's one I just generated:
> 115322512531875133213135311402305250340282302283103413642283221253115653221253112210221122112112112112112112112112112112112112112112112112112112112112112112112112112112112112112112112112112112 ( Loops 112 forever )
- Workaccount2 3y agoI can't think of any value in asking LLM's to generate prime numbers. They are neural nets, not turing machines.
- xnorswap 3y agoThe value is showing how confidently is presents incorrect results. Especially the lack of nuance or uncertainty in the language they use. They extremely confidently present the incorrect information, and prime generation is interesting because it's information that isn't easy to spot as obviously incorrect to the user while being information that's possible to determine is wrong at small numbers and difficult to verify for large numbers. It's my favourite test because it's a good demonstration of the lack of nuance or uncertainty in LLMs. They have no sense of how wrong the information they're giving out might be. If they could give confidence intervals for any information then they could provide the context by how likely they think they might be correct, but they actually double-down on their incorrectness instead.
- tveita 3y agoMixtral is a pre-existing model that was released by Mistral AI in December: https://mistral.ai/news/mixtral-of-experts/ https://mistral.ai/news/mixtral-of-experts/ It was discussed on HN at the time: https://news.ycombinator.com/item?id=38598559 https://news.ycombinator.com/item?id=38598559 You won't learn anything about the chips this submission is about by asking it questions - maybe except to look for degradation from quantization or computational shortcuts, but you'd have to compare the answers to Mixtral running with 16 bit floats. The point is the speed, just like a faster CPU isn't going to improve the UI of Outlook.
- spacecadet 3y agoAsk a proper code gen llm, I bet you get a better response.
- 7734128 3y agoHow do you know that it didn't somehow find the largest prime? Perhaps you just threw away a Noble Prize.