4 ms·

How reliable is this? I'm half joking too but I wouldn't mind reading a report comparing this on OpenAI and various other LLMs. Somehow I doubt it'll be 100%..

How reliable is this? I'm half joking too but I wouldn't mind reading a report comparing this on OpenAI and various other LLMs. Somehow I doubt it'll be 100%... right?

est 1y ago
its as accurate as how Tiktokenizer deal with numeral characters
geor9e 1y ago
normally llms are pretty bad at math, but in this case it should just look at the final digit and map 10 values which I can't imagine going wrong
- crazysim 1y ago
  I'm just curious how _low_ can you go before it does go wrong.
  - bigbuppo 1y ago
    I mean if this were a check for primes the answer would be 1.
- daemonologist 1y ago
  The tokenizer might lump the last digit together with some preceding digits though. I know o200k_base (OpenAI -o models) tends to give groups of three (900001 for example is 900-001). Anyway, I wouldn't be surprised if a non-finetuned model made some mistakes.
- timewizard 1y ago
  is 0x2a even?
- gruez 1y ago
  I tried with this on chatgpt.com (anonymous) and it was wrong: >You are an AI assistant designed to answer questions about numbers. You will only answer with only the word true or false. >Is 393330370227914821469106615363204944758938252979261537157082994586230072180858944545028761701928694832864623009988147774229437650643225379825905427239525512110359581021414640894111281701792224552922491447051506246553646282117414112976459608594044929244664050172002138933343230226871897567 an even number? response: >True The prompt was the same used as the library: https://github.com/Calvin-LL/is-even-ai/blob/b00dbfcbb89a197ff8a476fb6ba44b23c5361443/src/IsEvenAiOpenAi.ts#L4-L14 https://github.com/Calvin-LL/is-even-ai/blob/b00dbfcbb89a197...
  - Too 1y ago
    To be fair, Javascript double point precision get tripped up by such large numbers as well.
nop_slide 1y ago
Might be more reliable to ask it to generate a JS function to compute is-even, then just js eval the returned code with the args. What could go wrong

How reliable is this? I'm half joking too but I wouldn't mind reading a report comparing this on OpenAI and various other LLMs. Somehow I doubt it'll be 100%... right?

est 1y ago

its as accurate as how Tiktokenizer deal with numeral characters

geor9e 1y ago

normally llms are pretty bad at math, but in this case it should just look at the final digit and map 10 values which I can't imagine going wrong

crazysim 1y ago
I'm just curious how _low_ can you go before it does go wrong.
- bigbuppo 1y ago
  I mean if this were a check for primes the answer would be 1.
daemonologist 1y ago
The tokenizer might lump the last digit together with some preceding digits though. I know o200k_base (OpenAI -o models) tends to give groups of three (900001 for example is 900-001). Anyway, I wouldn't be surprised if a non-finetuned model made some mistakes.
timewizard 1y ago
is 0x2a even?
gruez 1y ago
I tried with this on chatgpt.com (anonymous) and it was wrong: >You are an AI assistant designed to answer questions about numbers. You will only answer with only the word true or false. >Is 393330370227914821469106615363204944758938252979261537157082994586230072180858944545028761701928694832864623009988147774229437650643225379825905427239525512110359581021414640894111281701792224552922491447051506246553646282117414112976459608594044929244664050172002138933343230226871897567 an even number? response: >True The prompt was the same used as the library: https://github.com/Calvin-LL/is-even-ai/blob/b00dbfcbb89a197ff8a476fb6ba44b23c5361443/src/IsEvenAiOpenAi.ts#L4-L14 https://github.com/Calvin-LL/is-even-ai/blob/b00dbfcbb89a197...
- Too 1y ago
  To be fair, Javascript double point precision get tripped up by such large numbers as well.

nop_slide 1y ago

Might be more reliable to ask it to generate a JS function to compute is-even, then just js eval the returned code with the args. What could go wrong