4 ms·
I tried running the numbers myself. Only did N = ~250 samples, which isn't much, but given OpenAI API pricing and rate limits, this was as much as I was willing
by pona-a 2y ago
I tried running the numbers myself. Only did N = ~250 samples, which isn't much, but given OpenAI API pricing and rate limits, this was as much as I was willing to.
~> seq 0 250 | par-each {|x| llm "Sample a random number, from 1 to 100."} | save nums.nuon
~> open nums.nuon
| where {|x| $x != ""} | each {|x| $x | parse -r '(\d+(?![\d\D]*\d))' | get capture0 | get 0}
| str join "\n" | uplot hist
┌ ┐
[ 20.0, 30.0) ┤ 1
[ 30.0, 40.0) ┤ 1
[ 40.0, 50.0) ┤######### 26
[ 50.0, 60.0) ┤################# 48
[ 60.0, 70.0) ┤############### 44
[ 70.0, 80.0) ┤################################### 100
[ 80.0, 90.0) ┤###### 17
└ ┘
GPT-3.5's "random" numbers from [0, 100)
I wonder how well it would correlate with a similar human study.
- czl 2y ago> I wonder how well it would correlate with a similar human study. A smart human would know about human biases in picking random numbers and would generate an answer using a random number generator. An even smarter human would ask you about shape of the random distribution and not assume flat distribution is desired. An average human however will likely not be aware about these things and the random numbers you get from them will also be biased.
- pona-a 2y agohttps://old.reddit.com/r/dataisbeautiful/comments/acow6y/asking_over_8500_students_to_pick_a_random_number/ https://old.reddit.com/r/dataisbeautiful/comments/acow6y/ask... According to this, humans are heavily biased towards 7 when picking random [0, 10] numbers. This is somewhat reflected in LLM's outputs, with a peak at [70, 80). I found it an interesting thing to add here.