6 ms·

It seems the answers are pre-programmed because I drew just a car and it said "I know! It's a police car" even though I'd drawn no siren. Also with the sweater

by daveheq 10y ago

It seems the answers are pre-programmed because I drew just a car and it said "I know! It's a police car" even though I'd drawn no siren. Also with the sweater drawing I'd drawn just a T-shirt accidentally then when drawing a long sleeve it said it knew it was a sweater when it could have been similar things. So this demonstration isn't about it identifying the drawing correctly, it's just about saying when it's found something considered close enough in an overly-broad range of ambiguous things based on the answer it's been given already. It did identify some partially drawn things, like a line or a circle or more complex things the partial drawing could have been, but the pre-loaded rigging made me stop the test.

marxidad 10y ago
It's just picking the closest thing from its limited training set.
zipppy 10y ago
I agree and believe that it's pre-programmed in some ways. I took the test twice, and in the first set I had 'zigzag' and in the second set I had 'stairs'. I drew the exact same doodle for each and for each it guessed correctly on its first try.
- OJFord 10y ago
  No, I believe it makes a series of guesses ranked for 'closeness'. If what you're supposed to be drawing is ranked 'sufficiently high' the game/training side of it moves on to the next.
  - zipppy 10y ago
    What? My point is that I drew the same doodle for different words, and for each word it put forth a different _first_ guess. The first guess for each doodle was the word that was provided -- if this software was completely legitimate, I would think it should have guessed the same word for each (and then moved on to other guesses).
    - simias 10y ago
      They weren't really the same though, right? Only similar. A good test for that would be to script the game and give it pixel-identical answers for each questions and see if the result is different.
      - zipppy 10y ago
        Correct that they were not identical, but they were quite close; five lines that looked like a 'w'.
        simias 10y ago
        FWIW it guessed my lightning correctly (a simple `z` like shape). Its 2nd guess was "zig-zag" and the 3rd one "star" for some reason. So it really looks like for this network a zig-zag and a star are more similar than I would've guessed. I guess a star is a zig zag looping on itself, kinda.
    - mthoms 10y ago
      I wonder if the zig-zag drawing and the stairs were actually as similar as you remember. For instance, our sub-conscious tendency when drawing stairs would be to have straight lines at 0 and 90 degrees but a "zig-zag" wouldn't necessarily have that feature. Thus, even though you thought the drawings were the same, there were sufficient detectable differences for the AI to recognize.
rtkwe 10y ago
It's because it's gathering training data from the things other people are drawing. So if it's just asking for police cars anything car like should just fall into the police car category.
- mikeash 10y ago
  It also stops you once it correctly identifies the drawing, even if you're far from done. If it incorporates that into its training, then it'll presumably learn to figure out what you meant earlier and earlier. It's interesting to consider whether it would end up making guesses that it sometimes gets correct, or whether it would actually learn to discriminate based on very early stages of drawing. It's conceivable to me that people might start out drawing "police car" differently from "car" in some subtle but detectable way, even if it doesn't involve what we humans would consider to be the distinguishing features of a police car.
asfaltboy 10y ago
I think the QUESTIONS are definitely pre-programmed, so the answer set is probably limited too.
gpsx 10y ago
There is no way it should have recognized my potato.
NiKMic 10y ago
I tested this. It asked for a shovel and I drew an aeroplane and it guessed aeroplane. I did the same thing a number of times and I got around a 60% correct response rate.
quirkafleeg 10y ago
> It seems the answers are pre-programmed because I drew just a car and it said "I know! It's a police car" even though I'd drawn no siren. Exactly the same thing happened to me (I did a CTRL+F on 'police' to find this post), which immediately turned me from 'this is cool' to almost quitting, and coming here to see if anyone else experienced the same thing (perhaps with different images). It also didn't guess my drawing of a bulldozer, which I was surprised at since my drawing wasn't bad, and easily recognizable to any human of any age as a bulldozer. I went back to complete the test, and it tells you what it guessed, which helped to explain both things. Here's what it thinks a bulldozer looks like: http://i.imgur.com/SU9edxi.gif http://i.imgur.com/SU9edxi.gif Here's the difference between "car" and "police car": http://i.imgur.com/YKpANPl.gif http://i.imgur.com/YKpANPl.gif You, like me, obviously don't draw cars from the Soviet Union. Edit: I just noticed (thanks to another post here) it also shows several examples of what it thinks an object looks like, and now I'm not sure why it didn't guess bulldozer, when couple of the images look like mine, certainly far more than the things it guessed.
JangoSteve 10y ago
I thought the same thing. It was a bit more subtle for me, but it prompted me to draw a sun, and when I first started drawing, it said "I see circle" but kept guessing, and with a few more lines it concluded, "I know, it's sun". Then a few drawings later I got circle, and in one second it said, "I know, it's circle". At first I was thinking, how could it know this is a circle, but for the sun, it knew circle wasn't it when I had only drawn a circle. But it just occurred to me that the program probably tells it whether or not it's right before it actually verbalizes the guess/answer. Then a few drawings after that, I got bus, and it guessed school-bus about 15 seconds in, and it wasn't until 10 seconds after that when it finally guessed just bus. So, I'm guessing this is the same for yours, it was simply guessing police car before car (probably because the NN takes probability into account and has gotten that as a right answer more often or something), and in your case it just happened to be right so it stopped guessing before it tried just "car."
ma2rten 10y ago
I had to draw a car, and I lost because it didn't recognize my drawing and said "police car" instead.
- DanBC 10y ago
  It's interesting to see what it doesn't recognise, especially to compare my failed drawings with the other successful versions. I couldn't tell the difference between my basket and most of the successful baskets, but the robot didn't like my drawing.
shanedrgn 10y ago
It isn't pre programmed to what I understand. All it does is guess what your drawing looks likes based on other peoples drawing, and continues to guess until it is correct. So when you drew a car with no siren, it's guesses could have been truck, car, police car. Since it guessed police car, it got the answer correct. The program wasn't told you had to draw a police car. That would be silly. Why would they create this and lie about how it works? What would be gained?

by daveheq 10y ago

marxidad 10y ago

It's just picking the closest thing from its limited training set.

zipppy 10y ago

I agree and believe that it's pre-programmed in some ways. I took the test twice, and in the first set I had 'zigzag' and in the second set I had 'stairs'. I drew the exact same doodle for each and for each it guessed correctly on its first try.

OJFord 10y ago
No, I believe it makes a series of guesses ranked for 'closeness'. If what you're supposed to be drawing is ranked 'sufficiently high' the game/training side of it moves on to the next.
- zipppy 10y ago
  What? My point is that I drew the same doodle for different words, and for each word it put forth a different _first_ guess. The first guess for each doodle was the word that was provided -- if this software was completely legitimate, I would think it should have guessed the same word for each (and then moved on to other guesses).
  - simias 10y ago
    They weren't really the same though, right? Only similar. A good test for that would be to script the game and give it pixel-identical answers for each questions and see if the result is different.
    - zipppy 10y ago
      Correct that they were not identical, but they were quite close; five lines that looked like a 'w'.
      - simias 10y ago
        FWIW it guessed my lightning correctly (a simple `z` like shape). Its 2nd guess was "zig-zag" and the 3rd one "star" for some reason. So it really looks like for this network a zig-zag and a star are more similar than I would've guessed. I guess a star is a zig zag looping on itself, kinda.
  - mthoms 10y ago
    I wonder if the zig-zag drawing and the stairs were actually as similar as you remember. For instance, our sub-conscious tendency when drawing stairs would be to have straight lines at 0 and 90 degrees but a "zig-zag" wouldn't necessarily have that feature. Thus, even though you thought the drawings were the same, there were sufficient detectable differences for the AI to recognize.

rtkwe 10y ago

It's because it's gathering training data from the things other people are drawing. So if it's just asking for police cars anything car like should just fall into the police car category.

mikeash 10y ago
It also stops you once it correctly identifies the drawing, even if you're far from done. If it incorporates that into its training, then it'll presumably learn to figure out what you meant earlier and earlier. It's interesting to consider whether it would end up making guesses that it sometimes gets correct, or whether it would actually learn to discriminate based on very early stages of drawing. It's conceivable to me that people might start out drawing "police car" differently from "car" in some subtle but detectable way, even if it doesn't involve what we humans would consider to be the distinguishing features of a police car.

asfaltboy 10y ago

I think the QUESTIONS are definitely pre-programmed, so the answer set is probably limited too.

gpsx 10y ago

There is no way it should have recognized my potato.

NiKMic 10y ago

I tested this. It asked for a shovel and I drew an aeroplane and it guessed aeroplane. I did the same thing a number of times and I got around a 60% correct response rate.

quirkafleeg 10y ago

> It seems the answers are pre-programmed because I drew just a car and it said "I know! It's a police car" even though I'd drawn no siren. Exactly the same thing happened to me (I did a CTRL+F on 'police' to find this post), which immediately turned me from 'this is cool' to almost quitting, and coming here to see if anyone else experienced the same thing (perhaps with different images). It also didn't guess my drawing of a bulldozer, which I was surprised at since my drawing wasn't bad, and easily recognizable to any human of any age as a bulldozer. I went back to complete the test, and it tells you what it guessed, which helped to explain both things. Here's what it thinks a bulldozer looks like: http://i.imgur.com/SU9edxi.gif http://i.imgur.com/SU9edxi.gif Here's the difference between "car" and "police car": http://i.imgur.com/YKpANPl.gif http://i.imgur.com/YKpANPl.gif You, like me, obviously don't draw cars from the Soviet Union. Edit: I just noticed (thanks to another post here) it also shows several examples of what it thinks an object looks like, and now I'm not sure why it didn't guess bulldozer, when couple of the images look like mine, certainly far more than the things it guessed.

JangoSteve 10y ago

I thought the same thing. It was a bit more subtle for me, but it prompted me to draw a sun, and when I first started drawing, it said "I see circle" but kept guessing, and with a few more lines it concluded, "I know, it's sun". Then a few drawings later I got circle, and in one second it said, "I know, it's circle". At first I was thinking, how could it know this is a circle, but for the sun, it knew circle wasn't it when I had only drawn a circle. But it just occurred to me that the program probably tells it whether or not it's right before it actually verbalizes the guess/answer. Then a few drawings after that, I got bus, and it guessed school-bus about 15 seconds in, and it wasn't until 10 seconds after that when it finally guessed just bus. So, I'm guessing this is the same for yours, it was simply guessing police car before car (probably because the NN takes probability into account and has gotten that as a right answer more often or something), and in your case it just happened to be right so it stopped guessing before it tried just "car."

ma2rten 10y ago

I had to draw a car, and I lost because it didn't recognize my drawing and said "police car" instead.

DanBC 10y ago
It's interesting to see what it doesn't recognise, especially to compare my failed drawings with the other successful versions. I couldn't tell the difference between my basket and most of the successful baskets, but the robot didn't like my drawing.

shanedrgn 10y ago

It isn't pre programmed to what I understand. All it does is guess what your drawing looks likes based on other peoples drawing, and continues to guess until it is correct. So when you drew a car with no siren, it's guesses could have been truck, car, police car. Since it guessed police car, it got the answer correct. The program wasn't told you had to draw a police car. That would be silly. Why would they create this and lie about how it works? What would be gained?