6 ms·

The network guessed some of the "correct" answers far to quickly. For example, with just an L shape (|_) it guessed suitcase. Feels like the model is suffering

The network guessed some of the "correct" answers far to quickly. For example, with just an L shape (|_) it guessed suitcase. Feels like the model is suffering from overfitting. Fun though.

rolfvandekrol 10y ago
For me a bit less extreme, but it guessed 'police car' when all i drew was a poor car doodle. there was nothing about that car the would make it a police car.
- dagw 10y ago
  I was asked to draw a car and it kept guessing police car, so I guess there is some strange over fitting going on.
- trymas 10y ago
  Similar for me. I was asked to draw 'eyeglasses', kept guessing 'glasses'.
joncalhoun 10y ago
I wonder if it has a very small pool of potential drawings. Eg if it only has "police car" and no other cars it might be able to jump straight to police car after seeing a car. It also seems to constantly have a "best guess" to some degree and if that happens to be correct it confirms pretty quickly.
- dagw 10y ago
  On the flip side of this, I was asked to draw a "telephone". It guessed first "cellphone" and then "phone" before giving up and saying it didn't know.
  - Mayzie 10y ago
    I was asked to draw a "phone". Drew a telephone (the old ones with a rotary dialler, because that was quickest to draw), and it guessed "telephone", but it was incorrect :-( EDIT: Drew a "cake", it guessed "birthday cake". Wrong answer apparently.
    - joncalhoun 10y ago
      It definitely doesn't have any knowledge of similar drawings.
pan69 10y ago
The premise seems to be a little odd, i.e. I tell you what to draw and then I guess what it is. I guess it would have been more of an A.I. challenge if the premise was; draw anything and I'll try to guess what it is.
- gcp 10y ago
  It's weird but the computer can choose not to cheat and forget the answer when guessing. Not possible for humans.
  - csydas 10y ago
    It's possible, but I'm not entirely convinced that this is honest, or at least not really accurate. It's far too eager to try to match it to something than it is to figure out what it has. It could figure out most of my drawings but it would get them well in advance of me completing anything substantial (like others, I would be asked to draw a leg and draw just a curved line and it would guess leg before I finished). Trying to draw what it asked for but with some unusual features (like lines or dot patterns around what it asked for before drawing it) and it gets extremely confused; it doesn't really seem to be good at filtering out any noise: http://imgur.com/a/oE1j2 http://imgur.com/a/oE1j2 (gallery of results and what it thought it saw. Drawing things it didn't ask for just to see what it was guessing resulted in some really strange responses and fits. The answer set it has is extremely limited, so something like a hand giving the horns (\m/) was last guess a duck. A moose was a scorpion, then a duck, then a hand. Godzilla (or a bipedal dinosaur if you prefer) was a vase, then a scorpion, then a boat. My loaf of bread was a washing machine, an anvil, then a postcard. The Deathstar was a bandage, a helicopter, and a lighthouse. And a chainsaw was considered an aircraft. Between the disruptive patterns and drawing things outside of it's vocabulary, the system seems really confused. Looking at the comparison results, I can see how when drawing some things it got it real fast. (Tennis Rackets were mostly defined by a crosshatch pattern, Harps by a series of parallel vertical lines). This makes sense. For other things, not as much. It might be a more convincing presentation to give the user a list of items the machine knows (the full list) and tell the user to try to draw some, and then the computer could check it off as it gets them. That seems like a better way of presenting this than "Draw a box. hey! you drew a box! Isn't that cool?"
tzaman 10y ago
For my L shape, it recognized a leg immediately (which was what I was supposed to draw)
evincarofautumn 10y ago
Yup, the minimalist Picassoesque renditions I was able to do on my laptop trackpad were still recognised as long as I gave a vague impression of the major shapes I figured other people would draw. The result was the kind of image where a human would frown at it until you said “kangaroo”.
Sohakes 10y ago
It's because it gets the most plausible answer, even if it's only a small chance of being correct. If there are not that many words, there are only a few objects that could be draw with a L shape like that. I mean, even if you draw something else starting with a L shape, it will probably be a slight different L shape (with a curve, for example). I think that's it, doesn't really mean it's overfitting, it would mean that if the probability was really high.
oliv__ 10y ago
Yeah it seemed to make the correct guess a liiiitle too fast for me.
Kesty 10y ago
The ai keep throwing guesses in 20 seconds until he got the right one. So if you draw a simple shape it start to go trough the list of things he recognize and end the game there. It works great for this game because he can have very fast answer but only work win the cases when you actually have a feedback that eliminate all the wrong guesses. Basically if it simply went trought the whole english dictionary fast enough he could get the same result without even looking at the pictures.

The network guessed some of the "correct" answers far to quickly. For example, with just an L shape (|_) it guessed suitcase. Feels like the model is suffering from overfitting. Fun though.

rolfvandekrol 10y ago

For me a bit less extreme, but it guessed 'police car' when all i drew was a poor car doodle. there was nothing about that car the would make it a police car.

dagw 10y ago
I was asked to draw a car and it kept guessing police car, so I guess there is some strange over fitting going on.
trymas 10y ago
Similar for me. I was asked to draw 'eyeglasses', kept guessing 'glasses'.

joncalhoun 10y ago

I wonder if it has a very small pool of potential drawings. Eg if it only has "police car" and no other cars it might be able to jump straight to police car after seeing a car. It also seems to constantly have a "best guess" to some degree and if that happens to be correct it confirms pretty quickly.

dagw 10y ago
On the flip side of this, I was asked to draw a "telephone". It guessed first "cellphone" and then "phone" before giving up and saying it didn't know.
- Mayzie 10y ago
  I was asked to draw a "phone". Drew a telephone (the old ones with a rotary dialler, because that was quickest to draw), and it guessed "telephone", but it was incorrect :-( EDIT: Drew a "cake", it guessed "birthday cake". Wrong answer apparently.
  - joncalhoun 10y ago
    It definitely doesn't have any knowledge of similar drawings.

pan69 10y ago

The premise seems to be a little odd, i.e. I tell you what to draw and then I guess what it is. I guess it would have been more of an A.I. challenge if the premise was; draw anything and I'll try to guess what it is.

gcp 10y ago
It's weird but the computer can choose not to cheat and forget the answer when guessing. Not possible for humans.
- csydas 10y ago
  It's possible, but I'm not entirely convinced that this is honest, or at least not really accurate. It's far too eager to try to match it to something than it is to figure out what it has. It could figure out most of my drawings but it would get them well in advance of me completing anything substantial (like others, I would be asked to draw a leg and draw just a curved line and it would guess leg before I finished). Trying to draw what it asked for but with some unusual features (like lines or dot patterns around what it asked for before drawing it) and it gets extremely confused; it doesn't really seem to be good at filtering out any noise: http://imgur.com/a/oE1j2 http://imgur.com/a/oE1j2 (gallery of results and what it thought it saw. Drawing things it didn't ask for just to see what it was guessing resulted in some really strange responses and fits. The answer set it has is extremely limited, so something like a hand giving the horns (\m/) was last guess a duck. A moose was a scorpion, then a duck, then a hand. Godzilla (or a bipedal dinosaur if you prefer) was a vase, then a scorpion, then a boat. My loaf of bread was a washing machine, an anvil, then a postcard. The Deathstar was a bandage, a helicopter, and a lighthouse. And a chainsaw was considered an aircraft. Between the disruptive patterns and drawing things outside of it's vocabulary, the system seems really confused. Looking at the comparison results, I can see how when drawing some things it got it real fast. (Tennis Rackets were mostly defined by a crosshatch pattern, Harps by a series of parallel vertical lines). This makes sense. For other things, not as much. It might be a more convincing presentation to give the user a list of items the machine knows (the full list) and tell the user to try to draw some, and then the computer could check it off as it gets them. That seems like a better way of presenting this than "Draw a box. hey! you drew a box! Isn't that cool?"

tzaman 10y ago

For my L shape, it recognized a leg immediately (which was what I was supposed to draw)

evincarofautumn 10y ago

Yup, the minimalist Picassoesque renditions I was able to do on my laptop trackpad were still recognised as long as I gave a vague impression of the major shapes I figured other people would draw. The result was the kind of image where a human would frown at it until you said “kangaroo”.

Sohakes 10y ago

It's because it gets the most plausible answer, even if it's only a small chance of being correct. If there are not that many words, there are only a few objects that could be draw with a L shape like that. I mean, even if you draw something else starting with a L shape, it will probably be a slight different L shape (with a curve, for example). I think that's it, doesn't really mean it's overfitting, it would mean that if the probability was really high.

oliv__ 10y ago

Yeah it seemed to make the correct guess a liiiitle too fast for me.

Kesty 10y ago

The ai keep throwing guesses in 20 seconds until he got the right one. So if you draw a simple shape it start to go trough the list of things he recognize and end the game there. It works great for this game because he can have very fast answer but only work win the cases when you actually have a feedback that eliminate all the wrong guesses. Basically if it simply went trought the whole english dictionary fast enough he could get the same result without even looking at the pictures.