6 ms·

I made a quick video explaining how the webapp works: http://youtu.be/Z2ZXW2HBLPM http://youtu.be/Z2ZXW2HBLPM Essentially, using the gridPositions.json to popu

I made a quick video explaining how the webapp works: http://youtu.be/Z2ZXW2HBLPM http://youtu.be/Z2ZXW2HBLPM Essentially, using the gridPositions.json to populate a Voronoi Grid (toggle display of the <canvas> in your browser). Then your cursor position is used against the grid to find which shape encloses it which maps back to the source JPEG which is then displayed.

snowwrestler 14y ago
I watched your video. It does not seem to explain how the fingertip position in each photo was calculated, which it seems to me is really the heart of the magic.
- Flam 14y ago
  He says how it's basically a json file with every image mapped to coords of each voronoi "tile". I guess it was handmade in advance
  - theon144 14y ago
    But then, why do I get a picture of some garden when I move my pointer in the top right corner?
    - gigawatt 14y ago
      You can just barely see the tip of a finger coming in from the right in that one.
  - snowwrestler 14y ago
    Not a bad guess but I'd be interested to actually know the details. Making it all by hand would be problematic for reasons beyond the sheer volume. How would one find a deep, diverse set of images of people pointing? How did the creators ensure they had images to cover all possible pixel positions within a certain proximity? It seems like it would take reviewing many more than 900 images to produce a final set of 900 that includes even coverage.
    - olalonde 14y ago
      Another guess here, but it could have been crowdsourced using Amazon Mechanical Turk. Assuming a conservative 0.05$ by picture, the total cost would be 901 * 0.05 = 45.05$.
      - nmb 14y ago
        Yeah, this seems likely. Here's how I'd do it: create a webapp similar to this one to show to your Turkers. The difference would be that it would show an image at random, and the Turker would decide whether or not someone is pointing in the image, and if so they would click where the person was pointing. For each image, take the mean position of mouseclicks. This becomes a seed for the Voronoi diagram. You can tell he/she "cheated" a bit for some the parts of the canvas that didn't get many seeds. For example, if you move your mouse within one of the large cells in the bottom left, the image just moves to keep up with the pointer. :P
      - cheatercheater 14y ago
        But then everyone in the photos would be Indian.
    - BHSPitMonkey 14y ago
      They could have had help from friends. Also, the design of the application makes it very easy to add new images to the set (just throw a new entry into the JSON file containing the x and y coordinates of the finger, and the script does the rest whenever the user loads the page).
    - bnegreve 14y ago
      > How did the creators ensure they had images You don't need a different picture for every single pixel: if the picture is large enough, you can easily crop it and use it to cover few thousands pixels.
- joejohnson 14y ago
  Yes, I was wondering the same thing. The difficult engineering problem here is first assigning pointing locations for every photo in the database. Because he didn't explain this step, I guess he just manually assigned each location for every photo.
  - Andi 14y ago
    That's it, the developer has defined the pointing position for every photo manually.
    - twymer 14y ago
      That's what I assumed until I pointed to ~20x20px from the top right corner and got a picture of a house in a flood.
paulgb 14y ago
If anyone doesn't care to explore the DOM themselves, this will show the Voronoi diagram and make the photo on top of it transparent. For Chrome, Ctrl+Shift+J and paste it into the console. document.getElementById('data').setAttribute('style',''); document.getElementById('content').setAttribute('style','opacity:0.7;left:0;top:0;margin:0'); document.getElementById('underlay').setAttribute('style','left:0;top:0;margin:0');

snowwrestler 14y ago

I watched your video. It does not seem to explain how the fingertip position in each photo was calculated, which it seems to me is really the heart of the magic.

Flam 14y ago
He says how it's basically a json file with every image mapped to coords of each voronoi "tile". I guess it was handmade in advance
- theon144 14y ago
  But then, why do I get a picture of some garden when I move my pointer in the top right corner?
  - gigawatt 14y ago
    You can just barely see the tip of a finger coming in from the right in that one.
- snowwrestler 14y ago
  Not a bad guess but I'd be interested to actually know the details. Making it all by hand would be problematic for reasons beyond the sheer volume. How would one find a deep, diverse set of images of people pointing? How did the creators ensure they had images to cover all possible pixel positions within a certain proximity? It seems like it would take reviewing many more than 900 images to produce a final set of 900 that includes even coverage.
  - olalonde 14y ago
    Another guess here, but it could have been crowdsourced using Amazon Mechanical Turk. Assuming a conservative 0.05$ by picture, the total cost would be 901 * 0.05 = 45.05$.
    - nmb 14y ago
      Yeah, this seems likely. Here's how I'd do it: create a webapp similar to this one to show to your Turkers. The difference would be that it would show an image at random, and the Turker would decide whether or not someone is pointing in the image, and if so they would click where the person was pointing. For each image, take the mean position of mouseclicks. This becomes a seed for the Voronoi diagram. You can tell he/she "cheated" a bit for some the parts of the canvas that didn't get many seeds. For example, if you move your mouse within one of the large cells in the bottom left, the image just moves to keep up with the pointer. :P
    - cheatercheater 14y ago
      But then everyone in the photos would be Indian.
  - BHSPitMonkey 14y ago
    They could have had help from friends. Also, the design of the application makes it very easy to add new images to the set (just throw a new entry into the JSON file containing the x and y coordinates of the finger, and the script does the rest whenever the user loads the page).
  - bnegreve 14y ago
    > How did the creators ensure they had images You don't need a different picture for every single pixel: if the picture is large enough, you can easily crop it and use it to cover few thousands pixels.
joejohnson 14y ago
Yes, I was wondering the same thing. The difficult engineering problem here is first assigning pointing locations for every photo in the database. Because he didn't explain this step, I guess he just manually assigned each location for every photo.
- Andi 14y ago
  That's it, the developer has defined the pointing position for every photo manually.
  - twymer 14y ago
    That's what I assumed until I pointed to ~20x20px from the top right corner and got a picture of a house in a flood.

paulgb 14y ago

If anyone doesn't care to explore the DOM themselves, this will show the Voronoi diagram and make the photo on top of it transparent. For Chrome, Ctrl+Shift+J and paste it into the console. document.getElementById('data').setAttribute('style',''); document.getElementById('content').setAttribute('style','opacity:0.7;left:0;top:0;margin:0'); document.getElementById('underlay').setAttribute('style','left:0;top:0;margin:0');