4 ms·

"'There were 90 markers on her face, and an equivalent amount on her body,' said Cage. 'She delivered the performance in one take.'" Neal Stephenson, populariz

"'There were 90 markers on her face, and an equivalent amount on her body,' said Cage. 'She delivered the performance in one take.'" Neal Stephenson, popularizer/coiner of the term "avatar" to refer to online bodies, seems to have nailed this one, too. Can we just start calling them "'sites" now? Fred Epidermis had put the stage into Constellation Mode. Miranda was looking at a black wall speckled with twenty or thirty thousand individual pricks of white light. Taken together, they formed a sort of three-dimensional constellation of Miranda, moving as she moved. Each point of light marked one of the 'sites that had been poked into her skin by the tat machine during those sixteen hours. Not shown were the filaments that tied them all together into a network— a new bodily system overlaid and interlaced with the nervous, lymph, and vascular systems. ... Outside, Fred Epidermis was wielding the editing controls, zooming in on her face, which was dense as a galactic core. By comparison, her arms and legs were wispy nebulas and the back of her head nearly invisible, with a grand total of maybe a hundred 'sites placed around her scalp like the vertices of a geodesic dome. The eyes were empty holes, except (she imagined) when she closed her eyes. Just to check it out, she winked into the mediatron. The 'sites on her eyelids were dense as grass blades on a putting green, but accordioned together except when the lid expanded over the eye. Perhaps the only thing he got "wrong" was that we might not need a stage. A kinect-like system might be enough to monitor and render the shifting systems from your very own living room.

VikingCoder 15y ago
...why use 90 markers? I saw a demo a while ago where they used Infrared makeup on someone, and used a sponge to make it really splotchy. The makeup doesn't show up in Red, Green, Blue wavelengths. So, they could simultaneously capture RGB, and IR. Using multiple cameras on the IR, and the splotchy makeup, they were able to capture thousands and thousands of points, and get color info at the same time. Seems like the right approach to me. I could totally see a Kinnect-like device taking advantage of this, allowing someone to do facial capture like this at home.
- BGyss 15y ago
  There already are a few papers on Kinect-based 3D face reconstruction (most of them seem to be behind paywalls though). Here's a video: http://youtu.be/nYsqNnDA1l4 http://youtu.be/nYsqNnDA1l4 The infrared makeup (pretty sure the only company that kinda pursued that path was Mova) was kind of unwieldy and it's kind of fallen out of use already. Most facial capture in the future will probably just utilize some kind of depth capture (light field, structured light) in concert with a normal witness camera to reconstruct facial features.
- sp332 15y ago
  I believe that was for The Curious Case of Benjamin Button, which was pretty unusual. See if you can find e.g. the motion capture for Pirates of the Carribean. But you're right that the IR capture seems to make more sense.
- MrScruff 15y ago
  You're probably talking about Mova. Facial motion capture (of that quality) is very different from full body motion capture since the face has so many more degrees of freedom. So generally it is done in an extremely controlled environment with many simultaneous cameras and specific lighting and very little head movement. The problem with doing high quality real time performance capture at home is you can't control the lighting or occlusion - sometimes even a human looking at the footage won't be able to discern what's going on. Not to say it won't ever happen but it will require extremely significant advances in computer vision.

VikingCoder 15y ago

...why use 90 markers? I saw a demo a while ago where they used Infrared makeup on someone, and used a sponge to make it really splotchy. The makeup doesn't show up in Red, Green, Blue wavelengths. So, they could simultaneously capture RGB, and IR. Using multiple cameras on the IR, and the splotchy makeup, they were able to capture thousands and thousands of points, and get color info at the same time. Seems like the right approach to me. I could totally see a Kinnect-like device taking advantage of this, allowing someone to do facial capture like this at home.

BGyss 15y ago
There already are a few papers on Kinect-based 3D face reconstruction (most of them seem to be behind paywalls though). Here's a video: http://youtu.be/nYsqNnDA1l4 http://youtu.be/nYsqNnDA1l4 The infrared makeup (pretty sure the only company that kinda pursued that path was Mova) was kind of unwieldy and it's kind of fallen out of use already. Most facial capture in the future will probably just utilize some kind of depth capture (light field, structured light) in concert with a normal witness camera to reconstruct facial features.
sp332 15y ago
I believe that was for The Curious Case of Benjamin Button, which was pretty unusual. See if you can find e.g. the motion capture for Pirates of the Carribean. But you're right that the IR capture seems to make more sense.
MrScruff 15y ago
You're probably talking about Mova. Facial motion capture (of that quality) is very different from full body motion capture since the face has so many more degrees of freedom. So generally it is done in an extremely controlled environment with many simultaneous cameras and specific lighting and very little head movement. The problem with doing high quality real time performance capture at home is you can't control the lighting or occlusion - sometimes even a human looking at the footage won't be able to discern what's going on. Not to say it won't ever happen but it will require extremely significant advances in computer vision.