11 ms·

A Canvas Made of Pixels

525 points by robk 11y ago

Tepix 11y ago
Nice job! If you have a camera that tracks the viewer, you can also make the image it displays 3D as seen in this video from 2007: https://youtu.be/Jd3-eiid-Uw?t=2m46s https://youtu.be/Jd3-eiid-Uw?t=2m46s (back then it used a Wiimote to track the head position).
- rhaps0dy 11y ago
  Do you think a camera would be precise enough? This looks pretty awesome.
  - gmac 11y ago
    I had a quick go at this a few months back using Apple's CIDetector[1], and even at 'high' accuracy that didn't seem precise enough — it was distractingly jiggly. [1] [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:@{CIDetectorAccuracy: CIDetectorAccuracyHigh, CIDetectorTracking: @YES}]
- Jack000 11y ago
  up close the lack of parallax would give it away, but I think it would work ok if the viewer is sufficiently far away.
Phemist 11y ago
>> A fun (but impractical and frustrating) variant of this feature is to have the image change as soon as the viewer looks away. So you’re looking at a painting, glance away to another room, and look back to find a new painting hanging on the wall. Re this comment. There is this thing called "saccadic masking", the gist of which is that we are effectively blind while our eyes move from focusing on one position to the next. Depending on the "distance" (angle) traveled by the eye, this can last up to tens of milliseconds. Enough time to do some cool stuff! One of the studies that conclusively showed this effect, a fairly long time ago, had participants wear some elaborate headgear that allowed the researchers to track where the eyes of their participants were looking at. They had them look at a standard sentence like "the quick brown fox jumped over the lazy dog", but with a twist! Every word in the sentence was masked, except for the word the participant would be focusing on at that moment. So if he focused on the word fox, the screen would show "xxx xxxxx xxxxx fox xxxxxx xxxx xxx xxxx xxx". Whenever the system detected a saccade by the eyes, they would recalculate which word the participant would be looking at (e.g. switch to dog), and changed the display of the screen accordingly, now showing "xxx xxxxx xxxxx xxx xxxxxx xxxx xxx xxxx dog" Participants were asked if they noticed anything strange about the sentence, but they reported there was nothing strange about it! (Disregarding having a heavy set of mirrors strapped to your head). My point: If you were able to detect saccades of the eyes, and relatively accurately calculate their position, you could have the region of the image that's unattended be colourful noise, the theory says that your visitors would be none the wiser (of course this would break down with multiple people looking at the same image) Even cooler: procedurally re-generate parts of the image that are unattended, so that you're looking at an ever shifting image, but wouldn't quite be able to pin down what's happening. Similar to this video: https://www.youtube.com/watch?v=ubNF9QNEQLA https://www.youtube.com/watch?v=ubNF9QNEQLA
- cocoflunchy 11y ago
  This is also the principle behind foveated rendering (https://en.wikipedia.org/wiki/Foveated_imaging https://en.wikipedia.org/wiki/Foveated_imaging) where you only render the graphics in high detail where the viewer is looking, and use a low-res image for the rest with the goal of saving computing power. Big area of development right now for VR!
  - Phemist 11y ago
    That's really cool! I'm not 100% sure, but what I remember from reading a few psych papers about this is that as long as the "mask" is similar to the original in a few easy to calculate sampling statistics (colour distribution, overall hue), it doesn't actually need to look anything like the original image, even calculating low-res imagery shouldn't be necessary. Our peripheral vision is just that bad (at least for static imagery, movement detection is actually pretty good). The foveated part is where the magic happens, and we've gotten pretty good at fooling ourself into believing this "foveated" part is much larger than it actually is.
    - kedean 11y ago
      Hey, something I know about! I actually worked on one of those research projects in college, programming the experiments. The idea is that there is a certain radius around the focal point where you stop being able to detect changes. I'm not sure what the final results were, but the theory was that you can calculate how blurry an image can be and still be discernable based on how far it is from the focal point of your vision. It was surprising how good people are at detecting changes in a blurry picture that's way out in their periphery.
      - Phemist 11y ago
        My prof used to joke that, whenever he'd discuss fovea and periphery, that if we had a fovea the size of our full field of view, we'd need a brain the size of an elephant to process all that information. It's interesting how we're very sensitive to sudden changes (thus movement) in our periphery, but are so bad at classifying/identifying static imagery. I remember reading about right eye and left eye dominance. Where they'd keep an image on the screen saccade invariant (ie, compensate for any saccades that were made). Slowing moving a letter/character/word/whatever to the edges of the participant's field of view and asking when the character was no longer legible. This happened surprisingly quickly, but at different positions for the left eye and right eye for pretty much all participants..
tomp 11y ago
So... when will you start a Kickstarter campaign?
- deleted 11y ago
  [deleted]
alex_duf 11y ago
A cool effect would be to have the painting actually looking at you using face recognition. Freaky.
- ddeck 11y ago
  An excellent idea. Perhaps combined with the look away detection mentioned in the article, such that the painting only tracks you when you aren't looking directly at it. Reminiscent of many a horror movie.
- tgb 11y ago
  Unless I'm mistaken, the way for a painting to actually look at you is unfortunately just to have the pupils dead ahead. Perspective on the painting then does the rest. Otherwise this would never even have been a cliche to riff on.
eliaskg 11y ago
Hogwarts paintings!
- nkrisc 11y ago
  Almost! Still need to network them.
Tinyyy 11y ago
Wow this is incredibly cool! I love the shy feature especially, to screw with the viewer :P One question I have, though, is that since the aspect ratio of the screen is fixed, does that mean that all the photos have to be rescaled? That is kinda inelegant, I wonder if there's a way to fix it.
- madaxe_again 11y ago
  I think the mat around the photo is also rendered, so you can have whatever aspect you want - although a portrait image might look a bit lost in the middle of it. Next step - mount the frame on a motorised bearing so it can rotate 90 degrees depending on the image aspect!
  - tobr 11y ago
    I don't think the mat is rendered. In the video with changing light conditions, you can see how the picture lags slightly, while the mat reacts immediately.
    - Zarel 11y ago
      The mat is also used to make the aspect ratio less extreme, so it can't be rendered.
- claybavor 11y ago
  Yes, I have to crop the images to the aspect ratio of the display. I considered doing a simulated digital mat which could change given the aspect ratio of the work being shown, but it wasted so much of the display that it was not worth it. I end up not having to crop much, because the aspect ratio I chose lines up pretty well with those of many good paintings and photographs.
chrismbarr 11y ago
I'm really surprised there's no photos of the build process. It's hard to tell if this is actually what is being described, or just a blog post around a photo of a framed image.
- ohitsdom 11y ago
  Build photos would have been really cool. Also, some detail on the software would be nice. I'm assuming it's running on OSX, but I still have lots of questions (what language, how can you change the picture, how does he manage it remotely, etc).
  - claybavor 11y ago
    First of all, yes the frame really exists. :) If I get around to pulling together build photos (which by the way aren't that exciting), I'll post them here. For now, just imagine a big hole in my wall (I had to reframe it), a photodiode sticking out of a hole on the right side of the picture frame, and an iMac. The software is just a simple Objective-C / Cocoa app running on Mac OS X. I use a Kensington presenter clicker to advance the images to change them remotely. I drop new images into a Google Drive folder from another Mac, which the picture frame automatically picks up and puts into the image rotation. And if I want to make major changes, I just use a wireless keyboard and mouse to work on the frame like I would a normal iMac. It is very unsophisticated.
- deleted 11y ago
  [deleted]
- roywiggins 11y ago
  There's a video further down demonstrating it adjusting to different lighting conditions; you can see a slight lag.
deleted 11y ago
[deleted]
m1sta_ 11y ago
I use a printer.
- glaberficken 11y ago
  you tree killer :P
dexterdog 11y ago
I was having a conversation regarding these things just a few days ago. I always loved the idea of a digital frame and have had many over the years, but no devices really nailed it and backlit was always the wrong way to go. Refresh time is not an issue at all as you would refresh on inactivity if you really don't want people to see the sausage made. Feeding data to these things is trivial.
Aandre 11y ago
Lionel Messi wins 5th time Ballon d’Or trophy http://goo.gl/D5rX2A http://goo.gl/D5rX2A
cabirum 11y ago
A color hi-res e-ink display would be a much better fit for the project: it requires power only to change the image and doesn't need light sensors. Too bad e-ink isn't being developed anymore.
- mschaef 11y ago
  > Too bad e-ink isn't being developed anymore. Has development seriously stalled? I'd thought Amazon was developing it for the non-LCD Kindles. (At least I thought the Voyager represented a relatively recent step forward for eInk.)
- Zenst 11y ago
  http://www.eink.com/ http://www.eink.com/ appear to be doing development and a year ago releasead a colour eink display. Alas the resolution and 4096 colour range is somewhat adrift from the requirements of such a use as in photo realism. Indeed the market that will boost this area will be supermarket labeling upon shelfs and the like. Even recently a few bus stops in London utilising eink for displaying a bus timetible http://www.gizmodo.co.uk/2015/12/tfls-trial-of-clever-eink-london-bus-timetables-is-frustratingly-limited/ http://www.gizmodo.co.uk/2015/12/tfls-trial-of-clever-eink-l... But totaly agree it is aperfect fit for such display technology, alas the technology is still a long way off being suitable for this niche usage. Albeit the whole refresh aspect and power usage make it not ideal for current technology, at least the aspect of screen burn-in and ghosting are less considerations from the old CRT days (though they could do a better colour range technology wise).
anarchitect 11y ago
Nicely done, though I would have loved to have seen some build pictures. There are a number of commercial players in this market (for example https://frm.fm/ https://frm.fm/) which combine hardware and marketplace, and the geek in me would love to see these succeed. I work in e-commerce selling art reproductions and have thought a lot about this. I believe as soon as you make the art ephemeral or changeable, its value (to you, rather than monetary) is diminished and it simply becomes a means of displaying pixels. I jumped at the chance to digitise my music collection, I just can't imagine doing the same for visual art.
- aeling 11y ago
  Interesting, I'd never seen these before. I can't seem to find any info on pricing or availability on Framed's site, though; am I missing something?
- otakucode 11y ago
  I would expect there are people who lovingly care for their collection of vinyl records and find the desire to digitize a music collection equally unimaginable.
  - anarchitect 11y ago
    Granted, though I think the difference is even more pronounced when the primary purpose of the physical object is to be appreciated visually. Not to detract from the visual appeal of record sleeves, but the primary goal is to house the album. I would also argue that sound can be reproduced digitally in a way that closely resembles the analog equivalent, but the same is not true of displaying art on screens vs on actual paper/canvas (despite the impressive efforts in the linked post).
    - TheOtherHobbes 11y ago
      Some issues: Paint on any medium is a slightly sculptural 3D. You can look at a masterpiece from four hundred years ago and see the brushstrokes. The reflection from the relief map changes as you move your head. Fine art inkjet prints are more subtly textured, because museum-grade paper is never perfectly flat. A lot of art is huge - many feet by many feet. The giant displays needed to reproduce it don't exist yet. But the biggest problem with electronic frames is working out how to power them without trailing an obvious power lead to the floor. You can solve this problem by plastering the display into the wall, or by building a false wall, or by ignoring it. None of those are ideal. I'd love to see a practical solution, but I'm not sure one is possible with current technology - although it may get closer with the new film displays that are appearing. I like the light sensing here though. That's a clever solution.
benbenolson 11y ago
He says in the article that "Apple did a good job with their monitor", but what he fails to realize is that the panel in that monitor is made by Samsung, and is used in numerous other monitors, such as a bunch of Dell Ultrasharp monitors. These are the exact same monitor, except with a different frame and stand, and are available for a fraction of the cost.
- RussianCow 11y ago
  Actually, unless it's changed with the 5k screen, the iMac displays are made by LG. Samsung makes the display for the iPad.
- kimolas 11y ago
  Which monitor would you recommend that uses the same display as the 5k iMac? Apparently Samsung made one half a year ago that costs as much as the iMac itself.
  - mayoff 11y ago
    Dell's 5k monitor has been as low as $1440 at amazon: http://camelcamelcamel.com/Dell-Monitor-UP2715K-27-Inch-LED-Lit/product/B00OKSFXZU http://camelcamelcamel.com/Dell-Monitor-UP2715K-27-Inch-LED-... Not that I would recommend it over the iMac for this project.
- guelo 11y ago
  Yea but the integrated quiet computer was also important for his project. Mounting a separate computer would have added a lot of complication.
  - yincrash 11y ago
    He does not say this. In fact, at the end of the article, he writes that having the whole iMac is fairly impractical and having a smaller Android system on a chip would be the better solution.
- vernie 11y ago
nathan_f77 11y ago
This is amazing! I had the same idea a few years ago but never got around to it. I was planning to just use some regular LCD screens and give them some nice frames. Probably wouldn't have looked that great, though. It's going to have to wait though, I'm living in an apartment now and can't really cut huge holes in our walls. Even if everything could fit inside the frame, it would be pretty ugly if you couldn't hide the power cable.
theon144 11y ago
Awesome! I really love the usage of the photodiode, as well as the camera, both are pretty thoughtful finishing touches that elevate this to more than "a cool digital frame". How much does the display cost, if I were to make a frame like that too?
- morsch 11y ago
  The cheapest 5k iMac is $1799, other comparable displays are roughly in the same ballpark. I imagine the other items (frame, sensors) are trivial compared to that expense. Replacement 5k panels for the iMac (LM270WQ1) are merely $800 on AliExpress, but you'd have to figure out how to output to them.
stevewilhelm 11y ago
"Visitors to Microsoft chairman Gates' new mansion in Medina will find themselves in an electronic Disneyland where they will be able to "call up" a favorite painting or photograph on combination TV-computer screens. "Right now, only a billionaire could afford to try $80,000 monitors and $150,000 computer-storage devices (the house could have several) that may be needed to make this all work." from "Bill Gates: 50,000 Computer Paintings On Call" (1991) http://community.seattletimes.nwsource.com/archive/?date=19910526&slug=1285524 http://community.seattletimes.nwsource.com/archive/?date=199...
- otakucode 11y ago
  I remember reading about the various technological features Gates put in when he built his home. I would be really interested to read/see (video would be awesome) a followup piece about whether many of these things either required regular upgrading or else have simply become antiquated. I would guess that his family kept upgrading the home, but details would be interesting.
- morgante 11y ago
  And now you only need to be a millionaire.
mtone 11y ago
I'm a big fan of flux, but brightness adjustment is another story. On my iPad, I quickly turned it off. The auto-brightness never hit the spot I wanted, and switched too strongly and too late. Reading on the bus, the brightness basically goes berserk at every shadow and never settles. At home it would be more manageable, but unfortunately desktop monitors don't support brightness adjustments as far as I know -- really wish it was standard through HDMI. Having to put an entire iMac computer on the wall to have access to brightness adjustments (at least that's my understanding of the solution) makes it a lot less practical. But kudos to the author for addressing that essential part of the problem.
- mayoff 11y ago
  Keep in mind that the cheapest 5K iMac is in the same price range as standalone 5K monitors. So you might as well use the iMac unless you can get a 5K monitor at a pretty steep discount.
otakucode 11y ago
I continue impatiently waiting for a large format true color electronic paper display at a quasi-reasonable price. I have had a different project idea for such a thing for several years now, but the wait continues. I imagine such a display would be excellent for the use described in this article. No need to mess about with matching light levels (although doing that was super cool!), battery would only need charged every few weeks or months, etc. There are actually a couple products that seem to be what I want, but for reasons I do not understand the image displayed can not be changed. I have no idea why someone would buy electronic paper signage whose image has to be provided to the manufacturer and which can not be changed later. I would expect a simple printed sign would be superior for any use that would be put to.
peter303 11y ago
Constrast is the lagging factor in good computer displays. These are called High Dynamic Range (HDR) displays. The claim is the eye can see a contrast factor of about a million at any one time, and a billion switching between sunny daytime and darkest night. An end-to-end camera-display system would need 20 bit amplitudes represent such a dynamic range. Many current systems are just 8 or 12 bits. SIGGRAPH often exhibits high end experimental displays. 4K resolution is close to what the eye can see. Four or six basis color systems improve color capture. But I generally just see an improvement in nature scenes. The huge win is HDR. A good HDR display is almost indistinguishable from looking through a window.
- alok-g 11y ago
  https://en.wikipedia.org/wiki/Human_eye#Dynamic_range https://en.wikipedia.org/wiki/Human_eye#Dynamic_range >> 4K resolution is close to what the eye can see. Depends on the distance to the display (whose resolution is independent on where the viewer is).
KingMob 11y ago
Thisi s cool, but to really reproduce art, you have to avoid RGB-based displays. I've always assumed the reason wide-gamut/CIE Lab*-influenced displays are uncommon is that RGB is "good enough" for most purposes outside of image-editing/printing. But they can only show a fraction of the colors humans can see. IIRC, the standard sRGB space used in most monitors can only show a third of the human-visible spectrum.
- morsch 11y ago
  Yes, that seems like a huge limitation. Though the 5k iMac is wide gamut, so quite a bit better than sRGB. Are there even (consumer?) displays that have a significantly wider gamut? http://blog.conradchavez.com/2015/10/26/a-look-at-the-p3-color-gamut-of-the-imac-display-retina-late-2015/ http://blog.conradchavez.com/2015/10/26/a-look-at-the-p3-col...
  - KingMob 11y ago
    Kudos to Apple for making the new iMacs wide-gamut. But I don't know that it's too common in the consumer space yet, mostly because once you start using multiple color spaces, you run into the problem of images created in one color space being shown in another. If the image data isn't tagged correctly, things will look over/under-saturated on different monitors. (E.g., is rgb(255, 0, 0) the reddest red in sRGB or the reddest red in P3/Adobe RGB?) I suspect that when monitor manufacturers tire of competing on pixel density, brightness, 3D, and size, they'll eventually get around to competing on gamut. Maybe Apple will lead the way. A lot of photo/video cameras already capture a larger gamut, so monitors are the last piece of the puzzle.
    - mark-r 11y ago
      Unfortunately as the gamut gets larger, so does the visual difference between adjacent colors. You need more than 8 bits per channel from end to end if you're going to use anything wider than sRGB. Any RGB based color space will be triangular shaped; if it's to be physically realized in a display, all 3 vertices must lie within the CIE gamut. Since the CIE gamut isn't perfectly triangular you will always be missing some colors outside of the triangle no matter what you do. You could do much better by adding a 4th primary to create a trapezoid, which would more completely fill the space. As the entire industry is RGB based this would be completely impractical, but it would be fun to see as a demo.
callmeed 11y ago
If I pull an LCD panel out of an old laptop, how do I connect it to something small like a raspberry pi or beaglebone? My understanding was that many lcd panels required custom drivers/firmware that was often embedded in the laptop/display board.
- Palomides 11y ago
  search for the part number on ebay and you'll probably find a driver board with vga or hdmi input, or a board that you can buy and ask the seller to program it for a specific display. Usually less than $30, but you'll need power, a case, etc..
SeanDav 11y ago
My first thought when I started reading was "wow what a waste of time and money to display a stupid picture" but by the time I was near the end of the article, my attitude had turned 180 degrees. This is amazing attention to detail and craftsmanship in the traditional sense. I can only admire the process - really impressed.
spdustin 11y ago
It brings to mind the "moving paintings" in the ride wait queues for "Harry Potter and the Forbidden Journey" (a dark ride with a moving car on an articulated arm). I saw those things up close and they had a wonderful canvas texture laid over the screens. It really seemed magical -- though I admit I'm one to give myself over to the magic of theme parks when I can -- and it made me wonder if this effect could be reproduced by using a matte silicone mold of actual canvas or something like that. As an aside, there is a lot of amazing technology in play in that ride; I read an unauthorized treatment of how the ride works and I was blown away at the systems/mechanical orchestration necessary to pull it off. It's no wonder it breaks down all the time.
frikk 11y ago
It's too bad this isn't on github. I've always wanted to make something similar, but using computational art instead of paintings. My dream is to make a thing that slowly adapts to the room its in, but doesn't seem like a computer hanging on a wall. Does anyone know where I can learn more about the lighting techniques described here?
rkalra4 11y ago
Anti-"Shy mode" with all such frames connected over intranet would be really cool!
hoorayimhelping 11y ago
Fantastic read, and a great idea. Very inspiring, thank you for posting it.

525 points by robk 11y ago

Tepix 11y ago

Nice job! If you have a camera that tracks the viewer, you can also make the image it displays 3D as seen in this video from 2007: https://youtu.be/Jd3-eiid-Uw?t=2m46s https://youtu.be/Jd3-eiid-Uw?t=2m46s (back then it used a Wiimote to track the head position).

rhaps0dy 11y ago
Do you think a camera would be precise enough? This looks pretty awesome.
- gmac 11y ago
  I had a quick go at this a few months back using Apple's CIDetector[1], and even at 'high' accuracy that didn't seem precise enough — it was distractingly jiggly. [1] [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:@{CIDetectorAccuracy: CIDetectorAccuracyHigh, CIDetectorTracking: @YES}]
Jack000 11y ago
up close the lack of parallax would give it away, but I think it would work ok if the viewer is sufficiently far away.

Phemist 11y ago

>> A fun (but impractical and frustrating) variant of this feature is to have the image change as soon as the viewer looks away. So you’re looking at a painting, glance away to another room, and look back to find a new painting hanging on the wall. Re this comment. There is this thing called "saccadic masking", the gist of which is that we are effectively blind while our eyes move from focusing on one position to the next. Depending on the "distance" (angle) traveled by the eye, this can last up to tens of milliseconds. Enough time to do some cool stuff! One of the studies that conclusively showed this effect, a fairly long time ago, had participants wear some elaborate headgear that allowed the researchers to track where the eyes of their participants were looking at. They had them look at a standard sentence like "the quick brown fox jumped over the lazy dog", but with a twist! Every word in the sentence was masked, except for the word the participant would be focusing on at that moment. So if he focused on the word fox, the screen would show "xxx xxxxx xxxxx fox xxxxxx xxxx xxx xxxx xxx". Whenever the system detected a saccade by the eyes, they would recalculate which word the participant would be looking at (e.g. switch to dog), and changed the display of the screen accordingly, now showing "xxx xxxxx xxxxx xxx xxxxxx xxxx xxx xxxx dog" Participants were asked if they noticed anything strange about the sentence, but they reported there was nothing strange about it! (Disregarding having a heavy set of mirrors strapped to your head). My point: If you were able to detect saccades of the eyes, and relatively accurately calculate their position, you could have the region of the image that's unattended be colourful noise, the theory says that your visitors would be none the wiser (of course this would break down with multiple people looking at the same image) Even cooler: procedurally re-generate parts of the image that are unattended, so that you're looking at an ever shifting image, but wouldn't quite be able to pin down what's happening. Similar to this video: https://www.youtube.com/watch?v=ubNF9QNEQLA https://www.youtube.com/watch?v=ubNF9QNEQLA

cocoflunchy 11y ago
This is also the principle behind foveated rendering (https://en.wikipedia.org/wiki/Foveated_imaging https://en.wikipedia.org/wiki/Foveated_imaging) where you only render the graphics in high detail where the viewer is looking, and use a low-res image for the rest with the goal of saving computing power. Big area of development right now for VR!
- Phemist 11y ago
  That's really cool! I'm not 100% sure, but what I remember from reading a few psych papers about this is that as long as the "mask" is similar to the original in a few easy to calculate sampling statistics (colour distribution, overall hue), it doesn't actually need to look anything like the original image, even calculating low-res imagery shouldn't be necessary. Our peripheral vision is just that bad (at least for static imagery, movement detection is actually pretty good). The foveated part is where the magic happens, and we've gotten pretty good at fooling ourself into believing this "foveated" part is much larger than it actually is.
  - kedean 11y ago
    Hey, something I know about! I actually worked on one of those research projects in college, programming the experiments. The idea is that there is a certain radius around the focal point where you stop being able to detect changes. I'm not sure what the final results were, but the theory was that you can calculate how blurry an image can be and still be discernable based on how far it is from the focal point of your vision. It was surprising how good people are at detecting changes in a blurry picture that's way out in their periphery.
    - Phemist 11y ago
      My prof used to joke that, whenever he'd discuss fovea and periphery, that if we had a fovea the size of our full field of view, we'd need a brain the size of an elephant to process all that information. It's interesting how we're very sensitive to sudden changes (thus movement) in our periphery, but are so bad at classifying/identifying static imagery. I remember reading about right eye and left eye dominance. Where they'd keep an image on the screen saccade invariant (ie, compensate for any saccades that were made). Slowing moving a letter/character/word/whatever to the edges of the participant's field of view and asking when the character was no longer legible. This happened surprisingly quickly, but at different positions for the left eye and right eye for pretty much all participants..

tomp 11y ago

So... when will you start a Kickstarter campaign?

deleted 11y ago
[deleted]

alex_duf 11y ago

A cool effect would be to have the painting actually looking at you using face recognition. Freaky.

ddeck 11y ago
An excellent idea. Perhaps combined with the look away detection mentioned in the article, such that the painting only tracks you when you aren't looking directly at it. Reminiscent of many a horror movie.
tgb 11y ago
Unless I'm mistaken, the way for a painting to actually look at you is unfortunately just to have the pupils dead ahead. Perspective on the painting then does the rest. Otherwise this would never even have been a cliche to riff on.

eliaskg 11y ago

Hogwarts paintings!

nkrisc 11y ago
Almost! Still need to network them.

Tinyyy 11y ago

Wow this is incredibly cool! I love the shy feature especially, to screw with the viewer :P One question I have, though, is that since the aspect ratio of the screen is fixed, does that mean that all the photos have to be rescaled? That is kinda inelegant, I wonder if there's a way to fix it.

madaxe_again 11y ago
I think the mat around the photo is also rendered, so you can have whatever aspect you want - although a portrait image might look a bit lost in the middle of it. Next step - mount the frame on a motorised bearing so it can rotate 90 degrees depending on the image aspect!
- tobr 11y ago
  I don't think the mat is rendered. In the video with changing light conditions, you can see how the picture lags slightly, while the mat reacts immediately.
  - Zarel 11y ago
    The mat is also used to make the aspect ratio less extreme, so it can't be rendered.
claybavor 11y ago
Yes, I have to crop the images to the aspect ratio of the display. I considered doing a simulated digital mat which could change given the aspect ratio of the work being shown, but it wasted so much of the display that it was not worth it. I end up not having to crop much, because the aspect ratio I chose lines up pretty well with those of many good paintings and photographs.

chrismbarr 11y ago

I'm really surprised there's no photos of the build process. It's hard to tell if this is actually what is being described, or just a blog post around a photo of a framed image.

ohitsdom 11y ago
Build photos would have been really cool. Also, some detail on the software would be nice. I'm assuming it's running on OSX, but I still have lots of questions (what language, how can you change the picture, how does he manage it remotely, etc).
- claybavor 11y ago
  First of all, yes the frame really exists. :) If I get around to pulling together build photos (which by the way aren't that exciting), I'll post them here. For now, just imagine a big hole in my wall (I had to reframe it), a photodiode sticking out of a hole on the right side of the picture frame, and an iMac. The software is just a simple Objective-C / Cocoa app running on Mac OS X. I use a Kensington presenter clicker to advance the images to change them remotely. I drop new images into a Google Drive folder from another Mac, which the picture frame automatically picks up and puts into the image rotation. And if I want to make major changes, I just use a wireless keyboard and mouse to work on the frame like I would a normal iMac. It is very unsophisticated.
deleted 11y ago
[deleted]
roywiggins 11y ago
There's a video further down demonstrating it adjusting to different lighting conditions; you can see a slight lag.

deleted 11y ago

[deleted]

m1sta_ 11y ago

I use a printer.

glaberficken 11y ago
you tree killer :P

dexterdog 11y ago

I was having a conversation regarding these things just a few days ago. I always loved the idea of a digital frame and have had many over the years, but no devices really nailed it and backlit was always the wrong way to go. Refresh time is not an issue at all as you would refresh on inactivity if you really don't want people to see the sausage made. Feeding data to these things is trivial.

Aandre 11y ago

Lionel Messi wins 5th time Ballon d’Or trophy http://goo.gl/D5rX2A http://goo.gl/D5rX2A

cabirum 11y ago

A color hi-res e-ink display would be a much better fit for the project: it requires power only to change the image and doesn't need light sensors. Too bad e-ink isn't being developed anymore.

mschaef 11y ago
> Too bad e-ink isn't being developed anymore. Has development seriously stalled? I'd thought Amazon was developing it for the non-LCD Kindles. (At least I thought the Voyager represented a relatively recent step forward for eInk.)
Zenst 11y ago
http://www.eink.com/ http://www.eink.com/ appear to be doing development and a year ago releasead a colour eink display. Alas the resolution and 4096 colour range is somewhat adrift from the requirements of such a use as in photo realism. Indeed the market that will boost this area will be supermarket labeling upon shelfs and the like. Even recently a few bus stops in London utilising eink for displaying a bus timetible http://www.gizmodo.co.uk/2015/12/tfls-trial-of-clever-eink-london-bus-timetables-is-frustratingly-limited/ http://www.gizmodo.co.uk/2015/12/tfls-trial-of-clever-eink-l... But totaly agree it is aperfect fit for such display technology, alas the technology is still a long way off being suitable for this niche usage. Albeit the whole refresh aspect and power usage make it not ideal for current technology, at least the aspect of screen burn-in and ghosting are less considerations from the old CRT days (though they could do a better colour range technology wise).

anarchitect 11y ago

Nicely done, though I would have loved to have seen some build pictures. There are a number of commercial players in this market (for example https://frm.fm/ https://frm.fm/) which combine hardware and marketplace, and the geek in me would love to see these succeed. I work in e-commerce selling art reproductions and have thought a lot about this. I believe as soon as you make the art ephemeral or changeable, its value (to you, rather than monetary) is diminished and it simply becomes a means of displaying pixels. I jumped at the chance to digitise my music collection, I just can't imagine doing the same for visual art.

aeling 11y ago
Interesting, I'd never seen these before. I can't seem to find any info on pricing or availability on Framed's site, though; am I missing something?
otakucode 11y ago
I would expect there are people who lovingly care for their collection of vinyl records and find the desire to digitize a music collection equally unimaginable.
- anarchitect 11y ago
  Granted, though I think the difference is even more pronounced when the primary purpose of the physical object is to be appreciated visually. Not to detract from the visual appeal of record sleeves, but the primary goal is to house the album. I would also argue that sound can be reproduced digitally in a way that closely resembles the analog equivalent, but the same is not true of displaying art on screens vs on actual paper/canvas (despite the impressive efforts in the linked post).
  - TheOtherHobbes 11y ago
    Some issues: Paint on any medium is a slightly sculptural 3D. You can look at a masterpiece from four hundred years ago and see the brushstrokes. The reflection from the relief map changes as you move your head. Fine art inkjet prints are more subtly textured, because museum-grade paper is never perfectly flat. A lot of art is huge - many feet by many feet. The giant displays needed to reproduce it don't exist yet. But the biggest problem with electronic frames is working out how to power them without trailing an obvious power lead to the floor. You can solve this problem by plastering the display into the wall, or by building a false wall, or by ignoring it. None of those are ideal. I'd love to see a practical solution, but I'm not sure one is possible with current technology - although it may get closer with the new film displays that are appearing. I like the light sensing here though. That's a clever solution.

benbenolson 11y ago

He says in the article that "Apple did a good job with their monitor", but what he fails to realize is that the panel in that monitor is made by Samsung, and is used in numerous other monitors, such as a bunch of Dell Ultrasharp monitors. These are the exact same monitor, except with a different frame and stand, and are available for a fraction of the cost.

RussianCow 11y ago
Actually, unless it's changed with the 5k screen, the iMac displays are made by LG. Samsung makes the display for the iPad.
kimolas 11y ago
Which monitor would you recommend that uses the same display as the 5k iMac? Apparently Samsung made one half a year ago that costs as much as the iMac itself.
- mayoff 11y ago
  Dell's 5k monitor has been as low as $1440 at amazon: http://camelcamelcamel.com/Dell-Monitor-UP2715K-27-Inch-LED-Lit/product/B00OKSFXZU http://camelcamelcamel.com/Dell-Monitor-UP2715K-27-Inch-LED-... Not that I would recommend it over the iMac for this project.
guelo 11y ago
Yea but the integrated quiet computer was also important for his project. Mounting a separate computer would have added a lot of complication.
- yincrash 11y ago
  He does not say this. In fact, at the end of the article, he writes that having the whole iMac is fairly impractical and having a smaller Android system on a chip would be the better solution.
vernie 11y ago

nathan_f77 11y ago

This is amazing! I had the same idea a few years ago but never got around to it. I was planning to just use some regular LCD screens and give them some nice frames. Probably wouldn't have looked that great, though. It's going to have to wait though, I'm living in an apartment now and can't really cut huge holes in our walls. Even if everything could fit inside the frame, it would be pretty ugly if you couldn't hide the power cable.

theon144 11y ago

Awesome! I really love the usage of the photodiode, as well as the camera, both are pretty thoughtful finishing touches that elevate this to more than "a cool digital frame". How much does the display cost, if I were to make a frame like that too?

morsch 11y ago
The cheapest 5k iMac is $1799, other comparable displays are roughly in the same ballpark. I imagine the other items (frame, sensors) are trivial compared to that expense. Replacement 5k panels for the iMac (LM270WQ1) are merely $800 on AliExpress, but you'd have to figure out how to output to them.

stevewilhelm 11y ago

"Visitors to Microsoft chairman Gates' new mansion in Medina will find themselves in an electronic Disneyland where they will be able to "call up" a favorite painting or photograph on combination TV-computer screens. "Right now, only a billionaire could afford to try $80,000 monitors and $150,000 computer-storage devices (the house could have several) that may be needed to make this all work." from "Bill Gates: 50,000 Computer Paintings On Call" (1991) http://community.seattletimes.nwsource.com/archive/?date=19910526&slug=1285524 http://community.seattletimes.nwsource.com/archive/?date=199...

otakucode 11y ago
I remember reading about the various technological features Gates put in when he built his home. I would be really interested to read/see (video would be awesome) a followup piece about whether many of these things either required regular upgrading or else have simply become antiquated. I would guess that his family kept upgrading the home, but details would be interesting.
morgante 11y ago
And now you only need to be a millionaire.

mtone 11y ago

I'm a big fan of flux, but brightness adjustment is another story. On my iPad, I quickly turned it off. The auto-brightness never hit the spot I wanted, and switched too strongly and too late. Reading on the bus, the brightness basically goes berserk at every shadow and never settles. At home it would be more manageable, but unfortunately desktop monitors don't support brightness adjustments as far as I know -- really wish it was standard through HDMI. Having to put an entire iMac computer on the wall to have access to brightness adjustments (at least that's my understanding of the solution) makes it a lot less practical. But kudos to the author for addressing that essential part of the problem.

mayoff 11y ago
Keep in mind that the cheapest 5K iMac is in the same price range as standalone 5K monitors. So you might as well use the iMac unless you can get a 5K monitor at a pretty steep discount.

otakucode 11y ago

I continue impatiently waiting for a large format true color electronic paper display at a quasi-reasonable price. I have had a different project idea for such a thing for several years now, but the wait continues. I imagine such a display would be excellent for the use described in this article. No need to mess about with matching light levels (although doing that was super cool!), battery would only need charged every few weeks or months, etc. There are actually a couple products that seem to be what I want, but for reasons I do not understand the image displayed can not be changed. I have no idea why someone would buy electronic paper signage whose image has to be provided to the manufacturer and which can not be changed later. I would expect a simple printed sign would be superior for any use that would be put to.

peter303 11y ago

Constrast is the lagging factor in good computer displays. These are called High Dynamic Range (HDR) displays. The claim is the eye can see a contrast factor of about a million at any one time, and a billion switching between sunny daytime and darkest night. An end-to-end camera-display system would need 20 bit amplitudes represent such a dynamic range. Many current systems are just 8 or 12 bits. SIGGRAPH often exhibits high end experimental displays. 4K resolution is close to what the eye can see. Four or six basis color systems improve color capture. But I generally just see an improvement in nature scenes. The huge win is HDR. A good HDR display is almost indistinguishable from looking through a window.

alok-g 11y ago
https://en.wikipedia.org/wiki/Human_eye#Dynamic_range https://en.wikipedia.org/wiki/Human_eye#Dynamic_range >> 4K resolution is close to what the eye can see. Depends on the distance to the display (whose resolution is independent on where the viewer is).

KingMob 11y ago

Thisi s cool, but to really reproduce art, you have to avoid RGB-based displays. I've always assumed the reason wide-gamut/CIE Lab*-influenced displays are uncommon is that RGB is "good enough" for most purposes outside of image-editing/printing. But they can only show a fraction of the colors humans can see. IIRC, the standard sRGB space used in most monitors can only show a third of the human-visible spectrum.

morsch 11y ago
Yes, that seems like a huge limitation. Though the 5k iMac is wide gamut, so quite a bit better than sRGB. Are there even (consumer?) displays that have a significantly wider gamut? http://blog.conradchavez.com/2015/10/26/a-look-at-the-p3-color-gamut-of-the-imac-display-retina-late-2015/ http://blog.conradchavez.com/2015/10/26/a-look-at-the-p3-col...
- KingMob 11y ago
  Kudos to Apple for making the new iMacs wide-gamut. But I don't know that it's too common in the consumer space yet, mostly because once you start using multiple color spaces, you run into the problem of images created in one color space being shown in another. If the image data isn't tagged correctly, things will look over/under-saturated on different monitors. (E.g., is rgb(255, 0, 0) the reddest red in sRGB or the reddest red in P3/Adobe RGB?) I suspect that when monitor manufacturers tire of competing on pixel density, brightness, 3D, and size, they'll eventually get around to competing on gamut. Maybe Apple will lead the way. A lot of photo/video cameras already capture a larger gamut, so monitors are the last piece of the puzzle.
  - mark-r 11y ago
    Unfortunately as the gamut gets larger, so does the visual difference between adjacent colors. You need more than 8 bits per channel from end to end if you're going to use anything wider than sRGB. Any RGB based color space will be triangular shaped; if it's to be physically realized in a display, all 3 vertices must lie within the CIE gamut. Since the CIE gamut isn't perfectly triangular you will always be missing some colors outside of the triangle no matter what you do. You could do much better by adding a 4th primary to create a trapezoid, which would more completely fill the space. As the entire industry is RGB based this would be completely impractical, but it would be fun to see as a demo.

callmeed 11y ago

If I pull an LCD panel out of an old laptop, how do I connect it to something small like a raspberry pi or beaglebone? My understanding was that many lcd panels required custom drivers/firmware that was often embedded in the laptop/display board.

Palomides 11y ago
search for the part number on ebay and you'll probably find a driver board with vga or hdmi input, or a board that you can buy and ask the seller to program it for a specific display. Usually less than $30, but you'll need power, a case, etc..

SeanDav 11y ago

My first thought when I started reading was "wow what a waste of time and money to display a stupid picture" but by the time I was near the end of the article, my attitude had turned 180 degrees. This is amazing attention to detail and craftsmanship in the traditional sense. I can only admire the process - really impressed.

spdustin 11y ago

It brings to mind the "moving paintings" in the ride wait queues for "Harry Potter and the Forbidden Journey" (a dark ride with a moving car on an articulated arm). I saw those things up close and they had a wonderful canvas texture laid over the screens. It really seemed magical -- though I admit I'm one to give myself over to the magic of theme parks when I can -- and it made me wonder if this effect could be reproduced by using a matte silicone mold of actual canvas or something like that. As an aside, there is a lot of amazing technology in play in that ride; I read an unauthorized treatment of how the ride works and I was blown away at the systems/mechanical orchestration necessary to pull it off. It's no wonder it breaks down all the time.

frikk 11y ago

It's too bad this isn't on github. I've always wanted to make something similar, but using computational art instead of paintings. My dream is to make a thing that slowly adapts to the room its in, but doesn't seem like a computer hanging on a wall. Does anyone know where I can learn more about the lighting techniques described here?

rkalra4 11y ago

Anti-"Shy mode" with all such frames connected over intranet would be really cool!

hoorayimhelping 11y ago

Fantastic read, and a great idea. Very inspiring, thank you for posting it.