9 ms·
3D scatterplot of an image
- rijoja 9y agoWould it be possible to make a human recognizable 3D-object in the colorspace that also is a pretty picture in 2D?
- Confiks 9y agoThe answer is quite obvious. You could start off with a grayscale 2D image on one side, and a pretty dotted 3D object on the other side. You'd then read the coordinates of the 3D object in a specific color space to obtain color values which you can pretty randomly (or even judiciously) apply to the grayscale image. Now I would pose another question: would it be possible to make a human-recognizable 3D-object in all the four color spaces that is also a pretty picture in 2D?
- enriquto 9y agothe transformations between color spaces are smooth maps, so each of the 3D objects would be the same thing (with some warping and stretching)
- 0xdeadbeefbabe 9y agoSounds like a good art project.
- rijoja 9y agoClever to start out with a greyscale image! But it still poses some constraints, let's say if we start with an all black image for an edge case to demonstrate the limitations, we can only have a 2D pattern in the 3D space, right? However the luminance component of the image could probably be adjusted within certain error margins and vice versa. Perhaps by some form of bruteforce! To know for sure I suppose coding is needed!
- 05greg 9y agoYeah, the points in the 3D colourspace have no mapping to a 2D location on an image, you could create one that gives a pretty image if you wanted.
- crispyambulance 9y agoWell, there's another neat thing that you can do if you want to generate an interesting 2D -> 2D representation: * Take fourier transform of the image for each color R, G, B * Randomly scramble the phase of the transformed image. * Perform the fourier inverse of that scrambled phase image. The result is another 2D image with the same colors AND spatial frequencies as the original image. It looks like you took the original photo and vaporized it into gas. Interesting.
- seanlinehan 9y agoDo you have any examples of this? Sounds cool
- yorwba 9y agoI did a Jupyter notebook http://nbviewer.jupyter.org/github/Yorwba/image_frequency_scrambling/blob/master/Image%20Frequency%20Scrambling.ipynb http://nbviewer.jupyter.org/github/Yorwba/image_frequency_sc... I should have probably written more comments, but I really need to go sleep now.
- crispyambulance 9y agoYep, that's it! Here's the same thing in R... library(imager) download.file("https://upload.wikimedia.org/wikipedia/en/2/24/Lenna.png",'lena.png', mode = 'wb') lena <- load.image("./lena.png") plot(lena) lena_fft <- FFT(lena) lena_pwr <- sqrt( lena_fft$real^2 + lena_fft$imag^2) lena_ph <- atan2(lena_fft$imag, lena_fft$real) lena_ph[1:512,1:512,1,] <- matrix(runif(512*512,-pi,pi),ncol=512) lena_mess <- FFT(lena_pwr*cos(lena_ph),lena_pwr*sin(lena_ph),inverse = TRUE) plot(sqrt(lena_mess$real^2 + lena_mess$imag^2))
- rijoja 9y agoYes yes fourier transforms is what make mathematicians go mad and or religious.
- deleted 9y ago[deleted]
- GistNoesis 9y agoYes, another way : take a colored picture, sample as many points as there are pixels on the surface of the 3D-object, then for each pixel of your picture pick without replacement its closest remaining color in your 3d object 'color palette'. The looping order still grants some artistic liberties.
- rijoja 9y agoIt just struck me that the resolution of the image can be modified, if more or less voxels are needed! Let's say if we have a 4096 times 4096 image we can reach and fill all values in the 3d representation, if we want to!
- vuknje 9y agoYou mean something like this? https://youtu.be/YaA3dAJz6sI?t=45s https://youtu.be/YaA3dAJz6sI?t=45s
- pavel_lishin 9y agoThis is cool, but I wonder why most of the demo images - and a lot of the images I uploaded - seem to have their colors aligned in a single plane.
- akrasuski1 9y agoHmm, I guess someone would have thought of that, but this could probably be used for compressing images by reorienting the color axes so that one of them is perpendicular to this color plane, and then sending less information about that coordinate.
- iainmerrick 9y agohttps://en.wikipedia.org/wiki/YUV https://en.wikipedia.org/wiki/YUV
- abainbridge 9y agoYeah, you can use principle component analysis to figure out the transform to make the colour space "optimal" for compression of a specific image. I tried that once: https://bainbridgecode.wordpress.com/2012/04/24/beating-png-part-2/ https://bainbridgecode.wordpress.com/2012/04/24/beating-png-... and https://bainbridgecode.wordpress.com/2012/05/07/beating-png-part-3-remembering-a-level-maths/ https://bainbridgecode.wordpress.com/2012/05/07/beating-png-... There are plots at the end of part 3 showing the image channels separately for YCrCb and my custom PCA derived space. They clearly show that there's a lot less duplication of data between channels with the latter. And that in turn is obviously good for compression.
- thomasahle 9y agoFinding the first primary component also appears to be a pretty good way greyscale an image.
- Overv 9y agoI'm not sure. It's hard to say because RGB is a bad representation of the color semantics anyway. Try visualizing the images in HSV/HSL/LAB space instead.
- ekianjo 9y agoSo what's the insight behind such visualizations? Can it be used to separate photographic pictures from paintings for example?
- Overv 9y agoOne thing it can be used for is to easily see what kind of palette a painter is using.
- gt_ 9y agoTons of artistic applications. I can’t think of anything this visualization gives that standard scopes in software like Nuke don’t provide, but those are not as intuitive or quick to read. Color in image reproduction is notoriously difficult to measure by simply looking at a whole image. Much of what we perceive as color characteristics are contextually dependent. And while the characteristics may not vary, the level of contextual dependence seems to. Females are more likely to have more reliable contextually affected color perception than males.
- vbarrielle 9y agoThis kind of visualization is useful to understand why filming an actor in front of a green screen makes sense for special effects. If you look at such an image, you will have two clusters of pixels, one for the green screen, and one for the actor. By selecting the appropriate cluster you now know which pixels you need to replace for compositing your actor in front of a CGI background.
- JD557 9y agoNice visualization, although an option to show HSV/HSL in a cylinder might help to better understand the distance between colors.
- neonnoodle 9y agoThis is really cool! I'm a painter and have spent a long time doing gamut analysis of paintings. Thank you for including both HSV and HSL. Not enough people appear to understand/care about the difference between the two, and for painters the distinction is everything.
- pygy_ 9y agoWhy is the distinction important for painters?
- discreditable 9y agoI'd guess it's because mixing colors in paint is subtractive—if you mix every color you get black. Whereas with light, mixing colors is additive—if you mix every color you get white. In HSL, fully saturated colors have .5 brightness, and adding white or black changes the brightness but not the saturation. My guess is that works more similarly to paint than HSV does.
- kqr 9y agoLab too! I'm a digital photographer and operations in Lab space is both integral to my work and once you get used to it, it's very intuitive.
- jacobolus 9y agoBoth are absolutely terrible for painting (or any other human purpose, especially things like this scatterplot), and should be replaced with some perceptually relevant model. (Disclaimer: I largely wrote the wikipedia article about HSL and HSV, years ago.)
- jathu 9y agoI'm not sure if they are mapping every single pixel or using some average pixels, but it's pretty fast. Here is a shameless plug for one of my color extracting library: https://github.com/jathu/UIImageColors/ https://github.com/jathu/UIImageColors/ It currently takes around ~0.3s on average to extract the colors. However, with my new PR (https://github.com/jathu/UIImageColors/pull/54 https://github.com/jathu/UIImageColors/pull/54), it takes around ~0.14s on average. IMO this is still slow, I would like to bring it below 0.1s. I tried to optimize this with k-means to reduce total number of colors, but the result was slower and worse color choices. If anyone has methods to improve the performance, please make a PR.
- mynewtb 9y agoWhat takes so long in your approach? I would think extracting simple statistical values from an image is a very easy to vectorise and parallelise task.
- deleted 9y ago[deleted]
- redcalx 9y agoWith 24 bit color you can create an array with an element for each of the 16 million colors and just build the histogram. Running k-means on that is going to be less efficient than just making the histogram buckets larger, e.g. 2x2x2 = 8 colors per bucket, or whatever. So yeh you should be able to do that in a few milliseconds I would have thought. For more speed look at using SIMD instructions.
- pducks32 9y agoI too have been playing with color quantization as an exercise so I won't like at your library as I've been trying to do it all on my own and don't want to see other approaches yet, but here he is not quantizing them so his is going to be faster. Also there really is a trade off in trying to reduce the number of pixels and then clustering versus just clustering on them all. How many times you loop and how much those loops costs isn't as cut and dry as I thought.
- zamazingo 9y agoThis is really cool! Is there a source repository we could explore?
- deleted 9y ago[deleted]
- Overv 9y agoI've now published the source of the website on GitHub as well: https://github.com/Overv/ColorScatterPlot https://github.com/Overv/ColorScatterPlot
- thechao 9y agoIn the late '90s I was asked to do image segmentation of multispectral data of brightfield microscope data. That is, given a picture of a bunch of cells, find all of the cells. There were two problems: image segmentation of sick cells is really hard (they're very blobby & fragmented), and the computer I had available took more than 20 minutes per image. It was literally both cheaper and faster to have a grad student count the cells. Let's be clear though: the researchers weren't interested in the cells; they were interested in having a pretty-darn-good count of cancerous cells vs. non-cancerous cells. Turns out, cancerous cells were a different color (brighter blue * brighter green) than healthy cells. So, instead, I just converted every image into RGB 3D space, counted all the pixels near 'dark green * dark blue', all the pixels near 'bright green * bright blue' then divided by the average area (in pixels) of each type of cell. The resulting counts were within 1% of the value the grad students could get, and the ratio was even more accurate. We could get the cell count in a single linear, forward pass, as fast as the camera could take pictures.
- nitrogen 9y agoSounds a lot like the simplifications I made to be able to process 3D Kinect data at near full speed on an ARM CPU with no FPU. Instead of trying to segment objects, remove backgrounds, do skeleton tracking, etc., just count pixels that fall within a 3D bounding volume. Simple solutions are sometimes underappreciated. It helps, as you allude to after "let's be clear," to know what problem you really are trying to solve.
- the_cat_kittles 9y agoi love these kinds of problems, because it seems like there is often a really simple solution like that. just finding something different in a very simple feature like color, or shape, or size, or location. there is a reason our brains pay attention to these kinds of things!
- tovacinni 9y agoSorry if this is a stupid question, but can't you do this without converting the image into an RGB 3D space? (i.e. iterate through the pixels and count the ones within a certain range of what you want)
- crispyambulance 9y agoThis reminds me of a machine vision application for large-scale bakeries. When you bake bread, the color of the surface changes as the bread rises and completes the baking process. In the application I remember, the average color of a loaf was tracked on an RBG plot as a function of time. The idea was that the baking process followed a trajectory in the RGB space that was dependent on the temperature of the oven and the elapsed time. This can be used as feedback for process control-- to signal when to pull out the bread or to change the temperature. In a similar way the color of paintings change as they age. The individual points in the OP's plots could be parameterized in time and according to paint's chemistry. It might be possible, after some calibration studies, to accurately simulate aging. Or better, simulate the reversal of aging to show what an old aged painting looked like when it was new.
- zwieback 9y agoI did the same thing for browning of french fries. It turned out that RGB was too highly correlated to be efficient so I mapped into an "application specific" color space where the rate was pretty much along a straight line. Kind of like HSV but more specific to the actual cucumber shape of the RGB distribution. https://www.google.com/patents/US5818953 https://www.google.com/patents/US5818953
- fenollp 9y agoRelevant: https://www.reddit.com/r/dataisbeautiful/comments/7584no/3d_rgb_scatterplots_of_colours_used_in_famous/ https://www.reddit.com/r/dataisbeautiful/comments/7584no/3d_...
- Overv 9y agoThat was actually the inspiration for this project :)
- dahart 9y agoSounds cool, I will check it out on a laptop. Just in case the author is here, I tried on an iPad Pro, and got "WebGL is not supported by your browser". WebGL is in fact supported by mobile Safari, so I'm guessing this site is using a user agent whitelist or some other fallible method of detecting WebGL. The best way to detect WebGL support is to attempt to create the 3d context and wait until after it actually fails. That, or simply do not check for WebGL failure; the browsers all handle it anyway.
- sus_007 9y agoHow much JS knowledge does one need to start plotting with D3.js or Plotly.js ?
- MegaLeon 9y agoFunny to see this, I made more or less the same thing in Processing for the google devart interactive competition a few years ago: https://www.youtube.com/watch?v=YaA3dAJz6sI&feature=youtu.be https://www.youtube.com/watch?v=YaA3dAJz6sI&feature=youtu.be Has a few extra features like being able to see different visualisation methods and how basic colour correction influences those - also animates the visualisation while going through the image Realluy caught me off-guard because I used both the starry sky and the monalisa painting back in the days as well!
- btbuildem 9y agoVery neat! I like how you aggregate the points with larger objects -- sort of a 3D contour plot. Do you have an online demo handy?
- adrianmonk 9y ago"based on the pixel values" I daresay technically this applies to all useful visualizations of the data. I don't think they'd be very useful if they weren't, in some way, based on the pixel values.
- _han 9y agoSure, but here the pixel values are directly used as spatial coordinates, so it makes sense to mention this.
- adrianmonk 9y agoI was trying to make a point about phrasing. "Based on" is wordy and nebulous. Just saying "of" would clearer, not to mention shorter.
- xbryanx 9y agoThis could be a fun add on for an art museum's online collection browser. "Investigate the collection by scatterplot."
- dontreact 9y agoThis is a good illustration of why this algorithm which reorganizes paintings into 2d color palettes works Using these scatter plots ou can see how you would use the principal components to lay out the pixels on a 2d surface https://github.com/ardila/paintingReorganize/blob/master/README.md https://github.com/ardila/paintingReorganize/blob/master/REA...
- psyklic 9y agoThere is a really cool use of clustering with these scatterplots to do color quantization (i.e. reducing the number of colors). If you want 16 colors, just find 16 clusters and their centroids (e.g. let k=16 for k-means clustering). Replace each pixel with the closest centroid. Then, your image is quantized (and can perhaps be stored more efficiently)!
- ska 9y agoThere is nothing specific about color here - the same approach can be used for intensity re-quantization for example. Effectively you are non-uniformly resampling based on distribution, so relative values/distances go out the window but you keep "detail" in the lower fidelity signal.
- sidhantgandhi 9y agoThis is beautiful. It's interesting to see how much of the space is not occupied — even by images that feel like they have a lot of colors. Next up: Finding patterns in the graph for certain artists, themes, styles, etc.
- bennettfeely 9y agoSome interesting images to upload "One Million Colors" https://upload.wikimedia.org/wikipedia/commons/thumb/d/d6/1Mcolors.png/800px-1Mcolors.png https://upload.wikimedia.org/wikipedia/commons/thumb/d/d6/1M... "RGB HSV Image" https://www.gimp.org/tutorials/Digital_Black_and_White_Conversion/rgb-hsv.png https://www.gimp.org/tutorials/Digital_Black_and_White_Conve...