6 ms·
Make photomosaics, GIFs, and murals from pictures in Python with ML/OpenCV
- aaaaaaaaaab 8y agoOk, but where is the ML part? We’ve been creating these mosaics for decades...
- simple10 8y agoIt's using the KMeans[1] library from sklearn.cluster. But this isn't really ML, is it? My ML knowledge is limited. Regardless, it's a cool project. OP might want to update the title to remove ML. [1] https://github.com/worldveil/photomosaic/blob/master/emosaic/image.py https://github.com/worldveil/photomosaic/blob/master/emosaic...
- simple10 8y ago"[Kmeans] algorithm has a loose relationship to the k-nearest neighbor classifier, a popular machine learning technique for classification that is often confused with k-means due to the name." Maybe an ML expert could elaborate? I've been curious on my own projects when to actually mention when they use true ML or not. https://en.wikipedia.org/wiki/K-means_clustering https://en.wikipedia.org/wiki/K-means_clustering
- muzakthings 8y agoRight. See my comment above. As to why it was in the project: If you treat each pixel as an example vector in 3 dimensions and cluster, you get the “dominant” colors for the image. It’s a primitive way to compress images as well. In this case I just was using it to generat fun cards that would use a minimal number of dominant colors. It’s still in the code if you’d like to use it but a bit hidden.
- _fullpint 8y agoK-nearest neighbor clusters based upon the k most similar objects. K-means, clusters on centroids that are means. After every interation new means are calculated and then reclustering occurs.
- muzakthings 8y agoNaw I had used the Kmeans part for extracting dominant colors. I was thinking about using it to generate cards and using the top K colors for the background of the card. As I mentioned above the only ML is the face classifier. There’s a flag that allows you to only include face pics in the photomosaics as well
- muzakthings 8y agoThe face montage building trains a linear classifier on top of the pretrained embedding network - it’s the portion that talks about creating a training folder of your face. But yes the photomosiacs strictly don’t use ML, unless you count the internal fun stuff Faiss (the similarity search lib) does to construct fast indexes.
- dfbrown 8y agoI'm not an expert in the topic, but my understanding is RGB is a poor color space for computing color difference. This could be why your mosaics end up so washed out. [1] suggests using a CIELAB color space [2]. Edit: Looking at the code more closely it looks like you were using Lab at one point but commented it out[3], so I'm guessing you're already aware of this. 1: https://stackoverflow.com/a/9019461/185171 https://stackoverflow.com/a/9019461/185171 2: https://en.wikipedia.org/wiki/CIELAB_color_space#CIELAB https://en.wikipedia.org/wiki/CIELAB_color_space#CIELAB 3: https://github.com/worldveil/photomosaic/blob/bb720efda11383fbb037a05bec41eea942ddb189/emosaic/image.py#L63 https://github.com/worldveil/photomosaic/blob/bb720efda11383...
- muzakthings 8y agoIt didn’t make a ton of difference empirically when I tried it. But you’re correct, generally that’s the space you want to be in.
- deleted 8y ago[deleted]
- gedy 8y agoIt is quite noticeable when you are using a limited selection of tiles or image has desaturated colors. I've done some similar work[1], but issue with L*ab color is it's terrifically slow to calculate diff, at least in JS. [1] https://imgur.com/a/g3EzcSV https://imgur.com/a/g3EzcSV
- muzakthings 8y agoTotally. Everything improves with more images since that’s your palette for painting, so to speak. The ones I posted were with less than 100 images so you can definitely do better. L*b was very slow, yes. It’s all done offline but I tend to like quicker feedback...
- itronitron 8y agothere is no perfect way to measure color distances, mostly because it needs to account for human perception of color and there are individual differences in color perception among people.
- androidgirl 8y agoThe gif with facial recognition is actually really really cool. Awesome work
- giladoved 8y agoThis is fantastic, great project!
- rmonroe 8y agoWay better implementation of the face alignment than what I did for our peru trip. Good going ;-)
- fireattack 8y agoIs the example image (https://github.com/worldveil/photomosaic/blob/master/media/readme/header.jpg https://github.com/worldveil/photomosaic/blob/master/media/r...) with or without opacity cheat?
- muzakthings 8y agoBoth! You can experiment with —-best-k and —randomness <1.0 and sort of get things in the middle. Basically what this will do is assign each tile less than 100% of the time randomly and then for each that isn’t, you choose among the top best K matches on L2 distance with equal probability. Gives it a little bit of both.
- fireattack 8y agoI'm talking about `--opacity` - because to me the mosaic images used in the sky can't be that blue originally.
- muzakthings 8y agoRight. The opacity setting is superimposed after the tile assignment. Thus you can have both. I’ve found that opacity if 0.7 is often a nice compromise.
- PavlovsCat 8y agoYeah that's using opacity, e.g. there's an image of a blue horizon that gets tinted red.
- itronitron 8y agolike others here, i really like the aligned face montage gif and it seems like it would be a great product for people to package up their selfies over a timeline