8 ms·
Over-engineering an emoji webcam filter with a neural network
- helsinki 4y agoI suspect calculating the average RGB of each emoji and comparing the distances between the average RGB of subsections of the image would have resulted in better emoji choices and better output (simpler, too). I mean, I get that maybe it’s not the point of this article, but I think the end result would have been a better ‘mosaic’.
- thfuran 4y agoRgb isn't a great color space to use with distance measures. CIELAB is better, though really perceptually uniform color difference measures are still nontrivial in that space as well.
- williamcotton 4y agoTake a look at the ‘12’ on the bottle of scotch. Higher resolutions than the dimensions of the emoji pixels would otherwise warrant are achieved by selecting emoji that work akin to subpixel antialiasing.
- Llamamoe 4y agoI'd reckon there's probably room for about 2x2 subpixels' worth of "antialiasing", making the search space 4x3=12 dimensional (probably in Lab colorspace). Is that doable? If yes I'd expect the approach to be close to optimal possible.
- auraham 4y agoI think there is an algorithm for determining the dominant color in an image (in this case, in a emoji). I do not its name, though. https://stackoverflow.com/questions/3241929/python-find-dominant-most-common-color-in-an-image https://stackoverflow.com/questions/3241929/python-find-domi... https://dev.to/producthackers/creating-a-color-palette-with-javascript-44ip https://dev.to/producthackers/creating-a-color-palette-with-...
- liuliu 4y agoI think author can try to encode location of the emoji and incorporate better scoring function (GAN)? But that is obviously really "over-engineered" :)
- ipsum2 4y agoForgive me if I'm misunderstanding how it works, is the model is predicting the original emoji from an augmented emoji, and doing that for every 36x36 patch in the webcam image?
- brrrrrm 4y agoauthor here! I grabbed them from here IIRC https://emojipedia.org/apple/ https://emojipedia.org/apple/ (there are a bunch of sites that have these) edit: yep - predicting the original based on the augmented version (as a hack to generalize for unseen inputs)
- ipsum2 4y agoI've updated my comment to clarify the question
- taylorius 4y agoNearest neighbour algorithm though, no?
- shreyshnaccount 4y agoYep, isn't that used to find dominant colours in images fast anyway?
- bluelightning2k 4y agoYou mention a lookup table would have been tedious, but surely an easier method. This might be a case of "to a hammer everything looks like a nail". The solution in my familiarity would be to align the emoji in a grid, take one screenshot and write a script to iterate over the pixel data to figure out the most dominant color and how dominant. So the build of the lookup set would be automatic. Less interesting for sure but easier (at least for a JS dayjob skillset). Similar to a below comment, I know this isn't the point. Thanks for an interesting write up
- auraham 4y agoI think a (N, D) array can be used as a lookup table. Here, N is the number of emojies and D is the dimensionality of the color space (say 3 for RGB). Each emoji is converted to a D-dimensional vector by computing the dominant color. Then, we can take a D-dimensional vector from an input image and compute the closest emoji by using cosine similarity. To minimize the loss function, we can increase the number of dominant colors, so the table would be (N, DxM). In this case, each emoji is represented by M D-dimensional vectors, each one encoding a dominant color.
- Llamamoe 4y agoDo you have any good(preferably light on dry math) resources on these kinds of multidimensional search problems?
- humanizersequel 4y agoHey, Bun in the wild!
- MuffinFlavored 4y agodoes Bun just wrap node.js?
- humanizersequel 4y agoI believe Bun is meant to replace Node. Could be wrong though, I am not a Javascript understander by any stretch of the imagination.
- lebaux 4y agoai already lost my guys