5 ms·
Show HN: Invertornot.com – API to enhance your images in dark-mode
Hi HN, I built (https://invertornot.com https://invertornot.com) it's an API that can predict whether an image will look good/bad while inverted. This is particularly useful for images in dark-mode as you can now safely invert them.
The conservative solution to adapt images for dark-mode consist in dimming the image, however there is a lot of images that can be inverted (graph for example). Using deep learning we can avoid heuristics and obtain a much more reliable solution.
The API uses an EfficientNet pre-trained model fine-tuned on a custom dataset (1.1k samples). EfficientNet was chosen as it was pre-trained and offered the best performance for its size.
The trained model is very small (16MB) which means you can easily run your own instance. This problem is very simple for deep learning as it's a simple binary classification.
For this project training the model wasn't the challenge as most of the time was spent on the construction of the dataset.
For the API I'm using FastAPI, Redis and ONNX Runtime to run the model. The API can be used by posting the images to the API, using URL and using SHA-1 for already processed images.
The API is free and open-sourced (http://github.com/mattismegevand/invertornot http://github.com/mattismegevand/invertornot).
- alanbernstein 3y agoNeat, as a reader, I've absolutely wanted this exact behavior many times. Do you have any examples of incorrectly classified images? As you say, the model seems quite simple, but I wonder if there are some pathological kinds of photographs that result in inversion.
- mattismegevand 3y agoSome graphs with lots of colors can be difficult for models to handle. In these cases, the model often chooses not to flip the graph to avoid making it hard to read, based on past problems with similar graphs. The model generally does a good job at deciding when to flip something. Although I haven't seen it flip photos, it's possible that there are examples out there.
- gwern 3y agoSome recently-added datapoints I think the current model mis-classifies: https://upload.wikimedia.org/wikipedia/commons/thumb/6/69/Proctor_Richard_Anthony_Other_Worlds_Mars_Chart_1896.jpg/800px-Proctor_Richard_Anthony_Other_Worlds_Mars_Chart_1896.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/6/69/Pr... https://upload.wikimedia.org/wikipedia/commons/thumb/f/f7/US_Presidential_elections_popular_vote_graph_since_1900.png/800px-US_Presidential_elections_popular_vote_graph_since_1900.png https://upload.wikimedia.org/wikipedia/commons/thumb/f/f7/US... https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Nature_Study_Club_of_Indiana_yearbook%2C_1927_-_DPLA_-_d833818284c065dc8e607873a24507d6_%28page_3%29.jpg/800px-Nature_Study_Club_of_Indiana_yearbook%2C_1927_-_DPLA_-_d833818284c065dc8e607873a24507d6_%28page_3%29.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Na... https://upload.wikimedia.org/wikipedia/commons/thumb/b/bb/Life_expectancy_in_some_European_countries.png/800px-Life_expectancy_in_some_European_countries.png https://upload.wikimedia.org/wikipedia/commons/thumb/b/bb/Li... https://upload.wikimedia.org/wikipedia/commons/thumb/a/a7/Pie_chart_of_stomach_cancer_types_by_relative_incidence.png/800px-Pie_chart_of_stomach_cancer_types_by_relative_incidence.png https://upload.wikimedia.org/wikipedia/commons/thumb/a/a7/Pi...
- hansihe 3y agoThis is really cool! Do you have some more example images somewhere?
- mattismegevand 3y agoThank you, yes you can try it directly (https://invertornot.com/docs https://invertornot.com/docs) or you can either upload an images or spam the button to get random images and see their prediction.
- SCUSKU 3y agoVery cool! Nice job! Out of curiosity, how much CPU/GPU/RAM resources do you need to allocate to run something like this? Like you mention you're using small models. For example, could this run on a t2.micro on EC2? (1 vCPU + 1GB RAM)
- mattismegevand 3y agoThank you! Very little resources are needed to run it, gwern is using the API and at first I was using a 6$ Droplet (exact same specs as the t2.micro) and it was running well.
- madeofpalk 3y agoSemi-related - I saw this on https://v8.dev https://v8.dev a while back, but `filter: hue-rotate(180deg) invert();` can be a neat CSS trick to 'dark mode' some kinds of graphics, while not screwing with the colours so much. The `hue-rotate` help a bit to keep 'blues blue' etc. It's far from perfect, but it's a neat one to have in your back pocket.
- therein 3y agoAnyone who thought Retool should have a dark mode has come across this. Definitely a useful trick.
- kinj28 3y agoHey This is cofounder of a retool alternative here. We did introduce dark mode in a manner where a creator can even make it dynamic for end user to choose a different theme. While I was loggerheads with my CTO cofounder on this - I am glad someone finds this useful in the community of users. I am curious what kind of use case you have for retool/ alternate tools where you see dark mode as a need
- klntsky 3y agoWhat's your accuracy?
- mattismegevand 3y agoClose to 100%, this problem seem to be easily solved by the model. I plan to do cross validation and also to explore the use of a smaller model.
- klntsky 3y agoDid you separate the test dataset from the training one?
- gwern 3y agoDoes it matter? It's not a research project trying to rigorously evaluate novel architectural modifications or something, but just a project trying to be useful within the limited resources of a hobbyist. If someone labeled a bunch of the remaining errors, that data would then be better used as more training data than to benchmark. In practice, the accuracy, whatever it is, appears to be very high and more than adequate to justify its use.
- klntsky 3y agoYes it does. You will always get close to 100% accuracy on small datasets if you evaluate the model using the train dataset - due to overfit.
- a2code 3y agoHow can you use or create this model for conventional non-API projects? Like to embed it within some C code?
- lights0123 3y agoThey're using ONNX Runtime to run the model, which has a C API: https://onnxruntime.ai/docs/get-started/with-c.html https://onnxruntime.ai/docs/get-started/with-c.html. You could also use any other ML inference library that supports the ONNX format. You'd need to look at this project's code to see how it converts input data to the expected matrix format, though.
- mattismegevand 3y agoExactly, the transformation done for inference are located here: https://github.com/mattismegevand/invertornot/blob/main/api/nn.py https://github.com/mattismegevand/invertornot/blob/main/api/...
- block_dagger 3y agoDark Reader needs this! Nice work.
- noisy_boy 3y agoWhat I am interested in is a plugin/extension for Firefox/Chrome that can invert a video running on a page (darkreader is not doing that).
- gwern 3y agoDoes that really make sense? Unlike a single static image, a video is not one thing. It might go rapidly between invertible and non-invertible. Would you expect it to switch strategies potentially every scene or second...?
- noisy_boy 3y agoYes it indeed has very specific use case of tutorial videos with white background; those generally stay the same until the switch to dark mode for showing code. It would be impressive if the add-on is smart enough to do what you said to maintain dark background throughout (I don't have a lot of idea of how technically challenging this might be).
- Dibby053 3y agoAs a one-off solution you could switch to light mode and invert the display.
- hunter2_ 3y agoDoes this distinguish white backgrounds from transparent backgrounds? For example, a graph consisting of anything-on-white should probably be inverted, while a graph consisting of pure red on transparent should probably not be inverted. For the latter, increasing the luminance (i.e. make it pink on transparent) might be best, followed by leaving it as-is, followed by inverting. But I guess with ML reinforced by human feedback, trends like this manifest without having to explicitly handle them as special cases at all...
- noduerme 3y agoHuh. This is neat. As a side, I have a great frustration balancing dark mode with other things. I use flux on my Mac, not for color temp but strictly for the "darkroom" mode which inverts everything and makes the (inverted) brightness values into red. I only do this when coding at night, to help with my sleep... but the trouble is it's made for people who use light mode during the day. I use dark mode all day. So I have to get out of dark mode and change my vscode theme to light in order for it to work. Not a huge deal, but then I open up some web page with a black background and go blind from all the red.
- pimlottc 3y agoI would be interested to see more samples. Naively it seems like you could just say "isPhoto ? REDUCE_BRIGHTNESS : INVERT"
- mattismegevand 3y agoThe two big cluster are indeed photo and charts. However chart or meme are not all invertible, charts with lots of color can for example be degraded by inversion. Also some chart are already done for dark theme, for those images the model will understand that it doesn’t need to invert them.
- Labo333 3y agoPretty cool! A nice addition would be an in-browser runtime!
- rvnx 3y agoIt used to be implemented in Chrome, it was part of the feature called "Force Dark Mode Web Contents" and it had a runtime tflite model that determines if the lightness should be inverted (and a simpler heuristics that just counts the number of colored pixels).
- singularity2001 3y agonot exactly the same but for colored pictures GPT recently suggested inverting the image and then turning the color space by 180° so that the original colors are restored... pretty genius
- rockwotj 3y agoI built dark mode for an email client’s (shortwave.com) HTML emails and this was the last major hurdle to solve. I wish that had existed then :) Generally making emails dark mode is a non trivial problem, there is an hope source library that is a good start from Tempo: https://github.com/yourtempo/tempo-message-dark-mode https://github.com/yourtempo/tempo-message-dark-mode However there are plenty of edge cases that plugin doesn’t handle that we had to fix at Shortwave. Examples include legacy HTML4 attributes, better support for emails that are half dark mode, emails that have background images of #fff (looking at you google play reciepts), the list goes on. It’s honestly pretty solid now
- bagels 3y agoIs inverting the colors in a photo ever the right choice? People come out looking really strange.
- juancampa 3y agoThat’s the point of using a NN. If it detects people, dim it (losing contrast). If it detect bright vector graphics, invert it (not losing contrast).
- medstrom 3y agoIf you don't have a lot of images, you can go over them all manually and write the "please invert me" request directly into the filename. CSS: img[src^="/images/invertable"] { filter: hue-rotate(180deg) invert(1); } Now all image names starting with "invertable" will be inverted.
- gwern 3y agoYep, or you can just set a class on each image. It's easy even in Markdown: `{.invertible}`. It's just that often you do have a lot of images and, more importantly, they may not be known in advance. If images are dynamic, then you obviously can't go over them manually, no matter how few of them there are. This was the principal motivation for me for InvertOrNot.com: I am willing to manually classify hundreds or thousands of images for inversion, and I did, but what I can't do is do that for all of the many Wikipedia popups on gwern.net - not just because there's 22k+ of them, but because they are constantly being added, WP articles are constantly changing, and they are fully recursive so a reader could pop up any WP article. A fully-automated API is the only possible solution. (Doesn't have to be neural nets, that's just the most obvious & easiest approach; I expect classical approaches would work too and would be easier to get running in-browser too.) And then if you have a high-quality fully-automated API, then you might as well drop the manual classifying. It's tedious and adds friction to writing.
- rob74 3y agoI'm confused... has the meaning of "invert" changed since I last looked? Because the bar chart given as an example on the page isn't actually inverted (the color of the bars and as far as I can see also the text stay the same, only the background changes from light to dark). So, additionally to this API, you need another model that does the actual "inversion"? Or just good old editing? EDIT: from another comment - actually "inverting" seems to refer to doing hue-rotate(180deg) invert() ...which (if I read it correctly) inverts the hue of colored areas before applying the actual inversion in order to keep the colors the same (more or less - see also https://stackoverflow.com/questions/65344006/why-does-filter-invert1-hue-rotate180deg-turn-red-into-a-peachy-pink-colo https://stackoverflow.com/questions/65344006/why-does-filter...) ? Not a frontend developer, so I wasn't familiar with this trick until now. TIL...