4 ms·

Show HN: Explore Websites by Chosen Color

96 points by ro_dobre 2y ago

I crawled lots of websites and extracted their color palettes (primary, secondary, etc) to build a searchable collection of website colors! It was a fun side-project and I learned a lot about color theory. Let me know what you think and what features you would like to see :) If you have any questions about the tech behind it, feel free to ask in the comments. Feedback is also greatly appreciated.

D_R_Farrell 2y ago
I wonder if you open up this to also deconstruct color variables (-var) into their respective hex codes if you'd get more results of sites that use design systems. Have you explored that much at all? Cool project!
- ro_dobre 2y ago
  That's a really interesting idea that I didn't get to experiment on, unfortunately. It's currently screenshotting each crawled website and extracting the colors from the pixel values. One filter I added to make the palette a bit more relevant is that I try to find out which is the background color (the most used color) and only pick colors that are at least somewhat contrasting with the background (assuming that websites would follow somewhat basic color contrast rules). When I tried extracting the palettes from CSS rules, what I ran into is that each website uses lots of vendors that render stuff (cookie consents, promotional pop-ups, etc.) and it's hard to distinguish what's rendered and what's not with pure CSS selectors, so I resorted to decomposing the website's theme from the perspective of a user. (through a screenshot) It's a great idea nonetheless, I'm sure I would have some good results with it if I manage to filter out the noise, thanks a lot!
pvaldes 2y ago
Very useful to spot webs with a particular theme, blue webs, sunny webs... but also all webs from a particular logo or party that often use the same palettes again and again as brand image. Allowing to chose proportions of colors could be even better. Hypothetical example: Lets suppose that I'm a policeman. How much percentage of black, white and red is in a classical svastika?. Then browse the dark web for finding all webs that contain this particular white, black and red in such combination Other possibilities: I want to know if my [bank/company/chess club] web is being cloned and used to criminal purposes without my knowledge. My logo contains magenta, lime green and burnt orange. Find me webs with this combination that aren't in a list of official webs from my company. Even better, check it at real time on email to spot scammers impersonating me on real time. The applications are endless: I want to detect webs that sell fake clones of my products; or find photos of pills of a particular shade of blue. Or don't show me webs with a high content in warm tones associated with porno images... Yep, that could be a really revolutionary startup.
- ro_dobre 2y ago
  Haha, lots of great ideas! I'm definitely considering adding an option to search by color proportions, just trying to find a UI/UX that is not confusing for people. I also have a lot more content (i.e.: the full HTML code of the website, the fonts and typography options) that I extracted for each website but it's tricky to come up with an easy to use UI/UX for this as a search filter, given that there's not many other websites that do this.
qingcharles 2y ago
I was looking for neon green sites for inspiration... Google says this site is now badware: millionfollowersbootcamp[.]com
- ro_dobre 2y ago
  Yeah, I need to curate the existing list of websites and perhaps swap the existing list for one with more popular websites. I'll definitely have a look, thanks!
muratsu 2y ago
This is really cool. We've recently built a website catalog (https://www.rapidpages.com/inspire https://www.rapidpages.com/inspire) and one of the difficult choices we needed to make was whether to include images on the site or not when extracting the color palettes. I'm curious if you've also experimented with this?
- ro_dobre 2y ago
  The site is looking great! I think that screenshots are a must, it's a lot easier for the end-user to preview the website's content. If you're referring to screenshots where certain colors are used, I don't think that's necessary. The first version of it, I didn't take any screenshots and it felt very empty. It's a bit hard to manage the images and deliver them rapidly, in an optimized form, but I think it's worth it.
  - muratsu 2y ago
    Oh, I was referring to calculating colors used on a website. To give you an example, we've stumbled upon pages where there's a product on the page that contained the color red, and nowhere else on the page was anything red. Our naive approach to finding avg colors was to simply take a screenshot of the page and process that image. Unfortunately, this method produces inaccurate results like red being part of the average colors when the design system has no red. I was curious if you managed to work around this issue. Happy to chat more if you're interested, you can reach me at murat[at]rapidpages.io
    - ro_dobre 2y ago
      Ah, sorry, I understand what you mean now. Yeah, I've also filtered out images when calculating the color percentages and there are definitely some funky results. There's definitely a middle-ground approach here, I just haven't found a good enough heuristic. It's true that images are part of the branding and they impact the color palette, like banners and logos, so they should be considered in some quantity. I thought of maybe averaging the results of image/no image but I don't have enough test data to validate it yet. Let's chat and discuss more! I'll also add you on Twitter/X
getwiththeprog 2y ago
Finally, I can see who is using this years Pantone style guide!
toataziuatestam 2y ago
Reeeally loving this one, great job !!
unittester 2y ago
Great Stuff, I could imagine this being handy when looking for design inspiration.
- ro_dobre 2y ago
  Haha, thanks! That's exactly what I had in mind when building. I've also collected what fonts and typography styles different websites use so I'm looking to create a visual for that as well, thanks for the feedback!
amadeuspagel 2y ago
Amazing. Maybe you can show the chosen color as the background of the hex code?
- ro_dobre 2y ago
  That's a great suggestion, will definitely do that in the next iteration. Thanks a lot!
GrumpyNl 2y ago
I like it and im gonna use it, great for inspiration and color schemes.
- ro_dobre 2y ago
  Love to hear that, let me know what other features you'd like to see :)
deleted 2y ago
[deleted]
AnnaKonda 2y ago
Haha, this project is amazing! Can it load more websites?
- ro_dobre 2y ago
  Thanks a lot! Yep, I'm working on an infinte loading feature, just trying to find a good heuristic so it doesn't start listing irrelevant websites. Thanks for the suggestion!
araes 2y ago
It's a neat idea, and bookmarked it just for ideas on possible designs. Weird parts though. Does not ever seem to return a website I have ever heard of. And (although it may have nothing to do with you, and everything to do with the modern web) a lot look rather algorithmically designed. Plus, there's sites like these people: https://www.beltnbags.com.au/ https://www.beltnbags.com.au/ that register on every pure tone and near pure-tone search because they're listed as (#0000ff, 24.9%, #f7f7ff, 13.0%, #f7f7f7, 13.0%, #00ff00, 24.2%, #ff0000, 24.9%) However, if you go to their website, I don't know what you're scraping, cause it sure does not "look" like its 1/4 red, 1/4 green, 1/4 blue, and 1/4 of dueling eggshell white. It looks white, black, teeny red labels, and lots of brown clothes.
- ro_dobre 2y ago
  There are definitely some odd ones. The screenshots were done a few hours after the initial crawl, some pop-ups may have appeared on the first crawl. The crawler also blocks image loading (to not take image colors into account because it's screenshotting the entire page and creating a frequency of each color from the screenshot, pixel-wise) so it's possible that some fallback CSS was displayed instead of the images. I definitely have to refine it more and it's far from perfect. There are only ~25k websites indexed and I used a list of 1M most popular websites created by the developers of a Chrome extension, based on what most people were visiting. I tried my best to sort by popularity but I'm sure that the list is not perfect. Plus, some websites are really unfriendly to crawlers and, as much as you try to hide it (by faking user agents, using realistic window dimensions, proxies, etc.), some will still detect you.
- 0a6d187a23dfb4c 2y ago
  > Does not ever seem to return a website I have ever heard of. Just playing devil's advocate, there are a LOT of domains out there. A quick search says there's still well over a billion websites on the internet.
  - araes 2y ago
    Totally realize there's a lot of websites. Just figured with fairly simplistic color tone choices, you might get something like Google 'occasionally'. Google as example: Current logo: #4086F4 (blue), #EB4132 (red), #FBBD00 (yellow), #31AA52 (green). Google (at least on my search) only shows up as a leftover of somebody else's search. Of course, there's also only 9 results.

96 points by ro_dobre 2y ago

D_R_Farrell 2y ago

I wonder if you open up this to also deconstruct color variables (-var) into their respective hex codes if you'd get more results of sites that use design systems. Have you explored that much at all? Cool project!

ro_dobre 2y ago
That's a really interesting idea that I didn't get to experiment on, unfortunately. It's currently screenshotting each crawled website and extracting the colors from the pixel values. One filter I added to make the palette a bit more relevant is that I try to find out which is the background color (the most used color) and only pick colors that are at least somewhat contrasting with the background (assuming that websites would follow somewhat basic color contrast rules). When I tried extracting the palettes from CSS rules, what I ran into is that each website uses lots of vendors that render stuff (cookie consents, promotional pop-ups, etc.) and it's hard to distinguish what's rendered and what's not with pure CSS selectors, so I resorted to decomposing the website's theme from the perspective of a user. (through a screenshot) It's a great idea nonetheless, I'm sure I would have some good results with it if I manage to filter out the noise, thanks a lot!

pvaldes 2y ago

Very useful to spot webs with a particular theme, blue webs, sunny webs... but also all webs from a particular logo or party that often use the same palettes again and again as brand image. Allowing to chose proportions of colors could be even better. Hypothetical example: Lets suppose that I'm a policeman. How much percentage of black, white and red is in a classical svastika?. Then browse the dark web for finding all webs that contain this particular white, black and red in such combination Other possibilities: I want to know if my [bank/company/chess club] web is being cloned and used to criminal purposes without my knowledge. My logo contains magenta, lime green and burnt orange. Find me webs with this combination that aren't in a list of official webs from my company. Even better, check it at real time on email to spot scammers impersonating me on real time. The applications are endless: I want to detect webs that sell fake clones of my products; or find photos of pills of a particular shade of blue. Or don't show me webs with a high content in warm tones associated with porno images... Yep, that could be a really revolutionary startup.

ro_dobre 2y ago
Haha, lots of great ideas! I'm definitely considering adding an option to search by color proportions, just trying to find a UI/UX that is not confusing for people. I also have a lot more content (i.e.: the full HTML code of the website, the fonts and typography options) that I extracted for each website but it's tricky to come up with an easy to use UI/UX for this as a search filter, given that there's not many other websites that do this.

qingcharles 2y ago

I was looking for neon green sites for inspiration... Google says this site is now badware: millionfollowersbootcamp[.]com

ro_dobre 2y ago
Yeah, I need to curate the existing list of websites and perhaps swap the existing list for one with more popular websites. I'll definitely have a look, thanks!

muratsu 2y ago

This is really cool. We've recently built a website catalog (https://www.rapidpages.com/inspire https://www.rapidpages.com/inspire) and one of the difficult choices we needed to make was whether to include images on the site or not when extracting the color palettes. I'm curious if you've also experimented with this?

ro_dobre 2y ago
The site is looking great! I think that screenshots are a must, it's a lot easier for the end-user to preview the website's content. If you're referring to screenshots where certain colors are used, I don't think that's necessary. The first version of it, I didn't take any screenshots and it felt very empty. It's a bit hard to manage the images and deliver them rapidly, in an optimized form, but I think it's worth it.
- muratsu 2y ago
  Oh, I was referring to calculating colors used on a website. To give you an example, we've stumbled upon pages where there's a product on the page that contained the color red, and nowhere else on the page was anything red. Our naive approach to finding avg colors was to simply take a screenshot of the page and process that image. Unfortunately, this method produces inaccurate results like red being part of the average colors when the design system has no red. I was curious if you managed to work around this issue. Happy to chat more if you're interested, you can reach me at murat[at]rapidpages.io
  - ro_dobre 2y ago
    Ah, sorry, I understand what you mean now. Yeah, I've also filtered out images when calculating the color percentages and there are definitely some funky results. There's definitely a middle-ground approach here, I just haven't found a good enough heuristic. It's true that images are part of the branding and they impact the color palette, like banners and logos, so they should be considered in some quantity. I thought of maybe averaging the results of image/no image but I don't have enough test data to validate it yet. Let's chat and discuss more! I'll also add you on Twitter/X

getwiththeprog 2y ago

Finally, I can see who is using this years Pantone style guide!

toataziuatestam 2y ago

Reeeally loving this one, great job !!

unittester 2y ago

Great Stuff, I could imagine this being handy when looking for design inspiration.

ro_dobre 2y ago
Haha, thanks! That's exactly what I had in mind when building. I've also collected what fonts and typography styles different websites use so I'm looking to create a visual for that as well, thanks for the feedback!

amadeuspagel 2y ago

Amazing. Maybe you can show the chosen color as the background of the hex code?

ro_dobre 2y ago
That's a great suggestion, will definitely do that in the next iteration. Thanks a lot!

GrumpyNl 2y ago

I like it and im gonna use it, great for inspiration and color schemes.

ro_dobre 2y ago
Love to hear that, let me know what other features you'd like to see :)

deleted 2y ago

[deleted]

AnnaKonda 2y ago

Haha, this project is amazing! Can it load more websites?

ro_dobre 2y ago
Thanks a lot! Yep, I'm working on an infinte loading feature, just trying to find a good heuristic so it doesn't start listing irrelevant websites. Thanks for the suggestion!

araes 2y ago

It's a neat idea, and bookmarked it just for ideas on possible designs. Weird parts though. Does not ever seem to return a website I have ever heard of. And (although it may have nothing to do with you, and everything to do with the modern web) a lot look rather algorithmically designed. Plus, there's sites like these people: https://www.beltnbags.com.au/ https://www.beltnbags.com.au/ that register on every pure tone and near pure-tone search because they're listed as (#0000ff, 24.9%, #f7f7ff, 13.0%, #f7f7f7, 13.0%, #00ff00, 24.2%, #ff0000, 24.9%) However, if you go to their website, I don't know what you're scraping, cause it sure does not "look" like its 1/4 red, 1/4 green, 1/4 blue, and 1/4 of dueling eggshell white. It looks white, black, teeny red labels, and lots of brown clothes.

ro_dobre 2y ago
There are definitely some odd ones. The screenshots were done a few hours after the initial crawl, some pop-ups may have appeared on the first crawl. The crawler also blocks image loading (to not take image colors into account because it's screenshotting the entire page and creating a frequency of each color from the screenshot, pixel-wise) so it's possible that some fallback CSS was displayed instead of the images. I definitely have to refine it more and it's far from perfect. There are only ~25k websites indexed and I used a list of 1M most popular websites created by the developers of a Chrome extension, based on what most people were visiting. I tried my best to sort by popularity but I'm sure that the list is not perfect. Plus, some websites are really unfriendly to crawlers and, as much as you try to hide it (by faking user agents, using realistic window dimensions, proxies, etc.), some will still detect you.
0a6d187a23dfb4c 2y ago
> Does not ever seem to return a website I have ever heard of. Just playing devil's advocate, there are a LOT of domains out there. A quick search says there's still well over a billion websites on the internet.
- araes 2y ago
  Totally realize there's a lot of websites. Just figured with fairly simplistic color tone choices, you might get something like Google 'occasionally'. Google as example: Current logo: #4086F4 (blue), #EB4132 (red), #FBBD00 (yellow), #31AA52 (green). Google (at least on my search) only shows up as a leftover of somebody else's search. Of course, there's also only 9 results.