6 ms·
What technologies are under your site
Super Happy Dev House at Google today lightning talk presenter - Ken
- chime 15y agoMinor bug: For my site, it says YUI. Having written it from scratch, I'm pretty sure there is no YUI anywhere. It appears to match jqueryui.js as yui.js. Link: http://underthesite.com/technologies/YUI-Library/matchers/124?site=zetabee.com http://underthesite.com/technologies/YUI-Library/matchers/12...
- tectonic 15y agoYou're right - This was a community-submitted matcher. I just reported the matcher as inaccurate: http://underthesite.com/technologies/YUI-Library/matchers/124?site=zetabee.com http://underthesite.com/technologies/YUI-Library/matchers/12...
- spontaneus 15y agoI had this same idea a while back, but never had the time to build it. It could be a great way to generate some affiliate money. Good luck!
- clobber 15y agoTested this on a website written in ColdFusion and it's reporting back as ASP.NET
- mikelbring 15y agoSame idea as http://builtwith.com/ http://builtwith.com/ ?
- ashwinurao 15y agoA Chrome/FF extension would be much more useful.
- tectonic 15y agoThere is a bookmarklet. Browser extensions are forthcoming.
- Nican 15y agoI would like to see the list of websites using IIS.
- deleted 15y ago[deleted]
- tectonic 15y agoHey guys, I made this site and just gave a talk about it at SHDH. Someone must have submitted it. Thanks for all your feedback, I really appreciate it!
- jbhelms 15y agoIt incorrectly stated that my site uses cold fusion.
- mmaunder 15y agoAwesome! Suggestions: Do a reverse lookup on the site's IP and add matchers for the hostnames that show up. Mention if they're round robining their DNS. Add nmap OS fingerprinting. Do a traceroute and log the IP of the closest router (final hop) to the site and add matching for that. Add a wiki interface and build a crunchbase like app. Add archiving of data and monitoring over time (as netcraft did in their original app).
- JoshTriplett 15y agonmap OS fingerprinting seems a bit aggressive, but reverse-DNS checking seems like a great way to check which hosting platform a site uses.
- deleted 15y ago[deleted]
- 16s 15y ago
- araneae 15y agoNice tool! Usually when I see a site I like I view source to see what it's made in, but it's not always that obvious.
- shuri 15y agomy 2c. Nice design (graphics). I would make the first page seem less busy. BuiltWith is going to be a tough competitor. You need to match them (precision/recall) and add stuff that they don't have (trending techs? Add info: who is the host provider? where in the world is it hosted? Response times? Bad link stats?...?)
- namank 15y agoCool! Though cannot reach this site: http://www.olin.wustl.edu/pages/default.aspx http://www.olin.wustl.edu/pages/default.aspx ...why
- christangrant 15y agofyi, If you click on the kid on the bottom right corner of http://underthesite.com/ http://underthesite.com/ he blinks.
- arc_of_descent 15y agoNice. but you have to click on the kid, NOT!
- deleted 15y ago[deleted]
- bstar 15y agoSo yahoo.com is using jQuery now? There's like 10 references to YUI and zero for jQuery in the source... interesting algorithm.
- tectonic 15y agoIt was found on yahoo.match.com, which is a bug because it shouldn't go off domain. I'll look into it. http://underthesite.com/technologies/jQuery/matchers/5?site=yahoo.com http://underthesite.com/technologies/jQuery/matchers/5?site=...
- JoshTriplett 15y agoI love this. I particularly like the broad definition of "technologies", and the variety of ways to write matchers. More importantly, I like that users can easily add their own technologies and matchers.
- a3_nm 15y agoI love how it gets confused by Hacker News: http://underthesite.com/sites/news.ycombinator.com http://underthesite.com/sites/news.ycombinator.com
- tectonic 15y agoI challenge you to come up with a matcher for a technology detectable on HN. It's hard!
- kingkilr 15y agoInteresting, but seems a little primitive compared to the cleverness in https://github.com/mitsuhiko/probe https://github.com/mitsuhiko/probe
- tectonic 15y agoDoes probe try and fetch signal urls? (Like admin pages.). UnderTheSite.com makes a point of only looking at data that would normally be fetched by your browser. Probing a site for urls could be considered offensive.
- kingkilr 15y agoYes, it does probe at site URLs, you can look at libprobe.py to see all the indicators it takes into account.
- bfirsh 15y agoAlso at http://probe.ep.io/ http://probe.ep.io/
- aphexairlines 15y agoThis doesn't really tell you anything about a site. Most of the interesting stuff happens on the server side.
- djcapelis 15y agoThat's getting less and less true with the amount of stuff happening in Javascript.
- aphexairlines 15y agoUnderthesite itself does its work behind a server: http://underthesite.com/sites/underthesite.com http://underthesite.com/sites/underthesite.com All you can tell from it querying itself is that the work happens somewhere behind rails.
- JoshTriplett 15y agoWhat triggers the error message "The pattern that you entered appears to be too general - can you make it more specific for this technology?"? I tried to add a technology for the use of rel="nofollow", using an XPath expression //a/@rel[contains(., "nofollow")] , and got that error message. What do I need to do to make it more specific?
- tectonic 15y agoGood question, I think rel=nofollow must match too many websites. I'll look into it.
- JoshTriplett 15y agoPlease consider allowing a small icon for a technology, to make lists of technologies more immediately recognizable.
- deleted 15y ago[deleted]
- deleted 15y ago[deleted]
- JoshTriplett 15y agoLooks like you've already started to get spam in the technologies list.
- nbertram 15y agoNice site guys; reminds me of www.quarkbase.com in some ways. But I like the focus on what's under the hood - keep up the good work.
- aorshan 15y agoNoticed that not a single website is using django. Am I missing something?
- tectonic 15y agoI'm not sure that there is a django matcher yet. Django is hard to reliably detect. Do you know of a good signature (like a header or form token) that it always uses?
- lloeki 15y agoYou could implement "maybe" matchers, that look for stuff which has a good chance of telling you what's underneath but not with 100% certainty. Now, the following is a hack, but CSRF stuff[0] gives a good indication: If there's a form, there's a good chance you'll find a construct like: <form action="." method="post"><input type='hidden' name='csrf_token' value="thetokenvalue" /> Also, there may be a 'csrftoken' value in the cookie. You will also most probably find the jQuery function that sets X-CSRFToken on XHRs (see the doc[0] at #ajax). For prototype it'll look like this [1] [0] https://docs.djangoproject.com/en/dev/ref/contrib/csrf/ https://docs.djangoproject.com/en/dev/ref/contrib/csrf/ [1] http://stackoverflow.com/questions/5551914/protecting-prototype-js-based-xhr-requests-against-csrf http://stackoverflow.com/questions/5551914/protecting-protot...
- nfm 15y agoBang on for my app's site! Very cool. In terms of features, I'd love to see more emphasis on the aggregate/comparison data. For example, most popular server side framework, most popular JS libraries, most popular hosting platforms and so on. I like it a lot :)
- tectonic 15y agoIt's not fully-baked yet, but I'm starting to do just that: http://underthesite.com/compare/jquery_and_prototype-javascript-framework http://underthesite.com/compare/jquery_and_prototype-javascr... Thanks for the feedback!
- ArchD 15y agoTo test the d3 matcher I just added, I tried the following URLs and get the error "Sorry, we were unable to reach the site": http://whatdoyouworkfor.appspot.com/index.html http://whatdoyouworkfor.appspot.com/index.html http://dustinphoto.iriscouch.com/gerrit/_design/app/index.html http://dustinphoto.iriscouch.com/gerrit/_design/app/index.ht... I could reach them with no problems. These sites are just from some comments on this thread: http://news.ycombinator.com/item?id=2746449 http://news.ycombinator.com/item?id=2746449
- wiradikusuma 15y agoI checked my website, http://underthesite.com/sites/lelanggokil.com http://underthesite.com/sites/lelanggokil.com Your analyzer mistakenly detected my website using Microsoft IIS and ASP.NET. Which is weird since it also detected Google App Engine (which is correct). ASP doesn't run on GAE. Just a thought, do you think website owners should mention technologies they use in HTTP header? For example your analyzer can't detect that I'm using Java and Spring framework.
- NARKOZ 15y agohttp://lineofthought.com/ http://lineofthought.com/ is much better
- deleted 15y ago[deleted]
- westi 15y agoThis is cool. Some bugs in the information returned though. For example: http://underthesite.com/technologies/WordPress-Batcache-Plugin http://underthesite.com/technologies/WordPress-Batcache-Plug... "WordPress Batcache Plugin is closed source. " Batcache is Open Source :)
- tectonic 15y agoFixed, thank you.
- paraschopra 15y agohttp://builtwith.com/ http://builtwith.com/ is another great alternative!
- dorkitude 15y agobuiltwith.com does a better job with my site. Under the Site misses MooTools, which is being loaded asynchronously on my site via Google Libraries API Impressively, Built With even managed to detect that I was making an AJAX call to StackOverflow's API.
- fus 15y agoI cannot stand that unreal cogs image in bottom-right corner . Please make sure search works with special characters.
- arctangent 15y agoGreat idea. How about saving the results you generate and allowing people to search for sites based on the technologies they use. For example, I might want to see all sites using jQuery which are using something other than Apache.
- tectonic 15y agoResults are saved just not exposed yet. I'm still trying to figure out how best to do that.
- jpr 15y agoI'm going to build my next site in UTF-8, just like news.ycombinator.com.
- yeahsure 15y agoGreat idea - I like it. I've been using a Firefox add-on for this, http://wappalyzer.com http://wappalyzer.com doesn't give as much info, but works fairly good.
- mcorrientes 15y agoAt first I thought this was a copy of lineofthought.com, but it has been created in a way it can stand for it's own.
- davidcuddeback 15y agoIt says that my company's site [1] is running ASP.NET on Microsoft IIS, which it's not. To be fair, it also mentions Ruby on Rails, Apache, and Phusion Passenger, which are all correct. Aside from these minor glitches, this is a pretty cool project. [1] http://identified.com http://identified.com
- tectonic 15y agoThis should be fixed now.
- deleted 15y ago[deleted]
- mike-cardwell 15y agoI'd add SSL/https. I'd also add "Strict-Transport-Security" and "Content-Security-Policy", both of which can be seen by looking at HTTP response headers.
- SallyG 15y agoGood start. Looks like http://w3techs.com/sites http://w3techs.com/sites does a bit more
- eurohacker 15y agodoes it recognize php frameworks - like cakephp, codeigniter, or libraries like flourish