8 ms·

Show HN: Diff.io

194 points by wickedlogic 11y ago

omarchowdhury 11y ago
Could use this to monitor competitor's landing pages and see what they split test.
- deleted 11y ago
  [deleted]
llamataboot 11y ago
I see some green blobs, some pricing info, and hear some marketing speak about letting my CMS be a "self-actuated change awareness system". I want to know how I integrate this API into my tests, what exactly it tells me, and how to use it. How about a free plan to test it out? How about developer docs? How about something more than a few screenshots and a 30 second video that is incomprehensible marketroid speak? Sorry for the harshness, but I really think visual diff tools are needed for integration testing. I want to figure out which ones are flexible and how they can be used. This site helps me do none of that.
- wickedlogic 11y ago
  Harshness is fine :) There is still more work to be done on making the service more flexible, mostly just opening up the rest of the api on my end. How would you describe a CMS that self documents externally? Perhaps this helps a little: http://blog.diff.io/show-hn-diff-io/ http://blog.diff.io/show-hn-diff-io/ (?)
  - egwor 11y ago
    This could be used as a way to make sure a web 2.0 app hasn't got some error and lost some content... e.g. some pane is empty
    - giovannibajo1 11y ago
      Have a look at Applitools for a similar tool with advanced features
    - rplnt 11y ago
      Would it find errors here? :) https://i.imgur.com/xg3BSNs.png https://i.imgur.com/xg3BSNs.png
      - wickedlogic 11y ago
        Hah, sounds like I need start tweaking that theme's css. That's bad. Email matt@diff.io the device/browser/resolution?
        rplnt 11y ago
        Weirdly, I can't reproduce it now, so you either fixed it, or it was a bug on my side (browser, extension, ..?). Anyway, it was a screenshot from Opera 29 (Chromium 42) on Windows 7 with uBlock as one possibly related extension.
        greggman 11y ago
        I got the same the first time but can't repo either
      - cstad 11y ago
        I've been getting the same kind of display errors on lots of different sites in chrome. Inline elements aren't given enough width, and then when I go to inspect them it is fixed.
- bfirsh 11y ago
  You might like this: https://github.com/bfirsh/needle https://github.com/bfirsh/needle Would be cool to get a Diff.io driver for it, or something like that...
  - Flenser 11y ago
    or these: https://hub.com/cburgmer/csscritic https://hub.com/cburgmer/csscritic - JavaScript, runs in browser https://hub.com/Huddle/PhantomCSS https://hub.com/Huddle/PhantomCSS https://hub.com/bslatkin/dpxdt https://hub.com/bslatkin/dpxdt - Python, Deploys to App Engine / CloudSQL / Compute Engine https://hub.com/BBC-News/wraith https://hub.com/BBC-News/wraith - Ruby https://hub.com/sindresorhus/pageres https://hub.com/sindresorhus/pageres - JavaScript, nodejs, PhantomJS
    - billyhoffman 11y ago
      I think you meant: https://github.com/cburgmer/csscritic https://github.com/cburgmer/csscritic - JavaScript, runs in browser https://github.com/Huddle/PhantomCSS https://github.com/Huddle/PhantomCSS https://github.com/bslatkin/dpxdt https://github.com/bslatkin/dpxdt - Python, Deploys to App Engine / CloudSQL / Compute Engine https://github.com/BBC-News/wraith https://github.com/BBC-News/wraith - Ruby https://github.com/sindresorhus/pageres https://github.com/sindresorhus/pageres - JavaScript, nodejs, PhantomJS
      - Flenser 11y ago
        whoops, yes. My extension to highlight words I'm interested in on HN must have removed them from the input box when I edited it.
morgante 11y ago
This seems like it could be great—can I drop this into my CI workflow to find visual regressions? That's the question you should be answering. Not trying to sell some "self-actuated change awareness" mumbo jumbo. The video literally sounds like an infomercial for a cult. Sorry for being harsh, but it sounds like you have some cool technology which is unfortunately overwhelmed by terrible marketing.
- wickedlogic 11y ago
  Good feedback, thanks. I do see your pointed about CI workflow, but I think visual regressions are the current obsession and a limited use case. A very valuable and useful use case, granted. But if something like diff.io was in place in your browser, in your content publishing system, in your social networks, having it just in your CI would seem limited. Would you agree?
  - morgante 11y ago
    > But if something like diff.io was in place in your browser, in your content publishing system, in your social networks, having it just in your CI would seem limited. Would you agree? No, I don't. I have about 0 interest in seeing diffs of social networks or most web pages. It might make sense in a CMS but even there I think most people are better served by traditional textual diffs. It sounds like you've found a good solution for a medium problem (preventing visual regressions) but want to expand it into a "movement."
- nodesocket 11y ago
  One solution would be to use an existing GitHub integration to trigger a screenshot on diff.io. Shameless plug: I'm the founder of https://commando.io https://commando.io, and we have a GitHub service that allows users to run scripts (we call them recipes) on push of a repo on servers. You could write a simple recipe in Commando.io in bash to trigger the screenshot on diff.io: http://public.recipes/Xr1eCr http://public.recipes/Xr1eCr
- unclebucknasty 11y ago
  >mumbo jumbo...cult...terrible marketing Meh, I don't think it's that big a deal, nor do I think the marketing is bad. You clearly understood that there was some "cool technology" there, right? And, CI is just one use case, right? Still, while the creator doesn't appear to have designed it so narrowly, you are nevertheless seeing the potential application and inquiring. I'd say that's a marketing win. Of course, with more interest and feedback, the product and message can be honed. So, maybe your feedback will help the creator, but there are better ways to communicate your opinion. Don't want to be "harsh"? Then, don't be. No need for the side commentary about cults, mumbo-jumbo, and terrible marketing. Those phrases contributed nothing to your otherwise potentially useful feedback. But, they may be discouraging early words for someone who's just poured a lot into a project.
bgraves 11y ago
Hmm, is this built on top of url2png by any chance? :)
- wickedlogic 11y ago
  Yup, at least that is the primary source of screenshots :)
Gigablah 11y ago
Your API documentation isn't public. That's an instant turn-off.
- wickedlogic 11y ago
  The dashboard api is still a work in progress, but the actual api is documented here: http://blog.diff.io/talking-to-api-diff-io-via-json-web-token/ http://blog.diff.io/talking-to-api-diff-io-via-json-web-toke...
stephentmcm 11y ago
I've half built a tool to do this using PhantomJS, and a really half-baked API. The image grab part is easy. The diffing is what's hard... well sort of. What I tried and what this site appears to do is a straight pixel change detection, which fails to account for how important that change is. Minor things change on a site all the time it's catching major breaking changes that's hard, say a CSS rule change that looks fine on desktop but ruins the site on mobile.
- Kimitri 11y ago
  I've used CasperJS, PhantomCSS and PhantomJS for visual regression testing on dozens of Drupal sites and, while not a perfect solution by any means, it's worked pretty well. PhantomCSS seems to be pretty configurable and makes it rather easy to detect breaking changes.
  - stephentmcm 11y ago
    O nifty I hadn't seen PhantomCSS before.
- koudelka 11y ago
  When I built something like this, I used http://wkhtmltopdf.org http://wkhtmltopdf.org and http://pdiff.sourceforge.net http://pdiff.sourceforge.net.
foton1981 11y ago
https://github.com/bslatkin/dpxdt https://github.com/bslatkin/dpxdt
- wickedlogic 11y ago
  It is an option, in fact there are a lot of open source options. The subscription model we offer is one way to gain the ability without having to worry about the complications of hosting it.
- heyalexej 11y ago
  Here's a Video that explains how the Google Survey team uses perceptual visual diffs (dpxdt) in their deployments: https://www.youtube.com/watch?v=UMnZiTL0tUc https://www.youtube.com/watch?v=UMnZiTL0tUc
audaxion 11y ago
https://github.com/facebookarchive/huxley https://github.com/facebookarchive/huxley
bcjordan 11y ago
Very nice! Your CMS integration looks like a fantastic way to mitigate the onboarding pain of most integration testing systems. Definitely a useful type of test to have and certainly comes from a place of pain. Last year I worked on a bootleg similar project during the YC Hacks event—it was hard to get it to work right. Recently Applitools Eyes[0] started gaining popularity for CI-based visual testing, I hooked it up to some Selenium integration tests at work earlier this year and the things it catches have consistently amazed me. Catches nearly all of the bugs that manage to slip past the typical unit / end-to-end tests. One hard part they navigated well has been the interface for being able to review changes, set new baselines, and set a certain area as "ignored". IIRC they even use some fancy computer vision algorithms to handle slight variations in screenshots (e.g. font alignment false positives). [0]: https://applitools.com/ https://applitools.com/
gcb0 11y ago
the license plates image is misleading as hell! for a moment you think it is smart enough to realize subject and background and only show you changes on the normalized objects.... but it is actually just an edited image on the other side to add a clean element for the diff.
- wickedlogic 11y ago
  This is strictly over images, not video. I guess I see your point, didn't consider it that way at all, since it was a picture of my plate and a modified img change to generate populate the diff. I could swap it with the real plate and the real plate with a sticker, or something. As a regression I thought it was straightforward. :| Was the confusion that you thought it was doing something like wordlens: http://1.bp.blogspot.com/-X95Ga7iMP74/T_vjrp2PF2I/AAAAAAAAFMA/bNfjJpp5qhM/s1600/word-lens-app-nemo-s-travels.png http://1.bp.blogspot.com/-X95Ga7iMP74/T_vjrp2PF2I/AAAAAAAAFM...
  - gcb0 11y ago
    yep. i assumed it did some image transformation to find the minimum diff between two images as to compensate for two differently angled pictures or something.
OneTwoFree 11y ago
The site shows an SSL warning message. The certificate details says it was issued by "Avast untrusted CA".
- swatkat 11y ago
  It's more likely a problem with Avast security software, not the website itself. Are you using Avast Anti-Virus or Web/Mail Shield? Avast installs a root certificate to intercept and scan HTTPS traffic[0]. [0] https://itnerd.wordpress.com/2015/05/21/avast-responds-to-my-post-about-their-anti-virus-product/ https://itnerd.wordpress.com/2015/05/21/avast-responds-to-my...
  - psykovsky 11y ago
    Why do antivirus seem to be more malicious than the virus they fight?
  - iLoch 11y ago
    Well that sounds like spyware if I've ever heard it.
  - OneTwoFree 11y ago
    Thanks. I use Avast Anti-Virus. I didn't know they are doing man-in-the-middle. But as I learned, this is quite common for modern antivirus softwares, so I turned off this option in Avast.
empressplay 11y ago
Your "bootstrap" level should be free. I don't see any bootstrapping startup paying for a service they don't need / can build themselves for free (if they're really desperate for it.) You want to capture that potential future business, and hope that you can convert them to paying customers as they become profitable; you don't want them to go "hey, great idea, but I'm not paying for that" and then implement their own solution.
- bliti 11y ago
  Isn't it better to just sell to people who can pay? I mean, why being so against asking for money? Its strange how you say that if they really needed it they could just build it. That's a bit of a stretch. Why build something you can pay for and focus on making money instead? Dunno. I'm just confused.
  - tperson 11y ago
    You barely have to build it, there is a great open source project already available. Not sure if it does everything diff.io does but it's a great resource to do a lot of it. https://github.com/webdriverio/webdrivercss https://github.com/webdriverio/webdrivercss https://github.com/webdriverio/webdrivercss-adminpanel https://github.com/webdriverio/webdrivercss-adminpanel
  - rplnt 11y ago
    You are severely limiting them if they don't pay. So I guess hooking them (or at least letting them try it) and then upselling sounds better for both parties.
    - bliti 11y ago
      I have the hardest time upselling from free. But its pretty easy to go from $300 to $500.
  - tlrobinson 11y ago
    Maybe, but many people (myself included) may be reluctant to use it if they can't even try it out without paying for it. Especially if the core product can be replicated with 3 lines of code using open source libraries.
nbevans 11y ago
Tip: Remove the "plans" page. This thing is too early stage to start talking about $ plans. Nobody even knows what the hell this is or what it is good for. Your business model should be as follows: 1. Make something new, useful and free. 2. Get people hooked on it, like a drug. 3. Grow a modest user base. 4. Introduce "value added" features, marked as preview or beta, for enterprise customers and integration into other web services. 5. Once features added in step 4 have matured, remove the beta clauses and slap a price tag on them.
- liquidise 11y ago
  i disagree with this assessment. What harm is there in having a pricing page? I'd support offering a free option, but why on earth would you ever disallow willing customers from paying you money? On the contrary, i am now tempted to build a "Buy" button on a basic marketing page before i ever write a line of code on future product ideas.
  - Mandatum 11y ago
    "Plans" was the first thing I clicked on because I wanted to try it out, and then there's no free option so I clicked that little "X" by my tab. I actually have a use case for this and so do a few people I know, and I'd love to try it out.. But you've got to be kidding if you think I'm going to give out my payment data to try a website version of FollowThatPage.
- blowski 11y ago
  I agree that the landing page needs some work to make it clear what the product does, and why I would want to pay for it. However I disagree that you must start by offering the service for free. By charging from day one, you can limit your user base only to those people who are prepared to pay for it. You're less likely to get explosive growth this way, but it's easier to keep the service running as you're not subsidising freeloaders. One of the assumptions in an MVP is that people are prepared to pay for the service, and you can only establish that by requiring people to part with cash as early as possible.
- bliti 11y ago
LaFolle 11y ago
Nice idea. But it would have been great if there could have been a trial/free version. I would like to test the service so that I understand what exactly it is about. Watching a video and reading text does not help much until you try it out by yourself.
joshcanhelp 11y ago
This looks fantastic. I was just looking into doing this "manually" with an automated screenshot process for a site I'm refactoring part by part.
bliti 11y ago
Your landing page might benefit from using wordpress as an example. I saw the service and immediately thought about how to integrate this into wp. Maybe even turn it into a wp plugin? Dunno. This does seem useful.
uptownfunk 11y ago
I want to know what this does, the youtube video does not explain clearly in layman's terms.
mastercoms 11y ago
Nice theme. http://startbootstrap.com/template-overviews/grayscale/ http://startbootstrap.com/template-overviews/grayscale/
schappim 11y ago
If you want a DIY version of this, try ImageMagick's compare command: compare bag_frame1.gif bag_frame2.gif compare.gif Documentation: http://www.imagemagick.org/Usage/compare/ http://www.imagemagick.org/Usage/compare/
- no_gravity 11y ago
  Interesting. And to render the html you can use cutycapt. Which is in the Debian repos too. I just tried it, and this rendered a nice diff of two pages for me: cutycapt --url=http://www.xyz.com/1 --out=1.png cutycapt --url=http://www.xyz.com/2 --out=2.png compare 1.png 2.png diff.png
  - easytiger 11y ago
    Now make it a webservice and charge $200/month!
    - prottmann 11y ago
      apt-get install the services and use them on my server ;-)
    - programmernews3 11y ago
      First part already done: http://archive.is/ http://archive.is/
    - _pmf_ 11y ago
      > Now make it a webservice and charge $200/month! I don't actually see the problem here. They provide a real service, so they can demand real money. There are multi billion dollar companies that do less for their users.
  - danieltillett 11y ago
    Very interesting approach.
  - dghf 11y ago
    This is why I love HN. Someone launches a web service with a flashy site, lots of marketing blurb and monthly plans from $10 to $200. And in the HN comments thread, two random people reimplement the functionality in three lines.
Permit 11y ago
I just want to say that I strongly disagree with the people here who are complaining about your pricing page. You need to validate that people will pay money for this product, and the quickest way to do so is to ask up front. I really don't buy this "Give it away for free, get them hooked and then staple on paid features" approach. PG said it best: "Better to make a few users love you than a lot ambivalent."[1] One of the best ways to find out if people love what you're building is to ask them to pay for it. [1]http://www.paulgraham.com/13sentences.html http://www.paulgraham.com/13sentences.html
olivierkaisin 11y ago
Great project! This is really a need for monitoring consistency of UX of any website. Although I just think you should simplify your pricing model. I don't understand why you are talking about "requests". Who cares about bandwidth today anyway? Just make people pay for the number of pages the frequency of the checks. Also, you should allow people to set up e-mail alerts when some parts of their website pages change (e.g. payment forms). I would definitely pay for such product. : )
kasparloog 11y ago
Monitoring is a good idea. However, pixel-based comparison services are quite pointless for web pages. Banners, dynamic content etc. simply drives spam. Try Browserbite with its feature-based comparison. There are other regression-oriented tools out there as well that use pixel-based methods as well.
cekanoni 11y ago
your ssl is signed by unknown vendor so i wouldn't trust it..
- kennycox 11y ago
  This error occurs while you are using a self-signed certificate, it is advisable to go through the trusted third party certificates. You can use SSL installation checker to diagnosis troubleshoots. https://www.ssl2buy.com/wiki/ssl-installation-checker/ https://www.ssl2buy.com/wiki/ssl-installation-checker/ https://www.sslshopper.com/ssl-checker.html https://www.sslshopper.com/ssl-checker.html https://ssltools.websecurity.symantec.com/checker/views/certCheck.jsp https://ssltools.websecurity.symantec.com/checker/views/cert...
zoidb 11y ago
Hey nice! I've actually done something similar myself as a weekend project - https://page-watch.com https://page-watch.com Also checkout the following projects, https://visualping.io/ https://visualping.io/ and https://dpxdt-test.appspot.com/ https://dpxdt-test.appspot.com/ If you are using something like phantomjs to generate website screenshots I wonder how you are dealing with dynamic content. A lot of pages have continuous animations that can screw with simple image diff comparisons. For this reason you may want the option to limit your compare to a subset of the page.
wldlyinaccurate 11y ago
This reminded me of Wraith, a screenshot comparison tool which The BBC developed for integration testing: https://github.com/bbc-news/wraith https://github.com/bbc-news/wraith
ivanhoe 11y ago
Looks like a useful service. Just a word of warning, your SSL certificate is not trusted by Chrome and possibly some other browsers. You have to install Intermediate certificates to make it work for everyone.
- rb2k_ 11y ago
  Works for me on Chrome 43.0.2357.81 (OS X) Green lock + "The identity of this website has been verified by COMODO RSA Domain Validation Secure Server CA but does not have public audit records." No warnings.
  - hrbrtglm 11y ago
    Damn, I had the exact same problem in another thread with an unrelated domain name : https://news.ycombinator.com/item?id=9645297 https://news.ycombinator.com/item?id=9645297 Also a comodo certificate.
  - ivanhoe 11y ago
    Now it works for me to, I guess they fixed it in a meantime.
    - wickedlogic 11y ago
      Yes, thank you for pointing that out. I had recently installed new certs, but failed to point it to the ssl-bundled cert and instead it was just the main cert without the intermediates. Things are happier now: https://www.ssllabs.com/ssltest/analyze.html?d=diff.io&hideResults=on https://www.ssllabs.com/ssltest/analyze.html?d=diff.io&hideR... "A+"
jwklemm 11y ago
Nice work! We have a similar feature built into Ghost Inspector: https://ghostinspector.com/#visual-regressions https://ghostinspector.com/#visual-regressions -- It includes a tolerance setting so you can basically say "I only want to hear about it if the screenshot changes by xx% or more." Effective visual regression testing at scale is tough though and is really more of a secondary feature for us. If anyone is interested in other solutions, ImageMagick has a nice "compare" tool built-in. There are also plenty of open source projects like Huxley, PhantomCSS, etc. I recently saw a demo of Applitools Eyes (https://applitools.com/ https://applitools.com/) and it's quite powerful (though it's a paid service). Lastly, I did a presentation that involved visual regression testing here: https://www.youtube.com/watch?v=mK0l__jmpTA https://www.youtube.com/watch?v=mK0l__jmpTA (starts around 12:25)
- Addie_B 11y ago
  Applitools Eyes has a free account option as well, which grants access to all automated visual testing capabilities as in the paid packages - for a single user.
marcelerz 11y ago
You are probably using Blink-Diff for the image comparison. Isn't it? https://github.com/yahoo/blink-diff https://github.com/yahoo/blink-diff For example: https://raw.githubusercontent.com/yahoo/blink-diff/master/examples/YDN_Sort_output.png https://raw.githubusercontent.com/yahoo/blink-diff/master/ex... You can change the highlight-color to green (it is a configuration option), just like it is on the page there: https://diffio.global.ssl.fastly.net/v1/diffs/2015-05-28/eee2f08b89fa67377fd18ccb990c7d45f1464f95/montage.png https://diffio.global.ssl.fastly.net/v1/diffs/2015-05-28/eee... And the "montage" is called a "composition" in Blink-Diff.
- wickedlogic 11y ago
  It isn't Blink-Diff, but I had looked at that project at one point. It has some interesting options. :)

Show HN: Diff.io

194 points by wickedlogic 11y ago

omarchowdhury 11y ago

Could use this to monitor competitor's landing pages and see what they split test.

deleted 11y ago
[deleted]

llamataboot 11y ago

I see some green blobs, some pricing info, and hear some marketing speak about letting my CMS be a "self-actuated change awareness system". I want to know how I integrate this API into my tests, what exactly it tells me, and how to use it. How about a free plan to test it out? How about developer docs? How about something more than a few screenshots and a 30 second video that is incomprehensible marketroid speak? Sorry for the harshness, but I really think visual diff tools are needed for integration testing. I want to figure out which ones are flexible and how they can be used. This site helps me do none of that.

wickedlogic 11y ago
Harshness is fine :) There is still more work to be done on making the service more flexible, mostly just opening up the rest of the api on my end. How would you describe a CMS that self documents externally? Perhaps this helps a little: http://blog.diff.io/show-hn-diff-io/ http://blog.diff.io/show-hn-diff-io/ (?)
- egwor 11y ago
  This could be used as a way to make sure a web 2.0 app hasn't got some error and lost some content... e.g. some pane is empty
  - giovannibajo1 11y ago
    Have a look at Applitools for a similar tool with advanced features
  - rplnt 11y ago
    Would it find errors here? :) https://i.imgur.com/xg3BSNs.png https://i.imgur.com/xg3BSNs.png
    - wickedlogic 11y ago
      Hah, sounds like I need start tweaking that theme's css. That's bad. Email matt@diff.io the device/browser/resolution?
      - rplnt 11y ago
        Weirdly, I can't reproduce it now, so you either fixed it, or it was a bug on my side (browser, extension, ..?). Anyway, it was a screenshot from Opera 29 (Chromium 42) on Windows 7 with uBlock as one possibly related extension.
        greggman 11y ago
        I got the same the first time but can't repo either
    - cstad 11y ago
      I've been getting the same kind of display errors on lots of different sites in chrome. Inline elements aren't given enough width, and then when I go to inspect them it is fixed.
bfirsh 11y ago
You might like this: https://github.com/bfirsh/needle https://github.com/bfirsh/needle Would be cool to get a Diff.io driver for it, or something like that...
- Flenser 11y ago
  or these: https://hub.com/cburgmer/csscritic https://hub.com/cburgmer/csscritic - JavaScript, runs in browser https://hub.com/Huddle/PhantomCSS https://hub.com/Huddle/PhantomCSS https://hub.com/bslatkin/dpxdt https://hub.com/bslatkin/dpxdt - Python, Deploys to App Engine / CloudSQL / Compute Engine https://hub.com/BBC-News/wraith https://hub.com/BBC-News/wraith - Ruby https://hub.com/sindresorhus/pageres https://hub.com/sindresorhus/pageres - JavaScript, nodejs, PhantomJS
  - billyhoffman 11y ago
    I think you meant: https://github.com/cburgmer/csscritic https://github.com/cburgmer/csscritic - JavaScript, runs in browser https://github.com/Huddle/PhantomCSS https://github.com/Huddle/PhantomCSS https://github.com/bslatkin/dpxdt https://github.com/bslatkin/dpxdt - Python, Deploys to App Engine / CloudSQL / Compute Engine https://github.com/BBC-News/wraith https://github.com/BBC-News/wraith - Ruby https://github.com/sindresorhus/pageres https://github.com/sindresorhus/pageres - JavaScript, nodejs, PhantomJS
    - Flenser 11y ago
      whoops, yes. My extension to highlight words I'm interested in on HN must have removed them from the input box when I edited it.

morgante 11y ago

This seems like it could be great—can I drop this into my CI workflow to find visual regressions? That's the question you should be answering. Not trying to sell some "self-actuated change awareness" mumbo jumbo. The video literally sounds like an infomercial for a cult. Sorry for being harsh, but it sounds like you have some cool technology which is unfortunately overwhelmed by terrible marketing.

wickedlogic 11y ago
Good feedback, thanks. I do see your pointed about CI workflow, but I think visual regressions are the current obsession and a limited use case. A very valuable and useful use case, granted. But if something like diff.io was in place in your browser, in your content publishing system, in your social networks, having it just in your CI would seem limited. Would you agree?
- morgante 11y ago
  > But if something like diff.io was in place in your browser, in your content publishing system, in your social networks, having it just in your CI would seem limited. Would you agree? No, I don't. I have about 0 interest in seeing diffs of social networks or most web pages. It might make sense in a CMS but even there I think most people are better served by traditional textual diffs. It sounds like you've found a good solution for a medium problem (preventing visual regressions) but want to expand it into a "movement."
nodesocket 11y ago
One solution would be to use an existing GitHub integration to trigger a screenshot on diff.io. Shameless plug: I'm the founder of https://commando.io https://commando.io, and we have a GitHub service that allows users to run scripts (we call them recipes) on push of a repo on servers. You could write a simple recipe in Commando.io in bash to trigger the screenshot on diff.io: http://public.recipes/Xr1eCr http://public.recipes/Xr1eCr
unclebucknasty 11y ago
>mumbo jumbo...cult...terrible marketing Meh, I don't think it's that big a deal, nor do I think the marketing is bad. You clearly understood that there was some "cool technology" there, right? And, CI is just one use case, right? Still, while the creator doesn't appear to have designed it so narrowly, you are nevertheless seeing the potential application and inquiring. I'd say that's a marketing win. Of course, with more interest and feedback, the product and message can be honed. So, maybe your feedback will help the creator, but there are better ways to communicate your opinion. Don't want to be "harsh"? Then, don't be. No need for the side commentary about cults, mumbo-jumbo, and terrible marketing. Those phrases contributed nothing to your otherwise potentially useful feedback. But, they may be discouraging early words for someone who's just poured a lot into a project.

bgraves 11y ago

Hmm, is this built on top of url2png by any chance? :)

wickedlogic 11y ago
Yup, at least that is the primary source of screenshots :)

Gigablah 11y ago

Your API documentation isn't public. That's an instant turn-off.

wickedlogic 11y ago
The dashboard api is still a work in progress, but the actual api is documented here: http://blog.diff.io/talking-to-api-diff-io-via-json-web-token/ http://blog.diff.io/talking-to-api-diff-io-via-json-web-toke...

stephentmcm 11y ago

I've half built a tool to do this using PhantomJS, and a really half-baked API. The image grab part is easy. The diffing is what's hard... well sort of. What I tried and what this site appears to do is a straight pixel change detection, which fails to account for how important that change is. Minor things change on a site all the time it's catching major breaking changes that's hard, say a CSS rule change that looks fine on desktop but ruins the site on mobile.

Kimitri 11y ago
I've used CasperJS, PhantomCSS and PhantomJS for visual regression testing on dozens of Drupal sites and, while not a perfect solution by any means, it's worked pretty well. PhantomCSS seems to be pretty configurable and makes it rather easy to detect breaking changes.
- stephentmcm 11y ago
  O nifty I hadn't seen PhantomCSS before.
koudelka 11y ago
When I built something like this, I used http://wkhtmltopdf.org http://wkhtmltopdf.org and http://pdiff.sourceforge.net http://pdiff.sourceforge.net.

foton1981 11y ago

https://github.com/bslatkin/dpxdt https://github.com/bslatkin/dpxdt

wickedlogic 11y ago
It is an option, in fact there are a lot of open source options. The subscription model we offer is one way to gain the ability without having to worry about the complications of hosting it.
heyalexej 11y ago
Here's a Video that explains how the Google Survey team uses perceptual visual diffs (dpxdt) in their deployments: https://www.youtube.com/watch?v=UMnZiTL0tUc https://www.youtube.com/watch?v=UMnZiTL0tUc

audaxion 11y ago

https://github.com/facebookarchive/huxley https://github.com/facebookarchive/huxley

bcjordan 11y ago

Very nice! Your CMS integration looks like a fantastic way to mitigate the onboarding pain of most integration testing systems. Definitely a useful type of test to have and certainly comes from a place of pain. Last year I worked on a bootleg similar project during the YC Hacks event—it was hard to get it to work right. Recently Applitools Eyes[0] started gaining popularity for CI-based visual testing, I hooked it up to some Selenium integration tests at work earlier this year and the things it catches have consistently amazed me. Catches nearly all of the bugs that manage to slip past the typical unit / end-to-end tests. One hard part they navigated well has been the interface for being able to review changes, set new baselines, and set a certain area as "ignored". IIRC they even use some fancy computer vision algorithms to handle slight variations in screenshots (e.g. font alignment false positives). [0]: https://applitools.com/ https://applitools.com/

gcb0 11y ago

the license plates image is misleading as hell! for a moment you think it is smart enough to realize subject and background and only show you changes on the normalized objects.... but it is actually just an edited image on the other side to add a clean element for the diff.

wickedlogic 11y ago
This is strictly over images, not video. I guess I see your point, didn't consider it that way at all, since it was a picture of my plate and a modified img change to generate populate the diff. I could swap it with the real plate and the real plate with a sticker, or something. As a regression I thought it was straightforward. :| Was the confusion that you thought it was doing something like wordlens: http://1.bp.blogspot.com/-X95Ga7iMP74/T_vjrp2PF2I/AAAAAAAAFMA/bNfjJpp5qhM/s1600/word-lens-app-nemo-s-travels.png http://1.bp.blogspot.com/-X95Ga7iMP74/T_vjrp2PF2I/AAAAAAAAFM...
- gcb0 11y ago
  yep. i assumed it did some image transformation to find the minimum diff between two images as to compensate for two differently angled pictures or something.

OneTwoFree 11y ago

The site shows an SSL warning message. The certificate details says it was issued by "Avast untrusted CA".

swatkat 11y ago
It's more likely a problem with Avast security software, not the website itself. Are you using Avast Anti-Virus or Web/Mail Shield? Avast installs a root certificate to intercept and scan HTTPS traffic[0]. [0] https://itnerd.wordpress.com/2015/05/21/avast-responds-to-my-post-about-their-anti-virus-product/ https://itnerd.wordpress.com/2015/05/21/avast-responds-to-my...
- psykovsky 11y ago
  Why do antivirus seem to be more malicious than the virus they fight?
- iLoch 11y ago
  Well that sounds like spyware if I've ever heard it.
- OneTwoFree 11y ago
  Thanks. I use Avast Anti-Virus. I didn't know they are doing man-in-the-middle. But as I learned, this is quite common for modern antivirus softwares, so I turned off this option in Avast.

empressplay 11y ago

Your "bootstrap" level should be free. I don't see any bootstrapping startup paying for a service they don't need / can build themselves for free (if they're really desperate for it.) You want to capture that potential future business, and hope that you can convert them to paying customers as they become profitable; you don't want them to go "hey, great idea, but I'm not paying for that" and then implement their own solution.

bliti 11y ago
Isn't it better to just sell to people who can pay? I mean, why being so against asking for money? Its strange how you say that if they really needed it they could just build it. That's a bit of a stretch. Why build something you can pay for and focus on making money instead? Dunno. I'm just confused.
- tperson 11y ago
  You barely have to build it, there is a great open source project already available. Not sure if it does everything diff.io does but it's a great resource to do a lot of it. https://github.com/webdriverio/webdrivercss https://github.com/webdriverio/webdrivercss https://github.com/webdriverio/webdrivercss-adminpanel https://github.com/webdriverio/webdrivercss-adminpanel
- rplnt 11y ago
  You are severely limiting them if they don't pay. So I guess hooking them (or at least letting them try it) and then upselling sounds better for both parties.
  - bliti 11y ago
    I have the hardest time upselling from free. But its pretty easy to go from $300 to $500.
- tlrobinson 11y ago
  Maybe, but many people (myself included) may be reluctant to use it if they can't even try it out without paying for it. Especially if the core product can be replicated with 3 lines of code using open source libraries.

nbevans 11y ago

Tip: Remove the "plans" page. This thing is too early stage to start talking about $ plans. Nobody even knows what the hell this is or what it is good for. Your business model should be as follows: 1. Make something new, useful and free. 2. Get people hooked on it, like a drug. 3. Grow a modest user base. 4. Introduce "value added" features, marked as preview or beta, for enterprise customers and integration into other web services. 5. Once features added in step 4 have matured, remove the beta clauses and slap a price tag on them.

liquidise 11y ago
i disagree with this assessment. What harm is there in having a pricing page? I'd support offering a free option, but why on earth would you ever disallow willing customers from paying you money? On the contrary, i am now tempted to build a "Buy" button on a basic marketing page before i ever write a line of code on future product ideas.
- Mandatum 11y ago
  "Plans" was the first thing I clicked on because I wanted to try it out, and then there's no free option so I clicked that little "X" by my tab. I actually have a use case for this and so do a few people I know, and I'd love to try it out.. But you've got to be kidding if you think I'm going to give out my payment data to try a website version of FollowThatPage.
blowski 11y ago
I agree that the landing page needs some work to make it clear what the product does, and why I would want to pay for it. However I disagree that you must start by offering the service for free. By charging from day one, you can limit your user base only to those people who are prepared to pay for it. You're less likely to get explosive growth this way, but it's easier to keep the service running as you're not subsidising freeloaders. One of the assumptions in an MVP is that people are prepared to pay for the service, and you can only establish that by requiring people to part with cash as early as possible.
bliti 11y ago

LaFolle 11y ago

Nice idea. But it would have been great if there could have been a trial/free version. I would like to test the service so that I understand what exactly it is about. Watching a video and reading text does not help much until you try it out by yourself.

joshcanhelp 11y ago

This looks fantastic. I was just looking into doing this "manually" with an automated screenshot process for a site I'm refactoring part by part.

bliti 11y ago

Your landing page might benefit from using wordpress as an example. I saw the service and immediately thought about how to integrate this into wp. Maybe even turn it into a wp plugin? Dunno. This does seem useful.

uptownfunk 11y ago

I want to know what this does, the youtube video does not explain clearly in layman's terms.

mastercoms 11y ago

Nice theme. http://startbootstrap.com/template-overviews/grayscale/ http://startbootstrap.com/template-overviews/grayscale/

schappim 11y ago

If you want a DIY version of this, try ImageMagick's compare command: compare bag_frame1.gif bag_frame2.gif compare.gif Documentation: http://www.imagemagick.org/Usage/compare/ http://www.imagemagick.org/Usage/compare/

no_gravity 11y ago
Interesting. And to render the html you can use cutycapt. Which is in the Debian repos too. I just tried it, and this rendered a nice diff of two pages for me: cutycapt --url=http://www.xyz.com/1 --out=1.png cutycapt --url=http://www.xyz.com/2 --out=2.png compare 1.png 2.png diff.png
- easytiger 11y ago
  Now make it a webservice and charge $200/month!
  - prottmann 11y ago
    apt-get install the services and use them on my server ;-)
  - programmernews3 11y ago
    First part already done: http://archive.is/ http://archive.is/
  - _pmf_ 11y ago
    > Now make it a webservice and charge $200/month! I don't actually see the problem here. They provide a real service, so they can demand real money. There are multi billion dollar companies that do less for their users.
- danieltillett 11y ago
  Very interesting approach.
- dghf 11y ago
  This is why I love HN. Someone launches a web service with a flashy site, lots of marketing blurb and monthly plans from $10 to $200. And in the HN comments thread, two random people reimplement the functionality in three lines.

Permit 11y ago

I just want to say that I strongly disagree with the people here who are complaining about your pricing page. You need to validate that people will pay money for this product, and the quickest way to do so is to ask up front. I really don't buy this "Give it away for free, get them hooked and then staple on paid features" approach. PG said it best: "Better to make a few users love you than a lot ambivalent."[1] One of the best ways to find out if people love what you're building is to ask them to pay for it. [1]http://www.paulgraham.com/13sentences.html http://www.paulgraham.com/13sentences.html

olivierkaisin 11y ago

Great project! This is really a need for monitoring consistency of UX of any website. Although I just think you should simplify your pricing model. I don't understand why you are talking about "requests". Who cares about bandwidth today anyway? Just make people pay for the number of pages the frequency of the checks. Also, you should allow people to set up e-mail alerts when some parts of their website pages change (e.g. payment forms). I would definitely pay for such product. : )

kasparloog 11y ago

Monitoring is a good idea. However, pixel-based comparison services are quite pointless for web pages. Banners, dynamic content etc. simply drives spam. Try Browserbite with its feature-based comparison. There are other regression-oriented tools out there as well that use pixel-based methods as well.

cekanoni 11y ago

your ssl is signed by unknown vendor so i wouldn't trust it..

kennycox 11y ago
This error occurs while you are using a self-signed certificate, it is advisable to go through the trusted third party certificates. You can use SSL installation checker to diagnosis troubleshoots. https://www.ssl2buy.com/wiki/ssl-installation-checker/ https://www.ssl2buy.com/wiki/ssl-installation-checker/ https://www.sslshopper.com/ssl-checker.html https://www.sslshopper.com/ssl-checker.html https://ssltools.websecurity.symantec.com/checker/views/certCheck.jsp https://ssltools.websecurity.symantec.com/checker/views/cert...

zoidb 11y ago

Hey nice! I've actually done something similar myself as a weekend project - https://page-watch.com https://page-watch.com Also checkout the following projects, https://visualping.io/ https://visualping.io/ and https://dpxdt-test.appspot.com/ https://dpxdt-test.appspot.com/ If you are using something like phantomjs to generate website screenshots I wonder how you are dealing with dynamic content. A lot of pages have continuous animations that can screw with simple image diff comparisons. For this reason you may want the option to limit your compare to a subset of the page.

wldlyinaccurate 11y ago

This reminded me of Wraith, a screenshot comparison tool which The BBC developed for integration testing: https://github.com/bbc-news/wraith https://github.com/bbc-news/wraith

ivanhoe 11y ago

Looks like a useful service. Just a word of warning, your SSL certificate is not trusted by Chrome and possibly some other browsers. You have to install Intermediate certificates to make it work for everyone.

rb2k_ 11y ago
Works for me on Chrome 43.0.2357.81 (OS X) Green lock + "The identity of this website has been verified by COMODO RSA Domain Validation Secure Server CA but does not have public audit records." No warnings.
- hrbrtglm 11y ago
  Damn, I had the exact same problem in another thread with an unrelated domain name : https://news.ycombinator.com/item?id=9645297 https://news.ycombinator.com/item?id=9645297 Also a comodo certificate.
- ivanhoe 11y ago
  Now it works for me to, I guess they fixed it in a meantime.
  - wickedlogic 11y ago
    Yes, thank you for pointing that out. I had recently installed new certs, but failed to point it to the ssl-bundled cert and instead it was just the main cert without the intermediates. Things are happier now: https://www.ssllabs.com/ssltest/analyze.html?d=diff.io&hideResults=on https://www.ssllabs.com/ssltest/analyze.html?d=diff.io&hideR... "A+"

jwklemm 11y ago

Nice work! We have a similar feature built into Ghost Inspector: https://ghostinspector.com/#visual-regressions https://ghostinspector.com/#visual-regressions -- It includes a tolerance setting so you can basically say "I only want to hear about it if the screenshot changes by xx% or more." Effective visual regression testing at scale is tough though and is really more of a secondary feature for us. If anyone is interested in other solutions, ImageMagick has a nice "compare" tool built-in. There are also plenty of open source projects like Huxley, PhantomCSS, etc. I recently saw a demo of Applitools Eyes (https://applitools.com/ https://applitools.com/) and it's quite powerful (though it's a paid service). Lastly, I did a presentation that involved visual regression testing here: https://www.youtube.com/watch?v=mK0l__jmpTA https://www.youtube.com/watch?v=mK0l__jmpTA (starts around 12:25)

Addie_B 11y ago
Applitools Eyes has a free account option as well, which grants access to all automated visual testing capabilities as in the paid packages - for a single user.

marcelerz 11y ago

You are probably using Blink-Diff for the image comparison. Isn't it? https://github.com/yahoo/blink-diff https://github.com/yahoo/blink-diff For example: https://raw.githubusercontent.com/yahoo/blink-diff/master/examples/YDN_Sort_output.png https://raw.githubusercontent.com/yahoo/blink-diff/master/ex... You can change the highlight-color to green (it is a configuration option), just like it is on the page there: https://diffio.global.ssl.fastly.net/v1/diffs/2015-05-28/eee2f08b89fa67377fd18ccb990c7d45f1464f95/montage.png https://diffio.global.ssl.fastly.net/v1/diffs/2015-05-28/eee... And the "montage" is called a "composition" in Blink-Diff.

wickedlogic 11y ago
It isn't Blink-Diff, but I had looked at that project at one point. It has some interesting options. :)