4 ms·

the problem with Phantom.js is that compared to an actual browser it still falls short. I've given up on Phantom.js all together because it is actually not a tr

the problem with Phantom.js is that compared to an actual browser it still falls short. I've given up on Phantom.js all together because it is actually not a true replacement for Selenium running on an actual browser. I found it far more reliable to have Firefox running on Xvfb for when I'm testing without worrying about Phantom.js behaving wildly.

smellf 12y ago
It's definitely for a different use case. Phantom/Nightmare is for automating web page interaction, whereas Selenium is for cross-browser testing.
- curiously 12y ago
  You can use Selenium for automation and web page interaction. There is just an overhead of running Xvfb but it's trivial to setup. What I mean is that PhantomJS engine still falls short of a real browser. Advantage of running a real browser with Xvfb is that you won't ever have to worry about if the page is going to render properly or have it behave the way you expect it to. The overhead is not of a big deal, in fact it's a tradeoff for rendering accuracy. Don't get me wrong headless browser like PhantomJS is headed in the right direction but if compatibility with website is crucial for your success, it is not enough and I just don't have a lot of confidence that it will replace a real browser with Xvfb. Basically the Javascript libraries are reinventing a lot of the wheels that have already been done in other languages before the explosion of Node.js.
EGreg 12y ago
How about slimer.js for FF?
- curiously 12y ago
  firefoxdriver already exists in many other languages, I don't know what is the point of writing it all over again for javascript. The callback stuff gets messy. I still much prefer Python or even Java to write my tests.
baudehlo 12y ago
How is it not a real browser? It's basically a full WebKit based browser.
- ivank 12y ago
  PhantomJS 1.9.7 is based on an ancient version of WebKit that lacks ES5 functions like Function.prototype.bind. It also crashes pretty frequently.
  - baudehlo 12y ago
    I can absolutely concur that it crashes frequently. I have high hopes v2 fixes this. Most of the crashes are due to WebKit bugs that have since been resolved in QT5.
    - curiously 12y ago
      I waited and waited but just gave up. I really hope they bring it up to speed.
tommyd 12y ago
I've reached the same conclusion for our suite of UI tests (which are based on Cucumber + Capybara + SitePrism) - Phantom has random issues that don't occur with any other browser/driver (e.g. elements not being found even if you add a long wait - our app is rich client side Angular so I suspect some JS issue is happening) and it's too hard to debug what is going on compared to Firefox + Xvfb (where you can just redirect the display to a real X display and see what is going on).
- curiously 12y ago
  Yes, with selenium you can actually watch the entire test happen and see where things go wrong. Very easy to debug and understand but PhantomJS makes it really tough to find out why something failed because its headless. Sometimes I would try to take screenshots to figure out what was going on and even that would fail at times. Maybe you'd be able to run more instances with PhantomJS compared to running Xvfb and a real browser because of lower memory consumption. I don't know if this is true.

smellf 12y ago

It's definitely for a different use case. Phantom/Nightmare is for automating web page interaction, whereas Selenium is for cross-browser testing.

curiously 12y ago
You can use Selenium for automation and web page interaction. There is just an overhead of running Xvfb but it's trivial to setup. What I mean is that PhantomJS engine still falls short of a real browser. Advantage of running a real browser with Xvfb is that you won't ever have to worry about if the page is going to render properly or have it behave the way you expect it to. The overhead is not of a big deal, in fact it's a tradeoff for rendering accuracy. Don't get me wrong headless browser like PhantomJS is headed in the right direction but if compatibility with website is crucial for your success, it is not enough and I just don't have a lot of confidence that it will replace a real browser with Xvfb. Basically the Javascript libraries are reinventing a lot of the wheels that have already been done in other languages before the explosion of Node.js.

EGreg 12y ago

How about slimer.js for FF?

curiously 12y ago
firefoxdriver already exists in many other languages, I don't know what is the point of writing it all over again for javascript. The callback stuff gets messy. I still much prefer Python or even Java to write my tests.

baudehlo 12y ago

How is it not a real browser? It's basically a full WebKit based browser.

ivank 12y ago
PhantomJS 1.9.7 is based on an ancient version of WebKit that lacks ES5 functions like Function.prototype.bind. It also crashes pretty frequently.
- baudehlo 12y ago
  I can absolutely concur that it crashes frequently. I have high hopes v2 fixes this. Most of the crashes are due to WebKit bugs that have since been resolved in QT5.
  - curiously 12y ago
    I waited and waited but just gave up. I really hope they bring it up to speed.

tommyd 12y ago

I've reached the same conclusion for our suite of UI tests (which are based on Cucumber + Capybara + SitePrism) - Phantom has random issues that don't occur with any other browser/driver (e.g. elements not being found even if you add a long wait - our app is rich client side Angular so I suspect some JS issue is happening) and it's too hard to debug what is going on compared to Firefox + Xvfb (where you can just redirect the display to a real X display and see what is going on).

curiously 12y ago
Yes, with selenium you can actually watch the entire test happen and see where things go wrong. Very easy to debug and understand but PhantomJS makes it really tough to find out why something failed because its headless. Sometimes I would try to take screenshots to figure out what was going on and even that would fail at times. Maybe you'd be able to run more instances with PhantomJS compared to running Xvfb and a real browser because of lower memory consumption. I don't know if this is true.