18 ms·

Show HN: Phantom JS interactive editor

120 points by jakozaur 12y ago

jakozaur 12y ago
Author here: oh no Google banned traffic from my AWS instance and google.com shows blank :-(.
- timdorr 12y ago
  Yeah, they'll do that :) Can we download this and run it locally? Is the code open sourced somewhere?
  - jakozaur 12y ago
    Not yet open-sourced or downloaded, but consider doing that. Right now the setup would be too hard to make it smooth.
    - zenlikethat 12y ago
      Put it in a Docker container! :)
- nkozyra 12y ago
  Appears to show for me.
- marijn 12y ago
  Hi. Small CodeMirror-related nitpick. If you set your editor to be the height of the panel it is in, and let it take care of its own scrollbar, it'll look better (the gutter will span the whole panel) and be more efficient in case of big documents (it'll render only the visible part).
- freshyill 12y ago
  At first I was like "I don't get it". Then I opened it in Chrome and now I get it. Just an FYI… the editor is collapsing in Safari.
angryrancor 12y ago
This is really cool. Just getting deep in to phantomjs myself, I'm sure this will help a bit :)
kmike84 12y ago
A shameless plug: https://github.com/scrapinghub/splash https://github.com/scrapinghub/splash is an open source tool similar to Phantom JS, and it has an interactive web editor built-in. The concurrency model in Splash scripts is not the same as in Phantom JS scripts: Splash allows to use loops/ifs/functions in async code without recursion tricks or things like "casper.thenOpen". See http://splash.readthedocs.org/en/latest/scripting-tutorial.html http://splash.readthedocs.org/en/latest/scripting-tutorial.h... for more info. We haven't done a public announcement, and Splash is not yet feature complete, but it is used in production for more than a year, there are hundreds of unit tests and it is in active (though a bit sporadic) development - so give it a try if you want something different :) Installation is just "docker pull scrapinghub/splash" - see http://splash.readthedocs.org/en/latest/install.html http://splash.readthedocs.org/en/latest/install.html.
- jakozaur 12y ago
  Cool. Haven't heard about it. Will have a look. PhantomJS development seems stagnate a bit. Try to contribute to it myself.
  - kmike84 12y ago
    I was under impression PhantomJS 2.x is rather active, but of course Splash contributions are welcome, that'd be great! Splash Web UI is quite similar to PhantomJS Pad UI - by default it shows a CodeMirror text editor (with a simple autocompletion) and a screenshot of the webpage. Also, by default it shows information about requests/responses (like a "Network" tab in Firefox/Chrome development console) and a textarea with rendered html. PhantomJS Pad UI ideas are good; tabs for images, an editor window which is open by default and a field with console logs make a lot of sense.
- tonyrice 12y ago
  There's also Automately (http://automate.ly/ http://automate.ly/) it's a pretty interesting platform and it also allows for use of browser automation but differently than PhantomJS it's a little confusing to get setup at first but once you're there it's easy from there on.
deanclatworthy 12y ago
I tried with typekit with no success: http://phantomjs-pad.ozaur.com/DrDvxMADoP2TdF652 http://phantomjs-pad.ozaur.com/DrDvxMADoP2TdF652 I was curious with this example, as I have had zero luck getting webfonts to render using PhantomJS so far. Have you had any success rendering webfonts?
- d4n3 12y ago
  The thing with phantomjs and webfonts is that it has an old webkit version which has some bugs with webfont support. Theres a branch from a phantomjs contributor on github that has bugfixes for webfonts support backported: https://github.com/Vitallium/phantomjs/tree/fix-WOFF-file-support https://github.com/Vitallium/phantomjs/tree/fix-WOFF-file-su... I use phantomjs for html to image rendering and I managed to get webfont support working with a custom build from that branch (working on Heroku)
  - warfangle 12y ago
    PhantomJS is falling further and further behind. It also doesn't, for example, support scripts triggered DOM events. The more time goes by, the more I'm considering removing it from our testing suite.
- jakozaur 12y ago
  Looks like the site is overloaded, so a lot of times it end up in timeout problems.
- webo 12y ago
  You need to exit at the end: phantom.exit()
  - firloop 12y ago
    Hm, that's what I added but I guess the link did not update like other pastebin sites. Oh well...
- firloop 12y ago
  Working example: http://phantomjs-pad.ozaur.com/DrDvxMADoP2TdF652 http://phantomjs-pad.ozaur.com/DrDvxMADoP2TdF652
mmccaff 12y ago
Neat. Did you have a real use-case for this, or just fun? I'm curious, what dependencies does it have that would make setup difficult? In the Yelp example, I was confused for a second by the variable name of 'pizza' when doing a Yelp search on seafood. Renaming that would make the example more clear. :) What does the browser-view show? I didn't try, but if the code traverses multiple locations, is it a screenshot of the last step?
- jakozaur 12y ago
  Right now it's for fun, but the real-use case is to make more productive. E.g. be able to write test way faster. Browser-view shows at the end and whenever you write page.render(). Making setup smooth is solve able problem, I just haven't spent time on that. It uses Meteor and PhantomJS.
AustinScript 12y ago
I am a big fan of CasperJS which sits atop PhantomJS. This tool certain looks like a fun way to get introduced to Phantom.
davidy123 12y ago
Since the formal semantic failed to become massively used, everyone is trying to make sense of the Web using scraping. It's something most viable sites should want to get behind, from ecommerce where you can find the best price faster to education. But I suspect if a very good shared scraping system were created we'd see countermeasures pretty fast, from timid content producers, as well as the Googles who want to have exclusive access to this level of information. Yet it's a natural layer for a shared commons, and what can be built on that, when finally get past the mess of arbitrary representations, is very exciting. Anyone know of any standard to create interoperable scrapers? In Javascript, of course. Here's one submission: https://github.com/zotero/translators https://github.com/zotero/translators
- biomimic 12y ago
  You are spot on with this. Data acquisition is the foundation of data science which in turn is the foundation of [fill in the blank]
daemonk 12y ago
phantomjs is great for rastering svgs (d3.js visualizations)
- landr0id 12y ago
  That's exactly what I've used it for. My only complaint is that the documentation is not as nice as it could be.
tomvangoethem 12y ago
Cool, comes in quite handy! You may want to up your security-game though. Check the ~/FIXME file on your sever for more info :-)
- jakozaur 12y ago
  Yeah, know it, but didn't spent that much time on it and it suddenly attract so much traction. Will definitely do something serious about it.
  - jakozaur 12y ago
    Fixed. More security related improvements are coming.
deleted 12y ago
[deleted]
pkrein 12y ago
would love to write the script using http://www.nightmarejs.org http://www.nightmarejs.org instead ;)

120 points by jakozaur 12y ago

jakozaur 12y ago

Author here: oh no Google banned traffic from my AWS instance and google.com shows blank :-(.

timdorr 12y ago
Yeah, they'll do that :) Can we download this and run it locally? Is the code open sourced somewhere?
- jakozaur 12y ago
  Not yet open-sourced or downloaded, but consider doing that. Right now the setup would be too hard to make it smooth.
  - zenlikethat 12y ago
    Put it in a Docker container! :)
nkozyra 12y ago
Appears to show for me.
marijn 12y ago
Hi. Small CodeMirror-related nitpick. If you set your editor to be the height of the panel it is in, and let it take care of its own scrollbar, it'll look better (the gutter will span the whole panel) and be more efficient in case of big documents (it'll render only the visible part).
freshyill 12y ago
At first I was like "I don't get it". Then I opened it in Chrome and now I get it. Just an FYI… the editor is collapsing in Safari.

angryrancor 12y ago

This is really cool. Just getting deep in to phantomjs myself, I'm sure this will help a bit :)

kmike84 12y ago

A shameless plug: https://github.com/scrapinghub/splash https://github.com/scrapinghub/splash is an open source tool similar to Phantom JS, and it has an interactive web editor built-in. The concurrency model in Splash scripts is not the same as in Phantom JS scripts: Splash allows to use loops/ifs/functions in async code without recursion tricks or things like "casper.thenOpen". See http://splash.readthedocs.org/en/latest/scripting-tutorial.html http://splash.readthedocs.org/en/latest/scripting-tutorial.h... for more info. We haven't done a public announcement, and Splash is not yet feature complete, but it is used in production for more than a year, there are hundreds of unit tests and it is in active (though a bit sporadic) development - so give it a try if you want something different :) Installation is just "docker pull scrapinghub/splash" - see http://splash.readthedocs.org/en/latest/install.html http://splash.readthedocs.org/en/latest/install.html.

jakozaur 12y ago
Cool. Haven't heard about it. Will have a look. PhantomJS development seems stagnate a bit. Try to contribute to it myself.
- kmike84 12y ago
  I was under impression PhantomJS 2.x is rather active, but of course Splash contributions are welcome, that'd be great! Splash Web UI is quite similar to PhantomJS Pad UI - by default it shows a CodeMirror text editor (with a simple autocompletion) and a screenshot of the webpage. Also, by default it shows information about requests/responses (like a "Network" tab in Firefox/Chrome development console) and a textarea with rendered html. PhantomJS Pad UI ideas are good; tabs for images, an editor window which is open by default and a field with console logs make a lot of sense.
tonyrice 12y ago
There's also Automately (http://automate.ly/ http://automate.ly/) it's a pretty interesting platform and it also allows for use of browser automation but differently than PhantomJS it's a little confusing to get setup at first but once you're there it's easy from there on.

deanclatworthy 12y ago

I tried with typekit with no success: http://phantomjs-pad.ozaur.com/DrDvxMADoP2TdF652 http://phantomjs-pad.ozaur.com/DrDvxMADoP2TdF652 I was curious with this example, as I have had zero luck getting webfonts to render using PhantomJS so far. Have you had any success rendering webfonts?

d4n3 12y ago
The thing with phantomjs and webfonts is that it has an old webkit version which has some bugs with webfont support. Theres a branch from a phantomjs contributor on github that has bugfixes for webfonts support backported: https://github.com/Vitallium/phantomjs/tree/fix-WOFF-file-support https://github.com/Vitallium/phantomjs/tree/fix-WOFF-file-su... I use phantomjs for html to image rendering and I managed to get webfont support working with a custom build from that branch (working on Heroku)
- warfangle 12y ago
  PhantomJS is falling further and further behind. It also doesn't, for example, support scripts triggered DOM events. The more time goes by, the more I'm considering removing it from our testing suite.
jakozaur 12y ago
Looks like the site is overloaded, so a lot of times it end up in timeout problems.
webo 12y ago
You need to exit at the end: phantom.exit()
- firloop 12y ago
  Hm, that's what I added but I guess the link did not update like other pastebin sites. Oh well...
firloop 12y ago
Working example: http://phantomjs-pad.ozaur.com/DrDvxMADoP2TdF652 http://phantomjs-pad.ozaur.com/DrDvxMADoP2TdF652

mmccaff 12y ago

Neat. Did you have a real use-case for this, or just fun? I'm curious, what dependencies does it have that would make setup difficult? In the Yelp example, I was confused for a second by the variable name of 'pizza' when doing a Yelp search on seafood. Renaming that would make the example more clear. :) What does the browser-view show? I didn't try, but if the code traverses multiple locations, is it a screenshot of the last step?

jakozaur 12y ago
Right now it's for fun, but the real-use case is to make more productive. E.g. be able to write test way faster. Browser-view shows at the end and whenever you write page.render(). Making setup smooth is solve able problem, I just haven't spent time on that. It uses Meteor and PhantomJS.

AustinScript 12y ago

I am a big fan of CasperJS which sits atop PhantomJS. This tool certain looks like a fun way to get introduced to Phantom.

davidy123 12y ago

Since the formal semantic failed to become massively used, everyone is trying to make sense of the Web using scraping. It's something most viable sites should want to get behind, from ecommerce where you can find the best price faster to education. But I suspect if a very good shared scraping system were created we'd see countermeasures pretty fast, from timid content producers, as well as the Googles who want to have exclusive access to this level of information. Yet it's a natural layer for a shared commons, and what can be built on that, when finally get past the mess of arbitrary representations, is very exciting. Anyone know of any standard to create interoperable scrapers? In Javascript, of course. Here's one submission: https://github.com/zotero/translators https://github.com/zotero/translators

biomimic 12y ago
You are spot on with this. Data acquisition is the foundation of data science which in turn is the foundation of [fill in the blank]

daemonk 12y ago

phantomjs is great for rastering svgs (d3.js visualizations)

landr0id 12y ago
That's exactly what I've used it for. My only complaint is that the documentation is not as nice as it could be.

tomvangoethem 12y ago

Cool, comes in quite handy! You may want to up your security-game though. Check the ~/FIXME file on your sever for more info :-)

jakozaur 12y ago
Yeah, know it, but didn't spent that much time on it and it suddenly attract so much traction. Will definitely do something serious about it.
- jakozaur 12y ago
  Fixed. More security related improvements are coming.

deleted 12y ago

[deleted]

pkrein 12y ago

would love to write the script using http://www.nightmarejs.org http://www.nightmarejs.org instead ;)