3 ms·
I have always wanted a program/script to check for changes in the NIH era commons site, specifically a page that indicates for a particular grant application, w
by plg 10y ago
I have always wanted a program/script to check for changes in the NIH era commons site, specifically a page that indicates for a particular grant application, what its score is. Problem is, to get to this magic page (which is, I think, dynamically created), you have to:
(1) log into the main era commons site with a username and pw
(2) then a new page opens up, on which you have to click on a particular link to get a list of your grants
(3) then click on the grant of interest
(4) then click on the competition date
(5) then the magic page opens up. It's this page I want to check for changes
I've done the requisite google searching and found lots of generics suggestions about cookies, etc, but I don't have the skills, obviously.
Is there a tool whereby I can click "record" and go through my various steps, and then the tool will keep track of the programmatic aspects of this surf-and-click dance, such that I can execute that script later (e.g. as a cron job)?
Thanks!
- irishcoffee 10y agoAn incredibly crude solution: http://sikulix.com/ http://sikulix.com/
- kej 10y agoI like the idea of something that records your actions and creates a script, but for your particular case this would be super easy to throw together with Python and the Requests library. You just create a Session object and tell it to perform the series of posts and gets that take you to the desired page, and then search the text of that page for whatever info you want.
- abhirag 10y agoAnother option could be to use mechanize (https://pypi.python.org/pypi/mechanize https://pypi.python.org/pypi/mechanize). Originally written in Perl, then ported to Python as well as Ruby.
- plg 10y agoGot it done with Python/Mechanize! Thanks for the suggestion!
- ice109 10y agoare you a researcher? PM and I'll do it for you. I did this ever day all day for the last 6 months so it won't take me more than an hour.
- plg 10y agothanks for this! I think though I'll try Python/Mechanize first. Also I'll try not being so neurotic about it and just wait like a human being for the email to be sent to me :)
- ice109 10y agoare you a researcher? PM and I'll do it for you. I did this ever day all day for the last 6 months so it won't take me more than an hour.
- deleted 10y ago[deleted]