8 ms·
Show HN: A Machine Learning Model for Salary Estimation
- calcsam 11y ago(1) this is awesome (2) http://glassbowl.info/ http://glassbowl.info/ is not leading for me (3) If you did scrape LinkedIn, you probably violated their ToS, so you may not want to admit this publicly in case this goes anywhere :)
- garrettheaver 11y agoCame back here to suggest your point 3. I sense a cease and desist is on the way...
- elaineo 11y agoCan I plead ignorance? :X
- garrettheaver 11y agoYou can always try I guess. Whether or not it works is another story :-p I started using tor for this type of thing. It's very handy in that it generates a new circuit every 10 minutes or so and hence you get a new exit node.
- elaineo 11y agoRe #2 - sorry! I had to restart the server. Try again now :)
- personjerry 11y agoFor me, it's now saying: "Oops! LinkedIn error. Please check your url."
- elaineo 11y agoUsually it goes away if you wait a little bit. I think they do some rate-limiting. Also they'll probably block requests from my server soon ;)
- prairiedogg 11y agoCould you get away with allowing folks to manually paste in the HTML of their own linkedin page, or perhaps upload a PDF version of their linked in page? Are you scraping the page for input params? Would it be possible to just expose an API (or web forms that point there) to let folks put in those parameters manually?
- elaineo 11y agooy, that would actually have been a much better way of doing it. I was trying to be clever and keep the UI simple, and totally didn't expect to hit rate limits so quickly.
- wisevehicle 11y agoI want to +1 this idea :)
- elaineo 11y agoOkay, I added a field to manually paste in the HTML! Thanks for the suggestion :)
- j2kun 11y agoIt would be interesting to see if the model produces biased results by race, gender, age, etc.
- hazelnut 11y agoIt's down for me
- mitcho309 11y agoGetting an internal server error
- _lce0 11y agocan you reversely try to guess how much other people earn?
- ctomaybe 11y agoIt's dead, Jim.
- stewhuk 11y agoI did something similar for my side project at: http://skill.report http://skill.report To avoid the LinkedIn issue, I crawled job ads (about 10M uk ads over two years), and applied analysis to them. I love your approach - I'm going to have a play this weekend :) LinkedIn has so many amazing datasets locked up in the platform. I was keen to see whether I could reproduce the skill graph that LinkedIn has, and apply it to other problems, such as valuing and comparing skill sets and knowledge from job titles alone.
- awfml 11y agoI like this one, it lets you play hundreds of simulated games per second, and even over thousands of years you still don't win. http://powerball-simulator.com http://powerball-simulator.com
- iraphael 11y agoMy linkedin URL has a prefix "br." because I signed up when I still lived in Brazil. That makes Glassbowl crash and alert me the URL is invalid...
- fasouto 11y agoMine has an "es." but if I remove the prefix it still works.
- jedberg 11y agoOne way to train your model would be to allow people to tell you their current salary in exchange for the analysis. Then you get more accurate salary data. Of course people could lie, but that's probably no better than the glass door data anyway.
- adamb0mb1 11y agoI'd love to talk more details with you sometime! We(PayScale) don't scrape linkedin, but we have been collecting, analyzing and predicting pay for over a decade.
- knicholes 11y agoOops! LinkedIn error. Please check your url. But I'm pretty sure my URL is correct. It's the gray little URL underneath my profile picture? Similar to the example shown in the text input on your page?
- totalrobe 11y agoSame
- _lce0 11y agoI've noted that app is expecting to start with "www." in the url.. which is not the case for my url. But if you change that portion, LinkedIn will still find my profile, and the page seems to be working; as always YMMV
- elaineo 11y agoSorry, it was LinkedIn's rate limiting :( try again!
- samstave 11y agoit fails for me :-(
- atomical 11y agoCould LinkedIn sue this person?
- Hydraulix989 11y agoThe demo seems to be down, but if it works the way I think it does, it is a clever honeypot for scraping warm lead LinkedIn profiles.