12 ms·
Show HN: Analysis of 2018 Hacker News “Who Is Hiring” Job Posts
- openbasic 8y agoScala, huh?
- sabas123 8y agoAnd julia over c++???
- Kurtz79 8y agoThis honestly seems just a mistake. Skimming a few of the posts, there are maybe 15+ references to C++ for post, and 1 for Julia at most. The detection algorithm seems to be a bit iffy.
- inetknght 8y agoI'd be willing to bet that the regex parser doesn't like the pluses in C++
- Mikulas0 8y agoDefinitely. I might be blind, but I don't see C++ at all in the list of skills.
- Kurtz79 8y agoIt has changed compared to the initial version. Pretty sure that a string like "c\+\+" was there before. It seems that in the new version the old patterns have been commented out, the search method was changed and c++ removed altogheter: https://github.com/venkateshthallam/hn-whoishiring-analysis/blob/master/src/main.go https://github.com/venkateshthallam/hn-whoishiring-analysis/...
- eddd 8y agoThe fact that Rust and Python are quite high in the rankings, makes me hopeful about the future of programming again. The fact that remote work is not growing is a bit concerning, but if people from the valley want to overpay - it is none of my business. I hope eventually the market will correct it.
- ptd 8y agoIf you don’t mind sharing, why do you find the lack of growth in remote work concerning?
- apercu 8y agoNot OP but due to a lack of investment in public transportation in NA over the previous decades, many people spend a few hours a day commuting and in many cases this is done in private vehicles that pollute. For many in our profession there is no need to go to an office to work.
- ptd 8y agoInteresting, I haven't put much thought into the environmental impact of resistance to remote work. I would disagree with your second point however, I believe that you lose something when work is performed remote, regardless of your role. That benefit certainly isn't worth destroying the planet.
- apercu 8y agoI (re)started my consultancy in part so I could work from home. My commute had gone from 2 hours a day to 3 hours a day with the rise of traffic in the Toronto area. I meet with clients a couple times a month and I am on the phone, slack or webex/ringcentral/google hangouts etc.. a few hours a week. I am _sooooo_ much more productive now (with fewer distractions) that I work on average 500 hours a year less than I used to. That doesn't even begin to factor in the quality of life issues. I now drive 8000km per year versus over 20,000km per year. Some jobs, some roles, yes, you need to physically go somewhere. For me, for what I do? There is no benefit.
- simonebrunozzi 8y agoWow, 2nd most common skill is “excel”. I would have never guessed.
- marsrover 8y agoI think the chart is a little weird. Looks like Rust is second and Excel is actually third.
- OisinMoran 8y agoI was also quite confused. I think it's so high because "excel" is also a verb so a lot of job posts will say things like "We are looking for people who excel at what they do" and this was counted in the analysis. Changing the search to be case sensitive might help here.
- Kurtz79 8y agoMyabe the regex pattern is checking for the pattern but not as a separte word? "scalable solution" "excellent benefits"
- fasouto 8y agoI've found that "scala" and "excel" are ranked exceptionally high. I don't know but maybe the author it's looking simply for "excel" and "scala" in the text without taking in account surrounding space. So: - "scala" would match "scalable" and "scalability" - "excel" would match "excellent"
- ptd 8y agoScala as the top skill for 2018. I thought it would be high, but I’m surprised to see it at number 1...
- RhodesianHunter 8y agoKeep in mind, HN will mostly have startup jobs, so you're more likely to see languages like Scala represented here than in the job market at large, where I would expect Java, PHP, C++ etc. To dominate.
- lsiunsuex 8y agoSo the next report / analysis needs to be: Of everyone who asked for work or applied for work via these posts - how many secured a position? I've long given up on these posts trying to find freelance work or other permanent work. There's to much competition (here) to stand out from the flood of replies.
- jedberg 8y agoAnecdotally, the one time I posted a job, I had about 80 replies for a remote work junior engineering job where I specified the salary at $60k/yr in the post (this was in 2015). Of those 80, about 10 were promising and I probably would have made an offer to at least one or two if we hadn't shifted priorities and stopped the hiring process.
- Domark 8y agoI applied to matching jobs for three months about five years ago. Also did the same earlier this year for three months. My advice: skip this site and weworkremotely.com. Both are complete wastes of time based on the responses. If you’re in IT, your best bet is LinkedIn (for a referral) or Careerbuilder/Dice/Indeed. At least those will result in face to face interviews. Also find companies you like and apply directly on their website.
- dotdi 8y agoTake that all you people crying "Scala is dead" after Java 8+ comes out. /s Anyway, I've personally seen (and interviewed for) jobs that were advertising Scala (or plans to move to Scala, or that Scala experience is a plus) but actually meant there was one proof-of-concept project in Scala, 18 months ago, nobody ever touches it nowadays, the person who wrote it is long gone and by the way, our legacy JDK5 application is on fire, do you mind fixing that for the rest of your life?
- markhalonen 8y agoI suspect there's false positives from "scalability" and "scalable"...
- dotdi 8y agoI hope OP took that into account. Seems like pretty low hanging fruit when cleaning your data.
- quickthrower2 8y agoYep just split on whitespace
- imron 8y agoOr preferably, just split on word boundaries.
- mountaineer 8y agoScala's in good shape, but not quite so popular as the posted analysis presents, based on my findings of the same data: https://www.hntrends.com/2018/nov-react-still-top-containers-hot.html?compare=Scala&compare=Java&compare=Clojure&compare= https://www.hntrends.com/2018/nov-react-still-top-containers...
- Cthulhu_ 8y agoOh for sure; I'd take any job listing with something cool with a huge grain of salt; hardly any proper software development employer actually uses hip technology, but they will use hip terms to try and draw in developers. Hype-driven development is a problem, and hype-driven recruitment another one. Just be a good java developer. Maybe push for Kotlin slowly. Don't pursue Scala for the language's sake. See also: blockchain, IoT, etc. I've seen a job advert that basically bolted those onto the not well hidden "we're just looking for decent java developers" job description.
- sephlietz 8y agoI wonder if some of the "scala" hits are actually from the word "scalable"...
- typon 8y agoReally hope the OP did full word searches...otherwise he would be double counting "react" and "reactjs" as well
- corn_dog 8y agoAs much as I like Scala I find that hard to believe. I actually look for Scala in the job listings and I never felt it came up that much, compared to say Python or Javascript. A dumb search on the page for scala mostly turns up scalable ...
- twoquestions 8y agoDitto with Rust, most of the matches are part of the word "trust" when using ctrl+f in Chrome.
- vram22 8y agoThis is one reason why we need regexes (to name just one point) in browser and other search. I've said before in other places that Gmail could benefit from an SQL-like query language, for example. So could many other web and non-web apps. Google search too. One potential issue is excessive load on the servers if it gets used a lot, though.
- stygiansonic 8y agoAs others have mentioned, it’s possible the analysis was done through straight string matching; would know for sure once the code is open sourced.
- OisinMoran 8y agoI think there are some flaws in this analysis that can be confirmed when the code is released. I think the reason Scala seems surprisingly popular is that is was simple string matching so a post that said "We are looking for someone who can build scalable infrastructure" would count as one mention of "scala". Could be wrong though. Also see: https://news.ycombinator.com/reply?id=18725655&goto=item%3Fid%3D18725425%2318725655 https://news.ycombinator.com/reply?id=18725655&goto=item%3Fi...
- JshWright 8y agoYeah, similarly with "Excel". Are there really that many posts looking for people with deep spreadsheet experience? Seems more likely that it's a lot of false positives for "excellent" and "excel at".
- osoba 8y agoExcel is probably mislabelled, it should say HTML instead. Look at the list above
- mchaver 8y agoProbably the same for rust, looking at the October 2018 data set with https://kennytilton.github.io/whoishiring/ https://kennytilton.github.io/whoishiring/ rust gives 50 results, trust gives 21, \brust gives 26. Similar results for scala.
- alehul 8y agoWould there be an issue with just changing the string to be matched to "scala " or " rust"?
- jgtrosh 8y agoThe correct method is probably something like "\<$lang\>"
- meritt 8y agoThere are numerous ways to fix the glaring flaws in this analysis but a very simple approach would be using regex word boundaries, that way it'll work not only for the space but for commas, periods, end-of-line, etc.
- cgdub 8y agoScala seems suspiciously high compared to the data on Hacker News Hiring Trends: https://www.hntrends.com/ https://www.hntrends.com/
- vthallam 8y agoOP here. As some people were commenting, Scala and excel I guess have some false positives which reflect the unusual popularity. I did try to limit the search to exact word, but think that I overlooked that part. I will try fix the code and update the skills chart soon. Thanks for pointing it out. Edit: I fixed the false positives using a word boundary on regex "\\bword\\b". The data now seems little sane, will keep looking for any false positives. updated the data and regenerated the chart using new data.
- dabernathy89 8y agoWhile you're at it, you've got `vuejs` and `vue.js` listed separately :)
- bcherny 8y agoSame for “react” and “reactJS”.
- notyourloops 8y agoIt's really cool, but it's not going to be useful until you correct these errors. You might want to provide a note to readers in the meantime.
- vthallam 8y agoThanks. I added a note at the skills section about the false positives. Really should have paid attention, but i guess i was just excited to show it off, first :(. I am on it fixing the regex.
- aerotwelve 8y agoYou might want to consider performing your text analysis using something with a negex implementation; this would distinguish an instance of "remote" from "no remote" or "can't hire remote".
- deleted 8y ago[deleted]
- mountaineer 8y agoCool to see that other folks find this data interesting. I've been analyzing languages/frameworks/skills in the "Who is Hiring" posts for a few years (https://www.hntrends.com/ https://www.hntrends.com/) as well. I know it's just one part of the analysis, but the skills list appears to be be pretty far off of what I've been seeing. React gets over 200 a month by itself. Are you capturing all pages of the postings in each month's thread? Here's the data (counts) I have through November, broken down by month and term - https://www.hntrends.com/data/data-20181101.js https://www.hntrends.com/data/data-20181101.js
- edpichler 8y agoScala is the most popular skill. Is there something I don't know about Scala and the reason for its success? PS: I use Kotlin/Java for web development.
- PhasmaFelis 8y agoThe OP didn't correctly exclude false positives, so anything with "scalable" was registering for "Scala". It's fixed now.
- deleted 8y ago[deleted]
- pbhowmic 8y agoA bit surprised, pleasantly I must admit, to see Django keep pace with Rails. I have never been quite able to explain the startup world's love affair with Rails. Nothing wrong with it but the money question is why?
- joelbluminator 8y agoWe all wanna see the stack we work in succeed, so we can pay them bills. I'm gonna assume yours is Django. Mine is Rails. Plenty of work for all of us, no need to fight...
- maze-le 8y agoWhen you build web apps with a fairly narrow set of business rules and requirements rails is THE server side framework if you ask me. Development of simple features, that conform to some aspect of the framework can be impressively fast and reliable. The trouble always starts when requirements come piling in that were never talked about when the project started -- But this is a management issue and definitly not unique to ruby backend world.
- pbhowmic 8y agoI could argue the same about Django too. And Python is expressive, for my money, more expressive than Ruby. Ruby isn't bad but outside of functional languages, nothing beats Python how compactly ideas can be expressed.
- albertgoeswoof 8y agoEverything in rails is figured out already, you don’t have to pick which ORM or library to choose, and then find out half way down the line it’s not working as expected. It just works consistently, reliably. There’s no other reason to use another server side framework if you’re creating. CRUD app or API (unless you’re specifically comfy in something else).
- chpmrc 8y agoDoesn't answer parent's question. Django and Rails are both web frameworks.
- startupdiscuss 8y agoSo one simple way to solve the scala and excel issue is just to match for the space at the end or punctuation. So search for "scala " and "scala." instead of "scala". Same for "excel " and "excel." In fact you could just combine them all into word + end where end can be [" ", ".", "!", "\n" etc]
- rplnt 8y agoWhy the reverse scroll on the map?
- jimijazz 8y agoGood work. Besides fixing the false positives others have suggested, I would really like to see the total numer of posts per category in comparison to the total number of posts per month, i.e. a relative quantity to better explore wether there is a trend or not.
- LarryDarrell 8y agoIt's not looking good for me, a remote C# developer.
- Topgamer7 8y agoIf you look in the data .net is up there in numbers. Seems like the author is just doing straight regex matches not taking into account matches embedded in other words. Scala and rust both are exceptionally high, which I wouldn't expect.
- equasar 8y agoKind of the opposite, I do .NET Core development with Azure and I get at least from 1 to 2 job offers for REMOTE positions, for on site I get around the double.
- UltimateFloofy 8y agoEven though react and angular and vue are different frameworks, they are all modern JS libraries and I would lump that data together instead of keeping javascript so fragmented in the data set.
- dvaughan 8y agoYou could make the argument that Go is underrepresented as the match is only for golang and not go.
- deleted 8y ago[deleted]
- hu3 8y agoYep. A quick glance at December's Who is Hiring shows at least 2 jobs that aren't included by searching for Golang only: https://news.ycombinator.com/item?id=18662978 https://news.ycombinator.com/item?id=18662978 https://news.ycombinator.com/item?id=18590124 https://news.ycombinator.com/item?id=18590124 There's probably more in December alone that the regex missed.
- sidcool 8y agoI am so thrilled to see Rust at #2.
- n-gatedotcom 8y ago"tRust" me, most of us are
- howard941 8y agoI'm not sure where embedded or firmware would fit - developing for it's a skill but it's not a language or framework - but I'm curious what % of posts taps the skill. Going by memory it feels like roughly the same number of firmware positions are on there every month, and it's the same guys looking to fill them.
- zachguo 8y agoSloppy regex.
- vanrysss 8y agoCheck your map, it looks like you may be matching "Remote, or XXX" with the town of Remote, Or (Oregon).
- musikele 8y agoI live in Italy, and the chart shows 24 jobs openings for Napoli on Hacker News (64 in Rome...). I checked and I could not find a single text match for the words Italy, Rome, Napoli, Neaples, Naples, etc. Can you better express how you extracted the location data?
- chpmrc 8y agoThere is "Oregon" on the map (with 54 jobs) but even if it was referring to Oregon City that's way off the actual location.
- j_m_b 8y agoWhat no Clojure? I find that hard to believe.
- fulafel 8y agoThere are Clojure jobs posted every month, just an omission in the list of language regexps s/he searched for.
- GrumpyNl 8y agoPage doesn't load properly in chrome, like the rendering doesn't get finished, its all blurred. Loads fine in firefox.
- Southland 8y ago"Engineering Manager": 271, "Principal Software Engineer": 25, "Product Designer": 243, "Product Manager": 425, "Program Manager": 35, "QA": 353, "Senior Software Engineer": 818, "Software Engineer": 2975, "Staff Software Engineer": 10 I think this section should have been ordered by volume and cleaned up a little bit. 'Software Engineer' and 'Staff Software Engineer' are the same.
- cialowicz 8y ago"Staff Software Engineer" is typically the level above senior, and below principal. It's not commonly used at smaller companies. I agree about the ordering/ranking.
- josh_carterPDX 8y agoThis is great, but I would have liked to see more data around how many jobs are entry level, management, executive, etc. Thanks for this! Really cool.
- niutech 8y agoHave you seen this interactive map of HN job offers? https://whoishiring.io/search/0/0/2?source=hn https://whoishiring.io/search/0/0/2?source=hn
- deleted 8y ago[deleted]
- g45y45 8y agoTIL Jinja is a place, its off the coast of Africa.
- xipho 8y agoThe graph of top skills doesn't remotely correspond to the keyword values (actual data). For example rails/ruby is nowhere on the graph, when both score over 50.
- quickthrower2 8y agoOne of the maps shows a disposed event handler error on my iPhone 6 w. Safari
- i6mi6 8y agoI get that too
- aprdm 8y agoWeird to not see any devops role in your analysis.
- csdreamer7 8y agoAs a Rails Dev, why does Python score so high? Is Django or scraping that important? I know Python can be used in practically anything, but what do you keep seeing companies ask for that is a recurring need with Python?
- JesseAldridge 8y agoData science probably.
- Vinnl 8y agoProbably data science.
- syndacks 8y agoI think you hit the nail on the head, "Python can be used in practically anything." Our Data Scientist was able to pickup Django pretty quickly because of his experience with Python. And yea, actually someone at my company who was doing a bit of scraping on the side was able to pick up Django pretty easily as well. And last night, I chatted with my buddy who works at a big tech company and they use Python for database intrusion analysis. I don't know the Ruby ecosystem very well, but it seems the biggest application of Ruby is Rails. I'm curious to hear what other ways people use Ruby, so please share.
- csdreamer7 8y agoWell I use Ruby also for Chef. I use it to keep my servers perfectly perfect documented and tested in code. Since I am a solo founder I can stay away for weeks from the actual configuration. When I forget something or something breaks I look at my Chef recipe. It takes a little longer to get setup I also use Chef to setup my local environment which is wonderful. When you are learning how to really dig deep into your text editor and writing your own custom snippets (and commands, and keybindings) for react. You don't how important it is to have your local env, your dotfiles and atom config automated and can be synced up with a single command no matter what computer you are on. This, with private git repos and syncthing is a wonderful thing.
- 8y ago
- dmode 8y agoI find it hard to believe only 5% jobs are willing to sponsor visa considering most engineering teams are full of H1 visa holders. I think in this case it may be worth looking at the negative text as the default assumption is that the job sponsors visa transfer
- shaklee3 8y agoYou're comparing an HN post, which is typically smaller companies, to an overall statistics where large companies hire from body shops.
- tinyvm 8y agoWow , is .NET that popular in startups ? I always assumed it was used heavily in enterprise but never that much it's would somehow , it's also a popular stack in the Silicon Valley and NY. That said London and UK are well know to be MS Stack users. Can some Engineers from SF or NY or voice their opinions on this ?
- dimgl 8y agoThe answer is less deep than that. Hacker News is now regularly used as a recruiting tool. It may be that bigger companies are now also posting every month. .NET has come a long way though and I have a friend who is using it at his place, but from anecdotal evidence I see a lot more Python and JavaScript than anything.
- zephyrfalcon 8y agoWhen you look at the job postings in your browser, and do a simple string search for ".net" (so not a regex search), it doesn't show up all that often. E.g. if I look at this month's posts (4 pages), I see it 31 times. By contrast, Python shows up 251 times. I suspect the posts were searched with a regex like /.net/i without escaping the ".".
- magissima 8y agoI was curious what the number would be for Washington, DC, but it seems that the scraper couldn't distinguish between DC and Washington state.
- someguydave 8y agoSo does hacker news consist of “engineering managers” who know html or is that simply who everyone wants to hire?
- sytelus 8y agoSome highlights I saw - * SF is center of the tech world. Next biggest is just around half of its size. (not withstanding HN bias). * Approx 25% jobs allow visa sponsorship * HTML, Python, .Net overwhelmingly dominates everything else. * Reduced interest in databases, SQL, Obj-C, Java * No TensorFlow or PyTorch in demand * 90% jobs in development/software engineering, 10% in management and misc.
- patrickfreed 8y ago>SF is center of the tech world. Next biggest is not even half of its size. The article lists SF as having 2922 jobs and New York 1746 jobs
- dmode 8y agoIt probably understates the SF market, TBH. As some of the largest employers in SV who hire en masse rarely post jobs in HN. Such as Google, Facebook, Apple, Salesforce, Uber etc. Also, Google, FB, Uber, Airbnb and others have a boot camp system, where job postings do not correlate to hiring since one posting is used to hire an entire bootcamp class
- luord 8y agoThe job posts for react are five times those of javascript :thinking:
- mrath 8y agoStrange Sydney has so many less jobs in HN than Melbourne. Whereas I think in actual job count it is opposite. This could be because of more startup like companies in Mellbourne which publish jobs on HN
- nl 8y agoMelbourne, Florida has a surprising number of people advertising on HN. Also I don't think this is de-duplicating, and many of the Melbourne FL ads are repeated.
- jrochkind1 8y agoI think it's become indisputable that python seriously beats ruby in the job market. It is sad to me.
- dr_wolf 8y agoWhy is Montreal missing from the map?
- cauchyschwartz 8y agoSo if I'm understanding, raw text is fed into the geotext python library, which parses out geographical locations and adds to the count. I'm very skeptical that there are 18 new jobs on the island of Jinja, and that instead the analysis is picking up the Jinja templating language Edit: same thing for "cambridge" in the same location as Jinja Island. Obviously probably refers to Cambridge MA, where Tufts/Harvard/MIT are all nearby, so tons of companies will be there.
- kaustyap 8y agoIt shows 18 jobs in Rennes, France. But couldn't find a single job post. Wonder which word could produce false positive for "Rennes"?