6 ms·

Show HN: A Who is Hiring app with AI filters

129 points by bernawil 3y ago

afropack 3y ago
This is cool. You should add counts to the filters and the result list.
- simonw 3y ago
  Here's that data with counts on the filters, via Datasette Lite: https://lite.datasette.io/?install=datasette-pretty-json&json=https://github.com/bernawil/hn-who-is-hiring/blob/main/src/HN_POSTS_CATEGORIZED.json#/data/HN_POSTS_CATEGORIZED?_facet_array=categorized_tech&_facet_array=categorized_positions https://lite.datasette.io/?install=datasette-pretty-json&jso...
NickC25 3y ago
You should also have filters for tech-tangential jobs, such as product, operations, design, etc... Not everyone here is a developer!
- spondylosaurus 3y ago
  Seconding this, I'd love a filter for technical writers :)
- bernawil 3y ago
  will probably do on a future iteration! The current job categories where jus the ones could come up with and then used the openAi api to check for conformance (or not) of each post to it :)
simonw 3y ago
Here's the underlying repo: https://github.com/bernawil/hn-who-is-hiring https://github.com/bernawil/hn-who-is-hiring This JSON file has the annotated data: https://github.com/bernawil/hn-who-is-hiring/blob/main/src/HN_POSTS_CATEGORIZED.json https://github.com/bernawil/hn-who-is-hiring/blob/main/src/H... Since it's JSON on GitHub you can explore using Datasette Lite like this: https://lite.datasette.io/?install=datasette-pretty-json&json=https://github.com/bernawil/hn-who-is-hiring/blob/main/src/HN_POSTS_CATEGORIZED.json#/data/HN_POSTS_CATEGORIZED?_facet_array=categorized_tech&_facet_array=categorized_positions https://lite.datasette.io/?install=datasette-pretty-json&jso... Here's an example of a custom SQL query: https://lite.datasette.io/?install=datasette-pretty-json&json=https://github.com/bernawil/hn-who-is-hiring/blob/main/src/HN_POSTS_CATEGORIZED.json#/data?sql=select+id%2C+description%2C+metadata%2C+json_extract%28metadata%2C+%27%24.company_location%27%29%2C+categorized_tech%2C+categorized_positions+from+HN_POSTS_CATEGORIZED https://lite.datasette.io/?install=datasette-pretty-json&jso...
- bernawil 3y ago
  Thanks for this, I didn't know about datasette. Neat!
  - mrkstu 3y ago
    simonw is the best kind of spammer- he brings his creation into the discussion- but he does so in a way that enriches the value of what is being discussed. His tool [Datasette] is of course such that it often is immensely handy at dissecting data in useful ways so is often exactly on point for a discussion on HN...
    - bernawil 3y ago
      haha right it's like, how did I make it this far without knowing about datasette?
      - b112 3y ago
        You never owned a Vic20?
    - throwaway14356 3y ago
      But sir, now that mega corps have monopolized attention on the internet and charge enormous fees in my humbles you should really rethink what it is you are trying to say? Personally I cant find anything anymore, google is trolling me, people are afraid to link to the homepage of their subject matter. People foolish enough to make websites have zero visitors because they fear linking to their own shit. LOL I cant name anything you've made and you can't name anything of mine. If you didn't mention it I wouldn't have known the tool was that shameful self promotion - now forbidden on the internet? Ill be quiet now or else it will turn into an article rather than a low effort monologue and who would want to read that? Thanks
bugglebeetle 3y ago
There should be a filter for breaking California law by not including salary details for companies over 15 people: https://californiapayroll.com/blog/californias-new-pay-transparency-act-sb-1162-requirements/ https://californiapayroll.com/blog/californias-new-pay-trans...
- bernawil 3y ago
  hah not a bad idea at all, watch out for next months update!
- jdksmdbtbdnmsm 3y ago
  Better idea #1: a filter for posts w/o salary details regardless of useless "laws" Better idea #2: Unionize and strike because historically nothing else has ever resulted in enforced universal workers' rights.
minimaxir 3y ago
What's the AI usage here? From the raw JSON data, it seems you wrote a prompt to an LLM to extract structured data from the Who is Hiring comments, although I am not sure if that counts as an "AI filter" since the filtering criteria are explicitly defined beforehand.
- bernawil 3y ago
  right, I'm feeding each post to several queries to the openAi API. I guess I put "AI filters" so people knew this is actually curated by post content and not just a contains() filter so you get posts with the text "we don't do remote!" when you select the remote checkbox
  - superfrank 3y ago
    Not knocking the approach, but how do you do quality control on the posts? Are you just spot checking? How often have you found bad data? I've thought about doing something similar (using ChatGPT to structure and categorize unstructured data) for a different project in a completely different space and I'm worried about ChatGPT hallucinating things, especially when it comes to numbers.
    - jonnycoder 3y ago
      The quality control is a good question, and one that can probably be addressed using evaluation as taught by some of the deeplearning.ai short courses (1). I made an interactive resume ai bot on my personal website and there is an instance where I can ask it "tell me about your intel experience" and it added in C++ as one of the languages, but that is untrue. I had done C++ at a different company. 1. https://www.deeplearning.ai/short-courses/ https://www.deeplearning.ai/short-courses/
      - superfrank 3y ago
        Can you give more details? No offense, but I'm not going to sign up for a random site to watch a video of unknown quality and length.
PaulRobinson 3y ago
When providing a huge list of technologies, structure them somehow. Alphabetically ordered, for example - I shouldn't have to Ctrl-F to find my preferred programming language. Great idea, just needs some UX love.
- bernawil 3y ago
  you're absolutely right, will get it on a next iteration. Just for now, know you can filter by your tech on the technologies list. Most people are looking for either remote or a specific location, and after choosing one of those honestly there are not that many posts left to sort through.
  - araes 3y ago
    I think that's what is being noted. That there's a huge list of technologies, and it would be nice if they could be sorted by some criteria (alphabetical, old->recent tech, grouped (all these are Javascript based), categories (compiled, interpolated, data science, graphics/imagery, ...) Also, is there a way to put in prior "Who is Hiring?" dates? If people keep putting out the same listing again and again it would be nice to have a way to find. Totally in the Nice to Have category.
    - bernawil 3y ago
      Gosh now I get it! thanks for the clarification, looks like I didn't get the point. Totally right. The search bar does filter the filters list. So if looking for "rust": 1. expand the technologies list 2. type rust in the search bar 3. select rust option. But yes, should probably sort it though. As for old posts, I just overwrite the old data and modify the month label just out of laziness. I Will keep posts history in future iterations!
hughdbrown 3y ago
I have a similar-but-hacky command-line app that I put together to find just Rust positions: https://github.com/hughdbrown/who-is-hiring https://github.com/hughdbrown/who-is-hiring It's built to be pretty fast by not pulling data it does not need. Since it operates in multiple passes on stored data, it would be easy to modify/add a pass to get what you want. Feel free to use parts you like. A couple of things I think would help: - sorted attributes (too hard to go through a hundred computer technologies to find Rust) - multiple geographic entries for the same name (multiple entries for Germany, USA, UK, Europe) - ability to select a month - if you are showing a static pull of the data, the ability to refresh some month would be helpful
Retr0id 3y ago
It would be nice to have negative filters, e.g. "anything that doesn't mention AI" Edit: This is not snark, I simply do not wish to work in the AI sector, which seemed over-represented in the most recent Who's Hiring thread.
- digging 3y ago
  By the title, that's exactly what I was actually expecting. It's also what I'd prefer to work with.
- hoosieree 3y ago
  The word "filter" is an auto-antonym. It means both to remove (filter out) unwanted things as well as to keep (filter in) wanted things.
  - Retr0id 3y ago
    And yet, if I hadn't specified, it would've been ambiguous.
  - faitswulff 3y ago
    Tangential, but this is why I still love Ruby's Enumerable module; it has `#select` and `#reject`, as well as the `#filter` method that people from other languages are used to.
    - rzzzt 3y ago
      Can you think of any reasonably popular language where a "filter" operation drops the element if the filtering expression evaluates to true?
      - faitswulff 3y ago
        No, I can’t. Why do you ask? My point was that it's much nicer to have method names that do what they say instead of the programmatic version of the word "biweekly."
godelski 3y ago
It would be really nice to add salaries. I remember there was a past effort where people would actually downvote listings that didn't show salaries. Now it's required in California (though plenty don't, including Mistral....). Honestly, it just saves everyone time and helps keep the market open. Maybe we should also restart the pressure? The template should be updated to reflect this (I swore it used to exist). And then allow us to filter by salaries.
- bugglebeetle 3y ago
  I commented the same elsewhere. It’s a 10K fine per listing and makes you liable for civil action in certain circumstances. I’m surprised HN even allows posts without salaries, as you’re in violation if anyone from CA could potentially fill the role (e.g. a US-based remote position).
  - jdksmdbtbdnmsm 3y ago
    >It’s a 10K fine per listing Is it? Has even a single party been charged? And if so, what was the enforcement process? Without unions that strike (key word: strike) workers have no power. Edit: the $10k is a maximum fine, not a minimum fine. In other words, it's a complete joke.
    - fancy_pantser 3y ago
      > Companies hit by wave of suits over job ads lacking pay info > A batch of proposed class actions filed in Washington state against major companies such as adidas AG, Home Depot Inc., and Marriott International Inc., will help employers gauge the litigation risk they face under a recent nationwide wave of pay transparency laws requiring salary ranges in job ads. The wheels are starting to turn. I think there are more class actions happening than individual cases but IANAL. https://news.bloomberglaw.com/daily-labor-report/adidas-home-depot-sued-in-test-cases-on-job-ad-pay-transparency https://news.bloomberglaw.com/daily-labor-report/adidas-home...
      - 3y ago
mingyeow 3y ago
Looks very good actually, what is the source of the jobs and ai engine used?
- bernawil 3y ago
  the who is hiring monthly post of this site, and the openAi API!
atraac 3y ago
Would be nice to get a link directly to a post. F.e. links are a bit malformed due to formatting and wanting to see an original post, I had to ctrl+f over the whole thread with parts of the sentence.
jdthedisciple 3y ago
Why does it seem so uncommon/unpopular to differentiate positions between entry level and senior in these WIH threads? I guess because people assume there are only seniors on Hackernews? I could imagine that being the reason and it would be pretty silly...
vsnf 3y ago
> USA (33) > USA (4) > US (3) > US (1) > UNITED_STATES (1) > Europe (5) > Europe (18) > EU (1) > Canada (8) > Canada (1) The filters need a bit of consolidation.
gajnadsgjoas 3y ago
Can't find any link to the comment, found a good posting but the link inside the comment is cut off

129 points by bernawil 3y ago

afropack 3y ago

This is cool. You should add counts to the filters and the result list.

simonw 3y ago
Here's that data with counts on the filters, via Datasette Lite: https://lite.datasette.io/?install=datasette-pretty-json&json=https://github.com/bernawil/hn-who-is-hiring/blob/main/src/HN_POSTS_CATEGORIZED.json#/data/HN_POSTS_CATEGORIZED?_facet_array=categorized_tech&_facet_array=categorized_positions https://lite.datasette.io/?install=datasette-pretty-json&jso...

NickC25 3y ago

You should also have filters for tech-tangential jobs, such as product, operations, design, etc... Not everyone here is a developer!

spondylosaurus 3y ago
Seconding this, I'd love a filter for technical writers :)
bernawil 3y ago
will probably do on a future iteration! The current job categories where jus the ones could come up with and then used the openAi api to check for conformance (or not) of each post to it :)

simonw 3y ago

Here's the underlying repo: https://github.com/bernawil/hn-who-is-hiring https://github.com/bernawil/hn-who-is-hiring This JSON file has the annotated data: https://github.com/bernawil/hn-who-is-hiring/blob/main/src/HN_POSTS_CATEGORIZED.json https://github.com/bernawil/hn-who-is-hiring/blob/main/src/H... Since it's JSON on GitHub you can explore using Datasette Lite like this: https://lite.datasette.io/?install=datasette-pretty-json&json=https://github.com/bernawil/hn-who-is-hiring/blob/main/src/HN_POSTS_CATEGORIZED.json#/data/HN_POSTS_CATEGORIZED?_facet_array=categorized_tech&_facet_array=categorized_positions https://lite.datasette.io/?install=datasette-pretty-json&jso... Here's an example of a custom SQL query: https://lite.datasette.io/?install=datasette-pretty-json&json=https://github.com/bernawil/hn-who-is-hiring/blob/main/src/HN_POSTS_CATEGORIZED.json#/data?sql=select+id%2C+description%2C+metadata%2C+json_extract%28metadata%2C+%27%24.company_location%27%29%2C+categorized_tech%2C+categorized_positions+from+HN_POSTS_CATEGORIZED https://lite.datasette.io/?install=datasette-pretty-json&jso...

bernawil 3y ago
Thanks for this, I didn't know about datasette. Neat!
- mrkstu 3y ago
  simonw is the best kind of spammer- he brings his creation into the discussion- but he does so in a way that enriches the value of what is being discussed. His tool [Datasette] is of course such that it often is immensely handy at dissecting data in useful ways so is often exactly on point for a discussion on HN...
  - bernawil 3y ago
    haha right it's like, how did I make it this far without knowing about datasette?
    - b112 3y ago
      You never owned a Vic20?
  - throwaway14356 3y ago
    But sir, now that mega corps have monopolized attention on the internet and charge enormous fees in my humbles you should really rethink what it is you are trying to say? Personally I cant find anything anymore, google is trolling me, people are afraid to link to the homepage of their subject matter. People foolish enough to make websites have zero visitors because they fear linking to their own shit. LOL I cant name anything you've made and you can't name anything of mine. If you didn't mention it I wouldn't have known the tool was that shameful self promotion - now forbidden on the internet? Ill be quiet now or else it will turn into an article rather than a low effort monologue and who would want to read that? Thanks

bugglebeetle 3y ago

There should be a filter for breaking California law by not including salary details for companies over 15 people: https://californiapayroll.com/blog/californias-new-pay-transparency-act-sb-1162-requirements/ https://californiapayroll.com/blog/californias-new-pay-trans...

bernawil 3y ago
hah not a bad idea at all, watch out for next months update!
jdksmdbtbdnmsm 3y ago
Better idea #1: a filter for posts w/o salary details regardless of useless "laws" Better idea #2: Unionize and strike because historically nothing else has ever resulted in enforced universal workers' rights.

minimaxir 3y ago

What's the AI usage here? From the raw JSON data, it seems you wrote a prompt to an LLM to extract structured data from the Who is Hiring comments, although I am not sure if that counts as an "AI filter" since the filtering criteria are explicitly defined beforehand.

bernawil 3y ago
right, I'm feeding each post to several queries to the openAi API. I guess I put "AI filters" so people knew this is actually curated by post content and not just a contains() filter so you get posts with the text "we don't do remote!" when you select the remote checkbox
- superfrank 3y ago
  Not knocking the approach, but how do you do quality control on the posts? Are you just spot checking? How often have you found bad data? I've thought about doing something similar (using ChatGPT to structure and categorize unstructured data) for a different project in a completely different space and I'm worried about ChatGPT hallucinating things, especially when it comes to numbers.
  - jonnycoder 3y ago
    The quality control is a good question, and one that can probably be addressed using evaluation as taught by some of the deeplearning.ai short courses (1). I made an interactive resume ai bot on my personal website and there is an instance where I can ask it "tell me about your intel experience" and it added in C++ as one of the languages, but that is untrue. I had done C++ at a different company. 1. https://www.deeplearning.ai/short-courses/ https://www.deeplearning.ai/short-courses/
    - superfrank 3y ago
      Can you give more details? No offense, but I'm not going to sign up for a random site to watch a video of unknown quality and length.

PaulRobinson 3y ago

When providing a huge list of technologies, structure them somehow. Alphabetically ordered, for example - I shouldn't have to Ctrl-F to find my preferred programming language. Great idea, just needs some UX love.

bernawil 3y ago
you're absolutely right, will get it on a next iteration. Just for now, know you can filter by your tech on the technologies list. Most people are looking for either remote or a specific location, and after choosing one of those honestly there are not that many posts left to sort through.
- araes 3y ago
  I think that's what is being noted. That there's a huge list of technologies, and it would be nice if they could be sorted by some criteria (alphabetical, old->recent tech, grouped (all these are Javascript based), categories (compiled, interpolated, data science, graphics/imagery, ...) Also, is there a way to put in prior "Who is Hiring?" dates? If people keep putting out the same listing again and again it would be nice to have a way to find. Totally in the Nice to Have category.
  - bernawil 3y ago
    Gosh now I get it! thanks for the clarification, looks like I didn't get the point. Totally right. The search bar does filter the filters list. So if looking for "rust": 1. expand the technologies list 2. type rust in the search bar 3. select rust option. But yes, should probably sort it though. As for old posts, I just overwrite the old data and modify the month label just out of laziness. I Will keep posts history in future iterations!

hughdbrown 3y ago

I have a similar-but-hacky command-line app that I put together to find just Rust positions: https://github.com/hughdbrown/who-is-hiring https://github.com/hughdbrown/who-is-hiring It's built to be pretty fast by not pulling data it does not need. Since it operates in multiple passes on stored data, it would be easy to modify/add a pass to get what you want. Feel free to use parts you like. A couple of things I think would help: - sorted attributes (too hard to go through a hundred computer technologies to find Rust) - multiple geographic entries for the same name (multiple entries for Germany, USA, UK, Europe) - ability to select a month - if you are showing a static pull of the data, the ability to refresh some month would be helpful

Retr0id 3y ago

It would be nice to have negative filters, e.g. "anything that doesn't mention AI" Edit: This is not snark, I simply do not wish to work in the AI sector, which seemed over-represented in the most recent Who's Hiring thread.

digging 3y ago
By the title, that's exactly what I was actually expecting. It's also what I'd prefer to work with.
hoosieree 3y ago
The word "filter" is an auto-antonym. It means both to remove (filter out) unwanted things as well as to keep (filter in) wanted things.
- Retr0id 3y ago
  And yet, if I hadn't specified, it would've been ambiguous.
- faitswulff 3y ago
  Tangential, but this is why I still love Ruby's Enumerable module; it has `#select` and `#reject`, as well as the `#filter` method that people from other languages are used to.
  - rzzzt 3y ago
    Can you think of any reasonably popular language where a "filter" operation drops the element if the filtering expression evaluates to true?
    - faitswulff 3y ago
      No, I can’t. Why do you ask? My point was that it's much nicer to have method names that do what they say instead of the programmatic version of the word "biweekly."

godelski 3y ago

It would be really nice to add salaries. I remember there was a past effort where people would actually downvote listings that didn't show salaries. Now it's required in California (though plenty don't, including Mistral....). Honestly, it just saves everyone time and helps keep the market open. Maybe we should also restart the pressure? The template should be updated to reflect this (I swore it used to exist). And then allow us to filter by salaries.

bugglebeetle 3y ago
I commented the same elsewhere. It’s a 10K fine per listing and makes you liable for civil action in certain circumstances. I’m surprised HN even allows posts without salaries, as you’re in violation if anyone from CA could potentially fill the role (e.g. a US-based remote position).
- jdksmdbtbdnmsm 3y ago
  >It’s a 10K fine per listing Is it? Has even a single party been charged? And if so, what was the enforcement process? Without unions that strike (key word: strike) workers have no power. Edit: the $10k is a maximum fine, not a minimum fine. In other words, it's a complete joke.
  - fancy_pantser 3y ago
    > Companies hit by wave of suits over job ads lacking pay info > A batch of proposed class actions filed in Washington state against major companies such as adidas AG, Home Depot Inc., and Marriott International Inc., will help employers gauge the litigation risk they face under a recent nationwide wave of pay transparency laws requiring salary ranges in job ads. The wheels are starting to turn. I think there are more class actions happening than individual cases but IANAL. https://news.bloomberglaw.com/daily-labor-report/adidas-home-depot-sued-in-test-cases-on-job-ad-pay-transparency https://news.bloomberglaw.com/daily-labor-report/adidas-home...
    - 3y ago

mingyeow 3y ago

Looks very good actually, what is the source of the jobs and ai engine used?

bernawil 3y ago
the who is hiring monthly post of this site, and the openAi API!

atraac 3y ago

Would be nice to get a link directly to a post. F.e. links are a bit malformed due to formatting and wanting to see an original post, I had to ctrl+f over the whole thread with parts of the sentence.

jdthedisciple 3y ago

Why does it seem so uncommon/unpopular to differentiate positions between entry level and senior in these WIH threads? I guess because people assume there are only seniors on Hackernews? I could imagine that being the reason and it would be pretty silly...

vsnf 3y ago

> USA (33) > USA (4) > US (3) > US (1) > UNITED_STATES (1) > Europe (5) > Europe (18) > EU (1) > Canada (8) > Canada (1) The filters need a bit of consolidation.

gajnadsgjoas 3y ago

Can't find any link to the comment, found a good posting but the link inside the comment is cut off