3 ms·
I also had a similar problem at some point, where it wasn't possible to use readily available filtering options to find the very narrow niche of job postings th
by dserban 4y ago
I also had a similar problem at some point, where it wasn't possible to use readily available filtering options to find the very narrow niche of job postings that I was looking for. So I hacked together a custom data pipeline that 1) scrapes search engine results every day 2) downloads those page results as HTML content to my laptop 3) breaks down every page into sections 4) for each section, computes several scores based on the presence or absence of certain words or terms, using text similarity tools wherever necessary 5) presents a fresh top-down, custom-scored list of job postings that were just seen on that day for the first time.
To give you an idea of the kind of custom-crafted query I'm using:
'site:jobs.lever.co inurl:apply data engineer'
The codebase isn't open-source, but I can share it with you if you're interested.