4 ms·
I've been tinkering on crawlers to doomscroll job boards for me, along with a half-formed hybrid of a CRM and Tinder for swiping through the various results and
by chaosharmonic 2y ago
I've been tinkering on crawlers to doomscroll job boards for me, along with a half-formed hybrid of a CRM and Tinder for swiping through the various results and then tracking them.
I eventually want this to be usable enough to share with others for local use, and to demo on my portfolio, but for now I'm just trying not to spend more time building the tools than I am using them (and harvesting the redirect links is already enough to solve a lot of what makes the search process miserable). In the meantime it's mostly a cluster of experiments that range from teaching myself Web scraping, to generally aiming for a leaner tech stack than my prior full-stack experiences, to specifically asking myself "how much of this can I do in pure CSS?"
While I'm not sure I want the support burden of maintaining the scraping code in public (and anyway it makes more sense imo to just enable bulk upload so you can bring your own sources), I do have a massive beast of a writeup[0] documenting a lot of what went into that, and the API code is at least presentable (see my GitHub) even if the UI is too skeletal for that as of now.
[0] https://bhmt.dev/blog/scraping https://bhmt.dev/blog/scraping (fair warning: this is 10000 words not including code samples)