4 ms·

This isnt one of what you described though. I've literally done this before. This is an extremely simple product that could be written in a morning by someone

• haswell 3y ago
  > by someone competent Which is why there will always be a market for products like this. The point is that someone will pay a developer (someone competent) for software that just solves the problem for them. Plenty of people run websites and have no idea how to even begin something like this. The curse of knowledge we have in the bubble we're in makes these things appear simpler than they are to most people.
• sph 3y ago
  It's a bit rude to assume I am incompetent. I am sorry, but you vastly underestimate the problem. Just read the other comments and see how many corner cases are there. It took me 3 days to implement a crawler than says OK on 200, and ERROR on 404, and 6 months to deal with exceptions to the rule, and apparently thanks to the feedback Ive received, there are many other exceptions to deal with. Sorry, this is a bit like that old Dropbox comment. The thing is, if your website is easily covered by a 50-line Python script, you do not need Bernard.
  • hasty_pudding 3y ago
    http is just a very simple request response protocol. maybe you're right though