3 ms·

don't know why this is broken, but I'll file a bug. thank you! One of these days, I'm going to run a bot that crawls all pages on yahoo.com and reports non-200

• akg_67 11y ago
  Or let users help you identify such pages. Instead of showing a vague message that indicates temporary failure due to time out, show the 404 with option to report the broken link. I miss picks because that's how I kept track performance of results from my own algos and compare with supposedly experts and crowds.
• 1024core 11y ago
  Why write a bot? Doesn't yapache log such events? Just run a Hadoop job on the error logs.
  • perryh2 11y ago
    I'd run it across the entire site. Sure, we do have logs, but it's not easy to access them for all *.yahoo.com. Logs from each property are ingested into different places. The infrastructure is way more complex than it should be making bare-minimum monitoring like alerts on non-200 responses on all properties difficult to maintain.
• cpeterso 11y ago
  Shouldn't the yahoo.com servers already log when they return error responses? :)
  • perryh2 11y ago
    Each property has their logs ingested, indexed, and archived differently. I only have access to my little arm of the company infrastructure. Logs should be monitored properly by the responsible owners of each property (or in a more sensible way), but unfortunately this is not the case... :)