3 ms·
This is nice! Would love to know whether found a need to define any custom logic for 'international' websites? For example a lot of global companies will have
by crowcroft 3y ago
This is nice!
Would love to know whether found a need to define any custom logic for 'international' websites?
For example a lot of global companies will have
www.theirdomain.com/example-page
But then also have
www.theirdomain.com/us/example-page
www.theirdomain.com/au/example-page
www.theirdomain.com/nz/example-page etc...
If you're crawling from one regional or the 'global' entry point you might not always find all of the regional variations of pages. Sometimes the site might redirect you to your previous region if you try to change.
A common tool SEO teams use is screaming frog which offers similar features, crazy that a web service isn't more commonly used for this kind of thing.
- sph 3y agoThat's a good question. Right now you can only enter root domains, but eventually I want to relax it and you'd be able to enter `example.com/en` and `example.com/au` separately. If the webserver redirects based on IP geolocation, it is planned way down the line to have a few Bernard workers spread around the globe, but I'll be honest, it's not my highest priority right now unless this is a common enough issue.
- crowcroft 3y agoInteresting, probably a good example of that last 2% of a product that could easily take 80% of your time. Big enterprises that might want this kind of thing are probably also the types of people that are least likely to use this kind of tool though.