3 ms·
You shouldn‘t really need to do any scraping tricks to get that, because it‘s data the websites (usually) want to give to bots. Or are people getting bot block
by kall 5y ago
You shouldn‘t really need to do any scraping tricks to get that, because it‘s data the websites (usually) want to give to bots. Or are people getting bot block screens from Cloudflare et all for that basic action these days?
It should be a matter of a simple GET request to fetch plain html and parse the OpenGraph meta tags out if that. There are many open source libraries to do that for you depending on your language.
If bot blocks really are a problem, a SaaS solution like Microlink could probably do it for you.
- wilg 5y agoBot blocks are definitely an issue for certain sites, I've implemented it that way currently. Microlink is a good tip, thanks!