I have been using lynx to integrate a chatbot with web content. Here's the Python function that converts a website to plain text: def website_to_text(url):

I have been using lynx to integrate a chatbot with web content. Here's the Python function that converts a website to plain text: def website_to_text(url): output = subprocess.check_output(["lynx", "-dump", "-nolist", url], stderr=subprocess.PIPE, universal_newlines=True) return output.strip() While a hosted service can be convenient, I find that using a text-based browser like Lynx gives almost the same results and consistent results.