3 ms·
Nice tool. I'm interested in Chinese websites primarily so here is some code I wrote to dump text from a Chinese website. import re import requests
by foobarqux 8y ago
Nice tool. I'm interested in Chinese websites primarily so here is some code I wrote to dump text from a Chinese website.
import re
import requests
from bs4 import BeautifulSoup
url='http://www.36kr.com'
r=requests.get(url)
soup= BeautifulSoup(r.text, 'lxml')
RE = re.compile(u'[^⺀-⺙⺛-⻳⼀-⿕々〇〡-〩〸-〺〻㐀-䶵一-鿃豈-鶴侮-頻並-龎]', re.UNICODE)
# txt = soup.find_all(string=RE)
txt = re.sub(RE, '', soup.get_text())
print(txt)