5 ms·
Before jumping into frameworks, if your data is lucky enough to be stored in an html table: import pandas as pd dfs = pd.read_html(url) Where ‘dfs’ is
by philshem 6y ago
Before jumping into frameworks, if your data is lucky enough to be stored in an html table:
import pandas as pd
dfs = pd.read_html(url)
Where ‘dfs’ is an array of dataframes - one item for each html table on the page.
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_html.html https://pandas.pydata.org/pandas-docs/stable/reference/api/p...
- andreilys 6y ago+1 this has saved me countless of hours
- mikesholiu 6y agowhat does this do
- SirSourdough 6y agoIt reads HTML and returns the tables contained in the HTML as pandas dataframes. It’s a simple way to scrape tabular data from websites.
- JosephRedfern 6y agoWoah. I’ve used pandas a fair amount and had no idea about this. Thank you!
- kortex 6y agoSometimes it's also helpful to use beautiful soup to isolate the elements you want, feed the text of the elements into StringIO and give that to read_html.
- philshem 6y agoYes, this is a good idea for more complicated cases.
- antisthenes 6y agoHoly crap, is there anything pandas can't do?
- Frost1x 6y agoIngest bamboo? Sorry, couldn't resist.
- Yenrabbit 6y agoHandy! I'm also a big fan of pd.read_clipboard() for specific selections.
- mritchie712 6y agoWe made a chrome extension that queries any html table in any open tab with SQL: https://chrome.google.com/webstore/detail/sqanything/naejbcfcmjhcgcjhbddfogknbaggdoek?hl=en-US https://chrome.google.com/webstore/detail/sqanything/naejbcf... You can export the results to Google Sheets too. One advantage of the extension is it works with JS rendered tables.