4 ms·
This is pretty great. It looks like it automatically writes to the next logical empty cell. curl https://www.bitstamp.net/api/v2/ticker/btcusd \ | grep 'la
by notaboutdave 9y ago
This is pretty great. It looks like it automatically writes to the next logical empty cell.
curl https://www.bitstamp.net/api/v2/ticker/btcusd \
| grep 'last": *"\d*\.\d*' -o | grep '\d*\.\d*' -o \
| tosheets -c A1 --spreadsheet=foo
^ That will append the last BTC price to column A of sheet 'foo'
- sillysaurus3 9y agoCan I nerd out over how unreasonably effective regexps are? That's basically a mini JSON parser in 48 characters.
- gcr 9y agoWith a domain-specific tool, it's even easier though. curl ... | jq -r .last
- sillysaurus3 9y agoSure. But you can't use jq to scrape arbitrary websites, for example. :)
- jldugger 9y agoOf course, if it's anything like HTML, the formatting will vary over time that you really want a more permissive parser like BeautifulSoup. I haven't found a cli interface, so I briefly wrote my own ages ago: https://github.com/jldugger/dotfiles/blob/master/bin/select.py https://github.com/jldugger/dotfiles/blob/master/bin/select.....
- nurettin 9y agoFor cases where a website is not a tutorial for websites, regex is a suitable tool for scraping.
- dpflan 9y agoJeff Atwood has an entertaining post about parsing HTML with regular expressions: https://blog.codinghorror.com/parsing-html-the-cthulhu-way/ https://blog.codinghorror.com/parsing-html-the-cthulhu-way/ “”” That's right, if you attempt to parse HTML with regular expressions, you're succumbing to the temptations of the dark god Cthulhu's … er … code. “””
- alexozer 9y agoLet's not forget about this masterpiece: https://stackoverflow.com/a/1732454/864310 https://stackoverflow.com/a/1732454/864310
- dpflan 9y agoIndeed, its quality cannot be ignored and must be shared; it’s referenced in the Atwood post.
- ams6110 9y agoParsing and scraping are different things though. You don't need to parse a web page to extract specific things from it.
- heptathorp 9y agoDo NOT use r̔ͩegͩ̾ͪͥͪeͮ͊ͨ̓xͫ͆͆̓ ͤt͊͗o̒̾͋ͬ̾̚ ͆̌͌̄p͠a͟r̐̎͆̽̄ͭ́se̒ͥ ͕̪̻̭̭̺̳ͣͬͪͫͪj̞̲ͤ̚s̡̳̟̤̳̖̤͒̋ͣ͊ͤ͗̿oͬ̀͆n̮̳͚̝̩͙͔̈͋ ̮͕̩̼̔̾̈̄̋ͦ́́̚ͅĤ̷̝̯̝ͯ̈́ͣ̔ͪ̊ͬ͜͡Eͬͧ҉̜̰̲̩̰̝̠̥ ̶̯̯͚̗̪̭̘ͨͩͭ̎ͧͮCͪ҉̖͍͔͚͚̯͕Ơ̰̻̂̅̋̇̓̅͌M̸̭̱̭̥͆̽ͨͦÊ̸̴̢̪͚̮̲̜̙̍ͤ͋̾ͦS̛̗̟͙͍̹͈̳ͣ͑̏̓ͤͦ̽
- dang 9y agoPlease don't do this here.