6 ms·

> Literally did a two liner to scrape a web page, parse it and call a REST endpoint with the parsed data as JSON in PowerShell. This sounds cool. Can you recom

> Literally did a two liner to scrape a web page, parse it and call a REST endpoint with the parsed data as JSON in PowerShell. This sounds cool. Can you recommend terse resources for getting to know PowerShell? Is MSDN the best place to look for docs or are there better places to go?

tim333 11y ago
Not an expert but there's a "PowerShell in 10 Minutes" here: http://social.technet.microsoft.com/wiki/contents/articles/15525.powershell-in-10-minutes.aspx http://social.technet.microsoft.com/wiki/contents/articles/1...
wumbernang 11y ago
TBH I couldn't find a decent book or resource and sort of hacked my way around for a year or so. It had a good build in manual (get-help). This is a good poke at the fundamentals of my example: Scrape a page: $flight = " LH3396" $url = "http://bing.com?q=flight status for $flight" $result = Invoke-WebRequest $url $elements = $result.AllElements | Where Class -eq "ans" | Select -First 1 -ExpandProperty innerText Hit a REST endpoint: $body = @{ Name = "So long and thanks for all the fish" } Invoke-RestMethod -Method Post -Uri "$resource\new" -Body (ConvertTo-Json $body) -Header @{"X-ApiKey"=$apiKey} Sources: [1] http://stackoverflow.com/questions/9053573/powershell-html-parsing-get-information-from-a-website http://stackoverflow.com/questions/9053573/powershell-html-p... [2] http://www.lavinski.me/calling-a-rest-json-api-with-powershell/ http://www.lavinski.me/calling-a-rest-json-api-with-powershe...
- hassiktir 11y ago
  This is really cool (and unfortunately I know nothing about powershell, I'm young and dumb) but I'm pretty sure this is replicable line for line with bash (and curl, maybe awk too idk). Am I wrong?
  - will_hughes 11y ago
    The main difference between bash/gnuutils stuff and Powershell is that every command returns an object (or an enumeration of objects), and commands can take an object (or an enumeration of objects) as input. This lets everything implicitly understand how to access named properties without everyone having to do string parsing. Sure, you can solve the same problems with bash/grep/awk/sed/etc - but sometimes it's a bunch simpler to solve in Powershell.
  - ygra 11y ago
    Probably not. In my experience it's not so much about things being impossible elsewhere, just that PowerShell can often be better at fiddling in a REPL until you got the results you wanted, in a way. The point where you need (or want) to upgrade to a more powerful language is IMHO earlier in bash than in PowerShell². PowerShell handles objects like Unix utilities handle text. You get a lot more orthogonality in commands, e.g. there's just a few commands dealing with JSON, CSV, XML, etc.¹ – they mostly do just the conversion of a specific text-based format to an object list or tree representation and back. Everything else after that can be done with just the same set of core commands which deal with objects. Filtering, projecting, and iterating over object sequences is probably the most common part of PowerShell scripts and one-liners and it's a part that's useful everywhere you need the language. Note that we have a bit of that in the samples here, too. ConvertTo-JSON is used to convert a hashmap to a JSON string to use in a request, and Invoke-WebRequest already handles parsing HTML for us so the following line can just filter elements by certain attributes. Where the ideal format for the core Unix tools you use most frequently, is free-form text, you often have special commands that work on other formats by replicating a few core tools' features on that format, e.g. a grep for JSON, a grep for XML, etc. There are others, of course, that do the conversion in a way that's friendly to text-based tools, but the representation still lacks fidelity. Finding XML elements with certain attributes quickly becomes an exercise in how to write robust regexes. Cutting up CSV by column numbers is a frequent occurrence – and since the tools do not understand the format it makes for not a pretty read in the script's code. Personal opinion here, based on lots of PowerShell written, some shell scripts written, lots of horrible things read in either (sure, awful PowerShell scripts do exist, but I'd argue that discovering a nice way is much easier where you can cut most of the ad-hoc parsers written in regex or string manipulation in pipelines). (One last point about orthogonality of commands: ls has a few options on how to sort or output the results, for example. Sorting a list of things? That's Sort-Object's domain. Formatting a list of things? Format-Table, Format-Wide, Format-List. That's quite a bit less each individual command has to do and it's all just for nicety to the user. For working with the output programmatically you don't (well, and can't) need them at all.) I have a few posts on SO where I tried to steer people into actually learning how the language works and that you should use the pipeline as much as possible, e.g. http://stackoverflow.com/a/7394766/73070 http://stackoverflow.com/a/7394766/73070 or http://stackoverflow.com/a/3104721/73070 http://stackoverflow.com/a/3104721/73070. That's not to say you can't write un-understandable things: http://stackoverflow.com/q/1018873/73070 http://stackoverflow.com/q/1018873/73070. And finally, let's not forget that PowerShell exists on Windows where text-based tools are mostly useless. Want to query the event log? The registry? WMI? Good luck. Windows has a long history of having non-text formats everywhere in the system and for administration it's a bit hard to pretend they don't exist. Jeffrey Snover elaborates a bit on that here: http://stackoverflow.com/a/573861/73070 http://stackoverflow.com/a/573861/73070. There may be a lot of developers getting by with Unix tools on Windows, but they're not the target audience for PowerShell³. And the previous approach to scripting things on Windows servers and domains was either batch files or VBScript/JScript. This ... uhm, got longer than anticipated and probably a lot less coherent than planned. Apologies for anything that makes no sense. I didn't have caffeine yet. ______ ¹ ConvertFrom-JSON and ConvertTo-JSON for JSON for example. For CSV there are also convenience commands that directly work on files as well, so there's four of them. XML processing is built-in since .NET can do that easily already. ² I probably don't get to live the rest of the day now, I guess. ³ I am a developer, though, who uses PowerShell daily for scripting, as a shell, and a .NET playground. The PowerShell team at MS was a bit surprised once when I told them that my background was not server administration (the Scripting Games were quite focused on that part and often involved doing things with WMI or AD – stuff I rarely, if ever, do.)
- flashman 11y ago
  Ha, now when you Google "flight status for LH3396" the fourth result is your StackOverflow link.
majkinetor 11y ago
Powershell in Depth is the best book about PS there is: http://www.manning.com/jones6/ http://www.manning.com/jones6/

tim333 11y ago

Not an expert but there's a "PowerShell in 10 Minutes" here: http://social.technet.microsoft.com/wiki/contents/articles/15525.powershell-in-10-minutes.aspx http://social.technet.microsoft.com/wiki/contents/articles/1...

wumbernang 11y ago

TBH I couldn't find a decent book or resource and sort of hacked my way around for a year or so. It had a good build in manual (get-help). This is a good poke at the fundamentals of my example: Scrape a page: $flight = " LH3396" $url = "http://bing.com?q=flight status for $flight" $result = Invoke-WebRequest $url $elements = $result.AllElements | Where Class -eq "ans" | Select -First 1 -ExpandProperty innerText Hit a REST endpoint: $body = @{ Name = "So long and thanks for all the fish" } Invoke-RestMethod -Method Post -Uri "$resource\new" -Body (ConvertTo-Json $body) -Header @{"X-ApiKey"=$apiKey} Sources: [1] http://stackoverflow.com/questions/9053573/powershell-html-parsing-get-information-from-a-website http://stackoverflow.com/questions/9053573/powershell-html-p... [2] http://www.lavinski.me/calling-a-rest-json-api-with-powershell/ http://www.lavinski.me/calling-a-rest-json-api-with-powershe...

hassiktir 11y ago
This is really cool (and unfortunately I know nothing about powershell, I'm young and dumb) but I'm pretty sure this is replicable line for line with bash (and curl, maybe awk too idk). Am I wrong?
- will_hughes 11y ago
  The main difference between bash/gnuutils stuff and Powershell is that every command returns an object (or an enumeration of objects), and commands can take an object (or an enumeration of objects) as input. This lets everything implicitly understand how to access named properties without everyone having to do string parsing. Sure, you can solve the same problems with bash/grep/awk/sed/etc - but sometimes it's a bunch simpler to solve in Powershell.
- ygra 11y ago
  Probably not. In my experience it's not so much about things being impossible elsewhere, just that PowerShell can often be better at fiddling in a REPL until you got the results you wanted, in a way. The point where you need (or want) to upgrade to a more powerful language is IMHO earlier in bash than in PowerShell². PowerShell handles objects like Unix utilities handle text. You get a lot more orthogonality in commands, e.g. there's just a few commands dealing with JSON, CSV, XML, etc.¹ – they mostly do just the conversion of a specific text-based format to an object list or tree representation and back. Everything else after that can be done with just the same set of core commands which deal with objects. Filtering, projecting, and iterating over object sequences is probably the most common part of PowerShell scripts and one-liners and it's a part that's useful everywhere you need the language. Note that we have a bit of that in the samples here, too. ConvertTo-JSON is used to convert a hashmap to a JSON string to use in a request, and Invoke-WebRequest already handles parsing HTML for us so the following line can just filter elements by certain attributes. Where the ideal format for the core Unix tools you use most frequently, is free-form text, you often have special commands that work on other formats by replicating a few core tools' features on that format, e.g. a grep for JSON, a grep for XML, etc. There are others, of course, that do the conversion in a way that's friendly to text-based tools, but the representation still lacks fidelity. Finding XML elements with certain attributes quickly becomes an exercise in how to write robust regexes. Cutting up CSV by column numbers is a frequent occurrence – and since the tools do not understand the format it makes for not a pretty read in the script's code. Personal opinion here, based on lots of PowerShell written, some shell scripts written, lots of horrible things read in either (sure, awful PowerShell scripts do exist, but I'd argue that discovering a nice way is much easier where you can cut most of the ad-hoc parsers written in regex or string manipulation in pipelines). (One last point about orthogonality of commands: ls has a few options on how to sort or output the results, for example. Sorting a list of things? That's Sort-Object's domain. Formatting a list of things? Format-Table, Format-Wide, Format-List. That's quite a bit less each individual command has to do and it's all just for nicety to the user. For working with the output programmatically you don't (well, and can't) need them at all.) I have a few posts on SO where I tried to steer people into actually learning how the language works and that you should use the pipeline as much as possible, e.g. http://stackoverflow.com/a/7394766/73070 http://stackoverflow.com/a/7394766/73070 or http://stackoverflow.com/a/3104721/73070 http://stackoverflow.com/a/3104721/73070. That's not to say you can't write un-understandable things: http://stackoverflow.com/q/1018873/73070 http://stackoverflow.com/q/1018873/73070. And finally, let's not forget that PowerShell exists on Windows where text-based tools are mostly useless. Want to query the event log? The registry? WMI? Good luck. Windows has a long history of having non-text formats everywhere in the system and for administration it's a bit hard to pretend they don't exist. Jeffrey Snover elaborates a bit on that here: http://stackoverflow.com/a/573861/73070 http://stackoverflow.com/a/573861/73070. There may be a lot of developers getting by with Unix tools on Windows, but they're not the target audience for PowerShell³. And the previous approach to scripting things on Windows servers and domains was either batch files or VBScript/JScript. This ... uhm, got longer than anticipated and probably a lot less coherent than planned. Apologies for anything that makes no sense. I didn't have caffeine yet. ______ ¹ ConvertFrom-JSON and ConvertTo-JSON for JSON for example. For CSV there are also convenience commands that directly work on files as well, so there's four of them. XML processing is built-in since .NET can do that easily already. ² I probably don't get to live the rest of the day now, I guess. ³ I am a developer, though, who uses PowerShell daily for scripting, as a shell, and a .NET playground. The PowerShell team at MS was a bit surprised once when I told them that my background was not server administration (the Scripting Games were quite focused on that part and often involved doing things with WMI or AD – stuff I rarely, if ever, do.)
flashman 11y ago
Ha, now when you Google "flight status for LH3396" the fourth result is your StackOverflow link.

majkinetor 11y ago

Powershell in Depth is the best book about PS there is: http://www.manning.com/jones6/ http://www.manning.com/jones6/