3 ms·

>Have you considered any sort of tiered index to allow one to classify a problem at hand and look up the relevant structures int he book? I have wondered a few

by asicsp 6y ago

>Have you considered any sort of tiered index to allow one to classify a problem at hand and look up the relevant structures int he book? I have wondered a few times if I can build some sort of guide/flowchart that allows one to pick tool(s) for a particular text processing job (for ex: when to choose head/tail/grep/sed/awk/pr/paste/etc). And then within the tool(s) chosen, how to break down a problem and select particular features of that tool. It has remained a fantasy as I haven't tried to actually start doing something about this. I dunno if this is feasible and/or whether I'll be able to do it. >The reason I ask is that I've never figured out a good way to make use of this type of material. There are several prerequisites mentioned in the Preface before you can use this book. You'll have to be familiar with cli environment as well as Perl basics. After that, I feel the best way would be to start using these one-liners in your everyday tasks. If your workflow doesn't suit this type of usage, you'll have to be either motivated enough to explore or take a risk to change/modify your workflow. I provide several examples for each topic and have exercises at the end of chapters. Broadly speaking, there are three main tasks (tools mentioned in parentheses is the fundamental task of that tool) * Filter an input record based on regexp, string or numeric comparisons (grep) * Search and replace (sed) * Field based filtering/substitution (awk) awk and perl can do all of these tasks. However, a tool specialized for a task is usually faster and has custom options for ease of use for those tasks. So, my best advice would be to practice and find reasons to incorporate cli based solutions in your everyday workflow.

kqr 6y ago
I consider myself an intermediate-or-advanced user of both sed and awk. Those are small enough languages that I was able to study them for a few hours and then know roughly how to solve any problem with them. Perl one-liners are different in that they rely on a large set of flags and language shortcuts which are harder to remember when your main language isn't Perl. So it's not about getting used to the POSIX utilities and text processing, it's more about it being infeasible to "just memorise" all these Perl shortcuts. The traditional solution when "just memory" won't cut it is some sort of index to speed up the search. The guide/flowchart thing is just what I'm talking about. Botanists use a tiered index to identify plants: they subdivide the flora first by large differences in characteristics and then down to differences you need a magnifier to see. Each observation cuts the potential space by a significant fraction. Something similar would be useful for this. E.g. "is your input structured or free text?" "Are records separated by linebreak or something else?" "Are there fields in the records?" "Are you looking to perform an aggregation?" And so on. I don't know. It would be neat for many reasons!
- asicsp 6y ago
  >"is your input structured or free text?" "Are records separated by linebreak or something else?" "Are there fields in the records?" "Are you looking to perform an aggregation?" Yeah, that'd be awesome. I guess I'll just have to give a shot at it sometime, get feedback, improve and repeat the process until there's a good enough resource. That'll likely give me good returns in terms of sales too. For now, writing one yourself could help. Since you know sed/awk, try writing equivalent ones with Perl with help of the book. For options, create a cheatsheet that you can refer quickly from the command line. Whenever you see something that isn't easily possible with sed/awk, put it under a special section so that you can build a list where perl would be better (I give some examples of this in the first section of introduction chapter). I also mention resources for getting started with Perl in the Preface chapter. I often use my books as a reference guide, because I can't remember all the syntax and gotchas. The situation is worse now compared to few years back because I have had to dive deeper to write these books. I'm old enough that learning new things has become difficult. I wish to create a cli lookup (something similar to the guide/flowchart), but for now the web version created with mdbook serves as the best way for me to look up something.

by asicsp 6y ago

kqr 6y ago

I consider myself an intermediate-or-advanced user of both sed and awk. Those are small enough languages that I was able to study them for a few hours and then know roughly how to solve any problem with them. Perl one-liners are different in that they rely on a large set of flags and language shortcuts which are harder to remember when your main language isn't Perl. So it's not about getting used to the POSIX utilities and text processing, it's more about it being infeasible to "just memorise" all these Perl shortcuts. The traditional solution when "just memory" won't cut it is some sort of index to speed up the search. The guide/flowchart thing is just what I'm talking about. Botanists use a tiered index to identify plants: they subdivide the flora first by large differences in characteristics and then down to differences you need a magnifier to see. Each observation cuts the potential space by a significant fraction. Something similar would be useful for this. E.g. "is your input structured or free text?" "Are records separated by linebreak or something else?" "Are there fields in the records?" "Are you looking to perform an aggregation?" And so on. I don't know. It would be neat for many reasons!

asicsp 6y ago
>"is your input structured or free text?" "Are records separated by linebreak or something else?" "Are there fields in the records?" "Are you looking to perform an aggregation?" Yeah, that'd be awesome. I guess I'll just have to give a shot at it sometime, get feedback, improve and repeat the process until there's a good enough resource. That'll likely give me good returns in terms of sales too. For now, writing one yourself could help. Since you know sed/awk, try writing equivalent ones with Perl with help of the book. For options, create a cheatsheet that you can refer quickly from the command line. Whenever you see something that isn't easily possible with sed/awk, put it under a special section so that you can build a list where perl would be better (I give some examples of this in the first section of introduction chapter). I also mention resources for getting started with Perl in the Preface chapter. I often use my books as a reference guide, because I can't remember all the syntax and gotchas. The situation is worse now compared to few years back because I have had to dive deeper to write these books. I'm old enough that learning new things has become difficult. I wish to create a cli lookup (something similar to the guide/flowchart), but for now the web version created with mdbook serves as the best way for me to look up something.