10 ms·
I am glad that someone is working on a new stream processing language, it is a very interesting paradigm. However I hope that they provide some very robust tool
by c3RlcGhlbnI_ 12y ago
I am glad that someone is working on a new stream processing language, it is a very interesting paradigm. However I hope that they provide some very robust tools for controlling input splitting. As I have spent too much time fighting with awk and wishing it was more flexible(it is frustrating always having exactly two levels of splitting with only matchers on the first and only inverse splitting on the second).
As is you would have to put in filters to resplit input into lines and that is very messy for something that you will need/want to do very often.
For example if you wanted to parse by character it would be wonderful to be able to do the following:
STDIN | /./{|c|
# stuff
}
Even better would be if you took it a step further and offered something like regex pattern matching for the block input. e.g.
STDIN | /\w+/{|word|
/house/ {
# when word is house
}
/car/ {
# when word is car
}
{
# default case
}
}
- nateabele 12y ago> However I hope that they provide some very robust tools for controlling input splitting Yes, splitting and combining. Looking at the code example, I feel like so much opportunity is just being squandered, where something like Rust's mapping constructs would feel so much better. Yours is definitely more in that vein, here's something a little closer: STDIN | /\w+/{|word| /house/ => "foo" /car/ => "bar" "literal" => "baz" _ => "dib" # (where _ is a magical symbol for the default case... could be anything) }
- rudolf0 12y agoStreaming, pattern matching, and regex combined in one sounds pretty damn cool. Might also be nice to borrow some of Scala's features for terser code than Ruby, like ary | { _ + 5 } instead of ary | { |el| el + 5 }
- spoiler 12y agoI prefer {|el| el + 5} more than { _ + 5 } the latter is too implicit for my taste, also I imagine it could become more confusing in a more complex context.
- rudolf0 12y agoPersonally I like it and find it clear. Compare: ary.map(_ * 2) ary.map(x => x * 2) "_" is perhaps an ugly choice of character (frankly I'm not sure why Scala is so obsessed with it, since it's used for so many featurse), but I think the semantics are sensible. Of course, in a more functional language you could perhaps just write ary.map(* 2)
- kbenson 12y agoProbably because underscore is often used by convention in forms and other typography to indicate "something goes here which is to be supplied." In that respect it's no different that Perl's $_ (actually it's the Exact same thing, as the $ in Perl just denotes a variable, so they are both the variable _), or Perl 6's "whatever" which is written as "*".
- Alphasite_ 12y agoWhat about a middle ground on swift's syntax? Arr.map { $1 * 2 }
- ryanlschneider 12y agoThat was going to be my suggestion too. With the added benefit that is supports multiple arguments.
- rudolf0 12y agoYeah, that'd be pretty cool.
- breuleux 12y agoThis is not a confusing context, though. This would be a confusing context: ary.map(f(_ * 2)) ==> ary.map(x => f(x * 2)) ? ==> ary.map(f(x => x * 2)) ? How does Scala interpret that one? I have no idea.
- ssmoot 12y ago
- breuleux 12y agoPersonally I would go for something more like this: STDIN | /\w+/ | {|each match| /house/ => "foo" /car/ => "bar" "literal" => "baz" _ => "dib" } Where "each x" in a parameter list wraps the body in a map and uses x as the loop variable, and "match" in a parameter list instructs that this is the parameter we are doing pattern matching on in the body (no need to name it). I think this is a more flexible solution, and more modifiers than each could be introduced. For instance, sum of squares could be: seq(100) | {|each x| x * x} | {|reduce (x = 0, y)| x + y}
- ori_b 12y agoSounds like a job for structural regex: http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf
- dwenzek 12y agoYes, this paper is interesting. It shows well how a slight shift on the problem expression can lead to a more powerful, simpler to use and general solution. In this case, the shift is to design stream processors (like `sed` and `awk`) to work on streams of text chunks defined by regular expressions rather on streams of lines matching regular expressions. Regarding this concern, it's hard to say what is the plan of Matz for Streem with a single code example. But the fact, that this sample code processes a stream of integers, lets hope a tool with more flexibility than `sed` or `awk` to specify what is processed along the streams.
- mackwic 12y agoAwesome ! Let's call it sed ! Reference: https://www.gnu.org/software/sed/manual/sed.html#Examples https://www.gnu.org/software/sed/manual/sed.html#Examples Sorry, I couldn't miss that one. :) Really, check the example. Sed is most powerful tool and can do astonishing work.
- GFK_of_xmaspast 12y agoEverything you can do in sed you can do in perl, there's been sed-to-perl converters shipped with perl since the ur-times.
- mackwic 12y agoYeah yeah, of course. The hacked syntax showed by GP made me really think of sed. :)
- c3RlcGhlbnI_ 12y agoThat manual is possibly the single most useful technical document that I have ever read. It has enabled me to write extremely powerful programs that no one I know can understand. It would however be nice to have a tool with similar power but simpler, more comprehensible syntax. One of the other commenters linked to an interesting document on sam, which has a better control flow but equally arcane syntax.
- mamcx 12y ago" new stream processing language"... which are the olds? I'm playing with the idea of build a language, where the functions are unix-like, with STDIN, OUT & ERR. So, instead of raise a exception, it put data in ERR... and make it easy to compose them.
- c3RlcGhlbnI_ 12y agoThere don't seem to be that many that work on text streams, which is what I was referring to. The ones I am aware of are awk, shell scripting languages(bash, zsh, ksh, etc.), and the ed like DSLs( sed, ed, sam, etc.). I think there are others that are not as widely used. If you google stream processing language you will also get a whole bunch that are not text specific. They are definitely worth looking at if you are interested in building any kind of stream based language.
- bglazer 12y agoI'd like to recommend a python command line utility called pyp. It allows you to do python string manipulation on text streams using the standard pipe operator. https://code.google.com/p/pyp/ https://code.google.com/p/pyp/
- vram22 12y agoYes, pyp is interesting. So are some other roughly similar Python tools for doing pipe-like stuff. I had blogged about some of them here, a while ago, including pyp, osh, and pipe (not the Unix pipe system call, but a Python module): Some ways of doing UNIX-style pipes in Python: http://jugad2.blogspot.in/2011/09/some-ways-of-doing-unix-style-pipes-in.html http://jugad2.blogspot.in/2011/09/some-ways-of-doing-unix-st... And that later inspired me to create an experimental tool called pipe_controller: Swapping pipe components at runtime with pipe_controller: http://jugad2.blogspot.in/2012/10/swapping-pipe-components-at-runtime.html http://jugad2.blogspot.in/2012/10/swapping-pipe-components-a... The above post links (recursively) to a few others, also by me, describing a couple of other ways of using pipe_controller, including this one: Using PipeController to run a pipe incrementally: http://jugad2.blogspot.in/2012/09/using-pipecontroller-to-run-pipe.html http://jugad2.blogspot.in/2012/09/using-pipecontroller-to-ru... and also includes the link for it on my Bitbucket account. There's also plumbum, another such Python module.
- pablovidal85 12y agoThis inspired me to build a similar tool which runs javascript instead. https://github.com/aynik/sx https://github.com/aynik/sx
- jkgeyti 12y agoI found it cumbersome to do branching in most of the existing python stream tools. I made an alternative, that allows you to use curly brackets for indentation: https://github.com/ircflagship2/pype https://github.com/ircflagship2/pype
- klibertp 12y agoTake a look at TXR: http://www.nongnu.org/txr/ http://www.nongnu.org/txr/ The link was posted on HN some time ago and was generally well received. I didn't have the time to use it much yet, but it looks very nice. Well, unless you hate lisp.
- bjourne 12y agoThis is how you would write it in factor: "~/yourfile.txt" utf8 file-contents "\\w+" findall [ first second ] map [ { { "house" [ "house stuff" print ] } { "car" [ "car" print ] } [ "default stuff: %u\n" printf ] } case ] each Tacit programming basically is the same thing as stream programming.
- marianov 12y agoAWK http://en.m.wikipedia.org/wiki/AWK http://en.m.wikipedia.org/wiki/AWK