10 ms·
Show HN: LINQ for Go
- krakensden 13y agoFrom the documentation: > So if you have 1M elements as input to a plinq function, you will end up 1M goroutines. > Theoretically, there is not much wrong with that. Understatement in the finest tradition of man pages and RFCs.
- aabalkan 13y agoI wrote that statement thinking about the experiments people are making using millions of goroutines and still end up low cpu utilization when goroutines mainly depend on IO. Go has a neat scheduling for that. Please correct me if I am wrong.
- kevingadd 13y agopretty much any task/job/thread scheduler on the planet is going to pay some absurd overhead costs to schedule and run a million tasks compared with a reasonable number (like 10, or 100), without a measurable improvement in performance. a very smart scheduler can minimize these costs but they will still be there; the state has to be stored and updated.
- RyanZAG 13y agoDoes it really sound alright to you? I mean it's not the end of the world or anything, but it's adding extra overhead which simply isn't needed when you're not gaining anything from it in this case. I assume people want to use golang for the speed advantages of servicing many connections at once. If each connection is spawning hundreds of goroutines it's not going to work so well.
- aabalkan 13y agoI gotta admit I never seen the problem that way, now I can clearly see I am wrong. Probably I need to limit number of goroutines spawned with a configurable default value. Thanks for pointing out Ryan!
- nathanwdavis 13y agoHere's a gist I created a couple months ago that kind of explains the performance characteristics of goroutine per loop vs. partitioning with goroutines https://gist.github.com/nathanwdavis/6290428 https://gist.github.com/nathanwdavis/6290428 Basically, for big lists a goroutine per loop is very slow in comparison.
- JeanPierre 13y agoJust for reference, a single goroutine has a 8 kb stack size. Having one million of these things in memory at once would require 8 Gb of memory. Obviously, the scheduler kicks in, but 8 kb chunks of memory still has to be assigned and freed for every routine started and stopped.
- krakensden 13y agoNot intended as a criticism- no one writes every possible feature immediately! It was me expressing enjoyment at the dry way you documented this implementation detail.
- kmontrose 13y agoAnother approach using code generation is gen: http://clipperhouse.github.io/gen/ http://clipperhouse.github.io/gen/ (Disclaimer, I work with the guy behind it) Personally these sorts of libraries just strengthen my opinion that golang really needs proper generics. However, if you're already gonna use Go they're pretty neat.
- aabalkan 13y agoYes I know that. I have been in touch with Matt several times about this. I really look for ways to improve it, even if it is one small step at a time. Just posted a thread to golang-nuts about this: https://groups.google.com/forum/#!topic/golang-nuts/nLwWkoS7Bh8 https://groups.google.com/forum/#!topic/golang-nuts/nLwWkoS7... noting Go really needs generics and maybe even lambdas some day. Just leaving this project here. Hopefully, I will maintain it as bugs etc arises. Who knows, maybe someday Go will have these features and there will be many apps instrumenting Go for their application logic and use it.
- burntsushi 13y ago> Go really needs ... lambdas some day Eh? Go has lambdas: func(x int) int { return x * x }(5)
- overgard 13y agoNeat. I guess Go can't do type inference on the function types though? Those anonymous function signatures are all kinds of gross.
- midas007 13y agoAnd the dots. Yuck.
- masklinn 13y ago> I guess Go can't do type inference on the function types though? It can't, and there are no generics which is why the input has to be cast back to a Student* in every function. Also why the Sum implementation and the min, max and order (sort) situations are absolutely gross.
- mratzloff 13y agoThis is one of two main complaints I have with Go[0], which I've used professionally for the last six months or so. There have been several times when I've thought "if only I had generics..." Instead I either have to use interfaces and casts (run-time when I would prefer compile-time) or write separate functions for each type (I wrote a Cartesian product library that abstracts the math involved to indexes and lengths, then requires a simple type-specific wrapper simply because of this silly language gap). Despite that, I still really love Go. [0] The other being that 1.2 randomly segfaults on my code base during compilation whereas 1.1.2 works fine. For this reason I haven't yet been able to upgrade until I determine the cause.
- codygman 13y agoI wrote Go for about 6-8 months and having to do so much stuff run-time when I wanted the guarantees of compile-time started grating on me along with generics and the type system being stringly typed. I found Haskell shortly thereafter and that is where I currently stay ;)
- mratzloff 13y ago
- DanWaterworth 13y agoyuck, I don't understand how go people can put up without parametric polymorphism.
- dsymonds 13y agohttp://commandcenter.blogspot.com.au/2011/12/esmereldas-imagination.html http://commandcenter.blogspot.com.au/2011/12/esmereldas-imag...
- drdaeman 13y ago"Can't imagine/understand" is a figure of speech, not literal statement to be taken by its exact meaning. Personally, I can imagine how someone could live without $conceptname at all. It's as trivial as just never introducing that concept to to the extent they learn it. Or, maybe, that someone knows the concept but unable to use it due to insuperable force, like a lack of necessary tools for target platform(s) or corporate standard dictating what's permitted and what's not. The point is, this is an unpleasant imagination one don't want to accept. Or something like that. But this is really getting off-topic...
- dsymonds 13y agoI'm fully aware that it's a figure of speech. I'm calling out the hyperbole because it doesn't enlighten, and lacks self-reflection. A much more useful comment would have been to describe how crucial $concept is to his programming practice, and how he tried working without $concept but found it a dreadful experience because of $x, $y and $z.
- DanWaterworth 13y agoThanks for the feedback, I was being quite dismissive. In this case, at least to me, it seems quite clear that parametric polymorphism would be a win - that the addition of parametric polymorphism would allow the library to be expressed and used in a more eloquent manner. The purpose of my comment was to draw attention to this.
- MichaelGG 13y agoThe real language part of LINQ isn't the simple sequence functions (.NET had them before LINQ), it's the LINQ monad embedded into the language. While personally I don't think I've ever used the LINQ monad syntax, and wish it had been implemented as a generic feature, not every language with support for anonymous functions or closures or function pointers is "LINQ".
- olmo 13y agoSome problems with the Go implementation: Looks like go does not have something similar to extension methods, so a result() call should be put at the end to unwrap the collection. The lambda syntax is even worst than C++ and no query expressions (aka monad syntax) Most important: No IQueryable<T> Expression<Func<...>> so doesn't work with remote data sources (SQL, mongo,...) But the library looks ok given the possibilities
- fleitz 13y agoGo pretty much only has extension methods.
- aabalkan 13y agoIn .NET, extension methods are functions defined on the classes you don't own, like string or System.Net.WebRequest. In Go, you cannot define functions on builtin types and exported types from packages other than your own package. So Go way of defining functions on types is not exactly extension methods.
- aabalkan 13y agoGreat comments. I have thought about creating an interface so people can just implement their own query evaluation methods and, as you said, do querying from remote data sources. That's still a long-term plan, though, I probably need to see somebody using it before digging that deep.
- MichaelGG 13y ago
- virtualwhys 13y agoNice job but sorry, that is some hideous syntax require to make LINQ Go ;-) Was expecting the concision and elegance of LINQ-to-SQL.
- codereflection 13y agoCan't say I'm a fan of the syntax, but nice work nonetheless.
- bad_user 13y agoIn LINQ, the extension methods defined on IEnumerable are useful, but not that interesting, because they are standard functionality in most modern languages. I also think the naming used (e.g. Where, Select) is unfortunate and it would be better to go with the naming used by other languages/frameworks that are much more standard, like filter() or map(). This implementation is also missing SelectMany (e.g. flatMap or bind), which is much more interesting than Select and Where (i.e. you can implement Select and Where in terms of SelectMany). What makes LINQ special is IQueryable, e.g. the ability to transform a C# expression at runtime into something else, like a database query. Whenever I see article titles like "LINQ in X", I somehow know that it doesn't refer to IQueryable, which is a pity. Anyway, these functions should be standard in any modern programming language and because Go is missing such standard functionality is one reason for why I don't like Go. Lack of generics has much to do with it, but also Go's interfaces are weaker than type-classes, combined with Google's disregard for everything we've learned about type-systems in the past 30 years (behold Dart's covariant generics) and so I do have reasons for not holding my breath waiting for a fix. Of course, Go is what many people want, so Go is what many people receive.
- V-2 13y agoHowever, most languages in common use are not "modern" by this definition (if by any) ;) I've switched from C# to Java and the clunkiness of it drives me crazy. I know that Java 8 brought a new LINQ-like API, but it won't be available for Android development for a while (not to mention there's existing codebase to maintain etc.).
- noelwelsh 13y agoThis is very true -- most languages in common use are 20 or more years old and were not cutting edge when they were created. However you don't have to accept the status quo. Scala is viable on the JVM, F# on .Net, and Haskell (and increasingly Rust) if you want to avoid a VM.
- V-2 13y agoC# has enough functional capabilities (for typical development needs like mine), so I never felt the need of using F#. Scala, on the other hand, looked quite neat.
- shadowmint 13y agoWorth noting that the syntax example is slightly incorrect. Since go auto-postpends ; to lines, you must have a trailing . on the line to allow the compiler to consider multiline statements like this as multiline statements instead of multiple (syntactically incorrect) single statements. I presume this is just done for readability in the docs, but it's a little misleading. Correctly, it would be: over18Names, err := From(students). // <--- Notice the trailing . to prevent ; Where(func (s T) (bool,error){ return s.(*Student).age >= 18, nil }). Select(func (s T) (T,error){ return s.(*Student).name, nil }). Results() It's argued now and then that this is one of the reasons go isn't useful for chain expression libraries like this; it becomes hard to tell at a glance the difference between two consecutive statements and one chained long statement. (The other reason typically given is it breaks error return expectations, but whatever. I've never really understood what's so hard about simple having the .Final() or .Resolve() or .Results() statement at the end of a chain to get the return code and/or error details shrug)