6 ms·

Some thoughts. The people I know who do work in machine learning (Jon Kleinberg, Thorsten Joachims, and their students) work in straight-up C, not Haskell,

by terra_t 16y ago

Some thoughts. The people I know who do work in machine learning (Jon Kleinberg, Thorsten Joachims, and their students) work in straight-up C, not Haskell, as the author suggests. Most of the dominant methods used today, such as the SVM, boosting, and the dynamic programming methods used in genomics are (i) numerical more than symbolic and (ii) involve moving a LOT of data around, so you want control of your data structures. (Sometimes these guys compress in-memory pointers!) I looked long and hard at Scala and decided that I didn't like what I saw. One problem is that, based on the JVM, type erasure is a big "broken window" that seriously damages the Scala type system. You can use generics for ~years~ in Java and never notice the problems that type erasure causes, but I find that 2/3 of the designs that I specifically want an advanced type system for do not work in Scala because of type erasure. Another issue with Scala is that the interface with Java, particularly using collections, is awkward; people who want to work in the JVM but want concision might be happier with Groovy. Scala's Lispy-Lists are convenient for the pattern matching capabilities in Scala, but are much less efficient than the vector-based Lists that come with Java when your data gets big. As for oCaml, the real excitement is in F#. You get the wonderful oCaml language and all of its features, plus you get an interface to the C#/.NET world that's much nicer than Scala's interface to the JVM. You've got access to the whole .NET framework base class library, plus all sorts of stuff that's been written for .NET. I'd also put C# on the border between a "mainstream" and "advanced" language. C#'s generics implementation just added support for covariant and contravariant inheritance, which is one of the features that had me interested in Scala. C# has good lambdas, really neat stuff in LINQ, and "expression trees" which offer metaprogramming capabilities above and beyond any static language I've seen. Note that there is a good open source implementation of the .NET framework, Mono, so you can develop in C# and F# and target Linux, MacOS and other platforms.

Artemidoros 16y ago
Note that there is a good open source implementation of the .NET framework, Mono, so you can develop in C# and F# and target Linux, MacOS and other platforms. Is Mono actually usable with F#? The last time I checked (2 months ago) the F# plugin for MonoDevelop didn't work and the questions on how to getting it to work (by various people) went unanswered by the developer. The runtime performance of F# code was shaky at best, the REPL had (on OS X at least) startup times to make it practically useless and without TCO the usefulness of F# really took a hit. Not wanting to bash on Mono or F# (which is my favorite language), but writing production software with this combination is currently not something I would recommend.
- jopamer 16y ago
  [Disclaimer: I'm a compiler dev working on F# at MSFT.] Performance and user experience on Mono are something we're definitely trying to improve - we spent a fair amount of time trying to make our Mono story better for our 2.0 release, and plan on investing more in this area as we move forward. We can't be everywhere at once, so your feedback is really important here - If you're having problems with our tools on Mono, please let us know via Microsoft Connect or fsbugs@microsoft.com. Also, don't be bashful about filing Mono bugs. So far, they've been really great about responding to any issues that have cropped up. Thanks!
  - Artemidoros 16y ago
    Thats awesome! Thanks for sharing. I'll give Mono/F# another spin in the next week and will report problems as suggested by you. Again thanks.
- vigalchin 16y ago
  Hello, On http://github.com/vasili/FSharpBinding is an FSharpBinding that works with MonoDevelop. Please read the README on this web site. I have tested new F# projects and several existing F# projects like Mandelbrot set, Stephen Wolfram's Rule 30, etc. This binding still needs polishing .... working on that with other colleagues. Kind regards, Vasili I. Galchin
- vigalchin 16y ago
  On http://github.com/vasili/FSharpBinding http://github.com/vasili/FSharpBinding is an FSharpBinding that works with MonoDevelop. Please read the README to under- stand current functionality. I have tested with new F# projects plus existing F# projects like Mandelbrot sets, Stephen Wolfram's Rule 30, etc. There is still a fair amount of polishing that needs to be done. That is a work-in-progress. Kind regards, Vasili I. Galchin
jorgeortiz85 16y ago
Starting with version 2.7.2, Scala added an experimental feature (which will no longer be experimental as of 2.8.0) called Manifests that allow you to selectively reify erased types. See this blog post for more: http://www.scala-blogs.org/2008/10/manifests-reified-types.html http://www.scala-blogs.org/2008/10/manifests-reified-types.h... This, combined with specialization (compiling primitive-specialized versions of code with generic types to improve performance of, for example, using function types on primitive collections) will make it easier to write performant, numeric code in Scala than in any other JVM language (up to the limits of the JVM, of course). The NLP communities at Stanford and Berkeley, which do a fair amount of machine learning, have a long history of working in Java (http://nlp.stanford.edu/software/ http://nlp.stanford.edu/software/, http://nlp.cs.berkeley.edu/Main.html#Software http://nlp.cs.berkeley.edu/Main.html#Software) and have recently started using Scala as well (http://www.scalanlp.org/ http://www.scalanlp.org/). Scala's interface with Java collections IS a bit clunky. Thankfully, this is a library issue, not a language issue, and can be solved with better libraries. See http://github.com/jorgeortiz85/scala-javautils http://github.com/jorgeortiz85/scala-javautils (for Scala 2.7.x) and http://github.com/scalaj/scalaj-collection http://github.com/scalaj/scalaj-collection (for Scala 2.8.x).
sid0 16y ago
Yeah, it's clear that more than a bit of PL research has fed back into C# as a language and .NET as a whole, in stark contrast to Java/JVM.
sreque 16y ago
Scala 2.8 has manifests, which reify generic types. What do you wish you could do with reified types that you can't do with manifests? Also, in Scala 2.7 I wrote my own library to wrap Java collections that didn't take much time or effort and allowed me to seamlessly pass scala collections to java code and use java collections the same as scala ones. Scala 2.8 is supposed to have such a library built in so you don't have to write it. Lastly, your comparison between Scala's functional lists and Java's ArrayList is pointless. Scala has an ArrayBuffer class too, so you can choose between either as appropriate. You can also pattern match against ArrayBuffer as well.
crux_ 16y ago
> Another issue with Scala is that the interface with Java, particularly using collections, is awkward; [...] Really? For me one of Scala's best attributes, by far, is the ability to use its implicit conversions to make awkward Java APIs smooth like butter. > One problem is that, based on the JVM, type erasure is a big "broken window" that seriously damages the Scala type system. I'm not sure I understand or agree. Understand: If you don't use casting, Scala's type system seems just as strong as ML, albeit not as good at inference. (F# has the exact same inference weakness as soon as you start using OO and inheritance.) Agree: having the ability to consciously drop into a dynamically typed/runtime-casted world for small bits of code is quite nice (true for F# too). And I've yet to see a place where erasure (a) caused, even in theory, a runtime type error, or (b) lost any type information within the scala language at compile or link time (which is where the type checking lives). I think it's a red herring / mere implementation detail. > As for oCaml, the real excitement is in F#. You get the wonderful oCaml language and all of its features, plus you get an interface to the C#/.NET world that's much nicer than Scala's interface to the JVM. Counterpoint: I've written production code in both Scala and F#; give me Scala any day of the week. I really wish its .NET support was more mature; I'd ditch it in a heartbeat for Scala if I could.
- Artemidoros 16y ago
  Using implicits is really nice to clean up the interface to some Java APIs (especially for APIs making heavy usage of anonymous inner objects) but I experienced more ugly boilerplate than I would like when having to use Java collection classes from Scala, too. To be more specific, when I want to transform a collection given by a Java library in a functional manner, I usually ended up with one or two localized import statements - to avoid e.g. scala.collection.mutable leaking into my other methods - and additional calls to convert the Java collection into a Scala specific and back again. This doesn't look like a big issue, but it increased the line count for a lot of cases from '1' to 3-5 ... which puts it uncomfortably close to 'dumb for loop territory'. Admittedly, this is a small price to pay compared to the constant annoyance that is Java, but for my part I fear that there might be a slight tendency in Scala to fix interesting problems preferably to useful ones. ... lost any type information within the scala language at compile or link time... How about this use case? match { case x: List[Foo] => ... case x: List[Bar] => ... } give me Scala any day of the week Why do you have the preference for Scala? I'm curious because Scala is my language of choice for my personal projects, but I would ditch it the second a credible F# derivative would appear on the JVM (F#'s stronger emphasis on functional programming, cleaner syntax and abstaining from 'fixing typing' in OOP are the reasons for my preferences).
10ren 16y ago
I was curious if F# is really that close to ocaml, and it seems they are related: http://stackoverflow.com/questions/179492/f-and-ocaml http://stackoverflow.com/questions/179492/f-and-ocaml with some differences (eg. F# doesn't seem to have those crazy +. *. etc float operators for type inference). Here's a ridicuously detailed table comparing F#, ocaml, haskell, scala, ML (ie. everything in the article but scheme), at the aptly named http://hyperpolyglot.wikidot.com/ml http://hyperpolyglot.wikidot.com/ml
derefr 16y ago
> The people I know who do work in machine learning work in straight-up C, not Haskell, as the author suggests. Is their work in applying ML theory, or creating more of it? ML almost always involves large datasets in practical use, but when you're trying to come up with a new method for ensuring correctness in some edge-case, you would be testing it with hypothetical extreme inputs, not feeding it mounds of data that won't trigger the problem. Haskell works well to model computer science research problems—that is, research into CS, not using CS to research other domains. Once your model has "solidified", you should by all means optimize it in C before running it on real-world data.

by terra_t 16y ago

Artemidoros 16y ago

Note that there is a good open source implementation of the .NET framework, Mono, so you can develop in C# and F# and target Linux, MacOS and other platforms. Is Mono actually usable with F#? The last time I checked (2 months ago) the F# plugin for MonoDevelop didn't work and the questions on how to getting it to work (by various people) went unanswered by the developer. The runtime performance of F# code was shaky at best, the REPL had (on OS X at least) startup times to make it practically useless and without TCO the usefulness of F# really took a hit. Not wanting to bash on Mono or F# (which is my favorite language), but writing production software with this combination is currently not something I would recommend.

jopamer 16y ago
[Disclaimer: I'm a compiler dev working on F# at MSFT.] Performance and user experience on Mono are something we're definitely trying to improve - we spent a fair amount of time trying to make our Mono story better for our 2.0 release, and plan on investing more in this area as we move forward. We can't be everywhere at once, so your feedback is really important here - If you're having problems with our tools on Mono, please let us know via Microsoft Connect or fsbugs@microsoft.com. Also, don't be bashful about filing Mono bugs. So far, they've been really great about responding to any issues that have cropped up. Thanks!
- Artemidoros 16y ago
  Thats awesome! Thanks for sharing. I'll give Mono/F# another spin in the next week and will report problems as suggested by you. Again thanks.
vigalchin 16y ago
Hello, On http://github.com/vasili/FSharpBinding is an FSharpBinding that works with MonoDevelop. Please read the README on this web site. I have tested new F# projects and several existing F# projects like Mandelbrot set, Stephen Wolfram's Rule 30, etc. This binding still needs polishing .... working on that with other colleagues. Kind regards, Vasili I. Galchin
vigalchin 16y ago
On http://github.com/vasili/FSharpBinding http://github.com/vasili/FSharpBinding is an FSharpBinding that works with MonoDevelop. Please read the README to under- stand current functionality. I have tested with new F# projects plus existing F# projects like Mandelbrot sets, Stephen Wolfram's Rule 30, etc. There is still a fair amount of polishing that needs to be done. That is a work-in-progress. Kind regards, Vasili I. Galchin

jorgeortiz85 16y ago

Starting with version 2.7.2, Scala added an experimental feature (which will no longer be experimental as of 2.8.0) called Manifests that allow you to selectively reify erased types. See this blog post for more: http://www.scala-blogs.org/2008/10/manifests-reified-types.html http://www.scala-blogs.org/2008/10/manifests-reified-types.h... This, combined with specialization (compiling primitive-specialized versions of code with generic types to improve performance of, for example, using function types on primitive collections) will make it easier to write performant, numeric code in Scala than in any other JVM language (up to the limits of the JVM, of course). The NLP communities at Stanford and Berkeley, which do a fair amount of machine learning, have a long history of working in Java (http://nlp.stanford.edu/software/ http://nlp.stanford.edu/software/, http://nlp.cs.berkeley.edu/Main.html#Software http://nlp.cs.berkeley.edu/Main.html#Software) and have recently started using Scala as well (http://www.scalanlp.org/ http://www.scalanlp.org/). Scala's interface with Java collections IS a bit clunky. Thankfully, this is a library issue, not a language issue, and can be solved with better libraries. See http://github.com/jorgeortiz85/scala-javautils http://github.com/jorgeortiz85/scala-javautils (for Scala 2.7.x) and http://github.com/scalaj/scalaj-collection http://github.com/scalaj/scalaj-collection (for Scala 2.8.x).

sid0 16y ago

Yeah, it's clear that more than a bit of PL research has fed back into C# as a language and .NET as a whole, in stark contrast to Java/JVM.

sreque 16y ago

Scala 2.8 has manifests, which reify generic types. What do you wish you could do with reified types that you can't do with manifests? Also, in Scala 2.7 I wrote my own library to wrap Java collections that didn't take much time or effort and allowed me to seamlessly pass scala collections to java code and use java collections the same as scala ones. Scala 2.8 is supposed to have such a library built in so you don't have to write it. Lastly, your comparison between Scala's functional lists and Java's ArrayList is pointless. Scala has an ArrayBuffer class too, so you can choose between either as appropriate. You can also pattern match against ArrayBuffer as well.

crux_ 16y ago

> Another issue with Scala is that the interface with Java, particularly using collections, is awkward; [...] Really? For me one of Scala's best attributes, by far, is the ability to use its implicit conversions to make awkward Java APIs smooth like butter. > One problem is that, based on the JVM, type erasure is a big "broken window" that seriously damages the Scala type system. I'm not sure I understand or agree. Understand: If you don't use casting, Scala's type system seems just as strong as ML, albeit not as good at inference. (F# has the exact same inference weakness as soon as you start using OO and inheritance.) Agree: having the ability to consciously drop into a dynamically typed/runtime-casted world for small bits of code is quite nice (true for F# too). And I've yet to see a place where erasure (a) caused, even in theory, a runtime type error, or (b) lost any type information within the scala language at compile or link time (which is where the type checking lives). I think it's a red herring / mere implementation detail. > As for oCaml, the real excitement is in F#. You get the wonderful oCaml language and all of its features, plus you get an interface to the C#/.NET world that's much nicer than Scala's interface to the JVM. Counterpoint: I've written production code in both Scala and F#; give me Scala any day of the week. I really wish its .NET support was more mature; I'd ditch it in a heartbeat for Scala if I could.

Artemidoros 16y ago
Using implicits is really nice to clean up the interface to some Java APIs (especially for APIs making heavy usage of anonymous inner objects) but I experienced more ugly boilerplate than I would like when having to use Java collection classes from Scala, too. To be more specific, when I want to transform a collection given by a Java library in a functional manner, I usually ended up with one or two localized import statements - to avoid e.g. scala.collection.mutable leaking into my other methods - and additional calls to convert the Java collection into a Scala specific and back again. This doesn't look like a big issue, but it increased the line count for a lot of cases from '1' to 3-5 ... which puts it uncomfortably close to 'dumb for loop territory'. Admittedly, this is a small price to pay compared to the constant annoyance that is Java, but for my part I fear that there might be a slight tendency in Scala to fix interesting problems preferably to useful ones. ... lost any type information within the scala language at compile or link time... How about this use case? match { case x: List[Foo] => ... case x: List[Bar] => ... } give me Scala any day of the week Why do you have the preference for Scala? I'm curious because Scala is my language of choice for my personal projects, but I would ditch it the second a credible F# derivative would appear on the JVM (F#'s stronger emphasis on functional programming, cleaner syntax and abstaining from 'fixing typing' in OOP are the reasons for my preferences).

10ren 16y ago

I was curious if F# is really that close to ocaml, and it seems they are related: http://stackoverflow.com/questions/179492/f-and-ocaml http://stackoverflow.com/questions/179492/f-and-ocaml with some differences (eg. F# doesn't seem to have those crazy +. *. etc float operators for type inference). Here's a ridicuously detailed table comparing F#, ocaml, haskell, scala, ML (ie. everything in the article but scheme), at the aptly named http://hyperpolyglot.wikidot.com/ml http://hyperpolyglot.wikidot.com/ml

derefr 16y ago

> The people I know who do work in machine learning work in straight-up C, not Haskell, as the author suggests. Is their work in applying ML theory, or creating more of it? ML almost always involves large datasets in practical use, but when you're trying to come up with a new method for ensuring correctness in some edge-case, you would be testing it with hypothetical extreme inputs, not feeding it mounds of data that won't trigger the problem. Haskell works well to model computer science research problems—that is, research into CS, not using CS to research other domains. Once your model has "solidified", you should by all means optimize it in C before running it on real-world data.