5 ms·

Why .NET isn't used for data sciences? Does it have numpy alternative?

InGodsName 8y ago
I don't know why would anyone downvote honest question. Damn you hn!
ankitasthana 8y ago
The base ML.NET framework comes with a variety of numpy alternatives, datatypes for heterogenous (IDataView) and homogenous (VBuffer) data, a variety of transforms for data pre-processing, feature extraction (e.g. OneHotCodeEncoding), Missing values, normalization etc. handy for common ML scenarios like classification, regression, recommendation etc. you can find the full list here. Types:https://github.com/dotnet/machinelearning/blob/master/docs/code/IDataViewDesignPrinciples.md https://github.com/dotnet/machinelearning/blob/master/docs/c... Transforms: https://docs.microsoft.com/en-us/dotnet/machine-learning/resources/transforms https://docs.microsoft.com/en-us/dotnet/machine-learning/res...
amhenk 8y ago
I think it's because there aren't as many mature libraries for data scientist to use. That and prototyping isn't as friendly as it is in something like Python. In terms of things similar to numpy there's Accord.NET which is a very solid library but as far as I know, it's really the only mature/common one used. Another thing is that .NET more or less focuses on traditional software development which I think can be considered cumbersome to work with if you're trying to test stuff quickly. Little disclaimer, I'm not a data scientist but these are things I've noticed while deving with .NET so I'm not sure how much they actually apply to the day to day for a data scientist.
- Rapzid 8y ago
  Prototyping in Python for a large subset of people is NOT friendly. To quote a famous book or something; "My God, it's full of shapes!". Shapes whose shape matter, just not so much to the language and tooling you interface with them from. It's like trying to drive a 3d world from a 2d world :| IMHO F# has the potential to be WAY more friendly for prototyping and more. It's all a matter of the libs.
  - avinium 8y ago
    > IMHO F# has the potential to be WAY more friendly for prototyping and more. It's all a matter of the libs. Yeah, I've been dropping down to Visual Studio's F# console regularly to test things in a REPL-like environment. Nowhere near the convenience of IPython though - I haven't come across anything way to import assemblies other than "#r /some/path/to/assembly.dll", which requires me to manually navigate through my project directories to find the correct folder/assembly. A notebook environment inside VS would be fantastic, too - rather than having a single line terminal-style input.
    - smoothdeveloper 8y ago
      https://fsprojects.github.io/Paket/paket-generate-load-scripts.html https://fsprojects.github.io/Paket/paket-generate-load-scrip... is probably what you need if you enjoy using fsi and easily reference external libraries from script files.
nickspag 8y ago
I started ML work in python coming from a .NET background. To echo one of the other comments here- I believe one main reason is due to the iterative nature of data science. You have to sort of re-build completely every time you make a change in .NET. There isn't much of a notebook concept in .NET- a running engine you can query with additional commands/cells/etc, which is beneficial. As well, real-world data feels easier to work with in a more forgiving environment (non-static typing, etc) like Python. However ML.NET is a cool endeavor and as the .NET data prep libraries get more mature we may find some more production benefits from the very typing/compiling system that, while making it difficult to iterate in, provide more stability in the wild.
- tluyben2 8y ago
  A proper repl seems the first step to start building such interactive/iterative tooling; F# has one so maybe that is a better start than C# for this? C#.NET Core suffers (as far as I know) from this; https://stackoverflow.com/questions/47394231/csharpscript-uses-incompatible-internal-type-representations-at-every-external-c https://stackoverflow.com/questions/47394231/csharpscript-us... I think Mono has a solution, but as far as I know .NET Core basically is unusable for scripting because of this for C# which makes the kind of scenario you want basically impossible from the get go. It is a bit weird not more attention is given to it given so many people ache for an interactive C# environment, especially for this purpose, but also for faster iteration with Xamarin on iOS/Android etc (all of my colleagues would save a lot of time on development with this for Xamarin mobile dev especially). If that has been fixed, there is still the lack of libraries but that and the rest of what is needed is at least not basically impossible to solve by mere mortals.
  - pjmlp 8y ago
    Visual Studio has a C# REPL sice version 2015. Then there are Xamarin Workbooks, like Swift playgrounds but for .NET.
    - tluyben2 8y ago
      That's not the same thing though; to be able to actually REPL a complete app or stick in an interpreter where you need it. I tried workbooks and it's more a toy (like Swift playgrounds by the way). It helps for sure, but when we are working on bigger existing apps, we want to plug in REPLs in particular places and iterate. Or generally iterate; both do not fit with the VS REPL or Workbooks.
n-gatedotcom 8y ago
In enterprises, its used a lot. A lot of HN is a niche so don't get your used/not used stats from here
yenwel 8y ago
Check out fslab libraries. Deedle is what you want I think. https://fslab.org/Deedle/ https://fslab.org/Deedle/ Also: (Diffsharp is AD like pytorch) https://fsharp.org/guides/math-and-statistics/ https://fsharp.org/guides/math-and-statistics/

I don't know why would anyone downvote honest question. Damn you hn!

The base ML.NET framework comes with a variety of numpy alternatives, datatypes for heterogenous (IDataView) and homogenous (VBuffer) data, a variety of transforms for data pre-processing, feature extraction (e.g. OneHotCodeEncoding), Missing values, normalization etc. handy for common ML scenarios like classification, regression, recommendation etc. you can find the full list here. Types:https://github.com/dotnet/machinelearning/blob/master/docs/code/IDataViewDesignPrinciples.md https://github.com/dotnet/machinelearning/blob/master/docs/c... Transforms: https://docs.microsoft.com/en-us/dotnet/machine-learning/resources/transforms https://docs.microsoft.com/en-us/dotnet/machine-learning/res...

amhenk 8y ago

I think it's because there aren't as many mature libraries for data scientist to use. That and prototyping isn't as friendly as it is in something like Python. In terms of things similar to numpy there's Accord.NET which is a very solid library but as far as I know, it's really the only mature/common one used. Another thing is that .NET more or less focuses on traditional software development which I think can be considered cumbersome to work with if you're trying to test stuff quickly. Little disclaimer, I'm not a data scientist but these are things I've noticed while deving with .NET so I'm not sure how much they actually apply to the day to day for a data scientist.

Rapzid 8y ago
Prototyping in Python for a large subset of people is NOT friendly. To quote a famous book or something; "My God, it's full of shapes!". Shapes whose shape matter, just not so much to the language and tooling you interface with them from. It's like trying to drive a 3d world from a 2d world :| IMHO F# has the potential to be WAY more friendly for prototyping and more. It's all a matter of the libs.
- avinium 8y ago
  > IMHO F# has the potential to be WAY more friendly for prototyping and more. It's all a matter of the libs. Yeah, I've been dropping down to Visual Studio's F# console regularly to test things in a REPL-like environment. Nowhere near the convenience of IPython though - I haven't come across anything way to import assemblies other than "#r /some/path/to/assembly.dll", which requires me to manually navigate through my project directories to find the correct folder/assembly. A notebook environment inside VS would be fantastic, too - rather than having a single line terminal-style input.
  - smoothdeveloper 8y ago
    https://fsprojects.github.io/Paket/paket-generate-load-scripts.html https://fsprojects.github.io/Paket/paket-generate-load-scrip... is probably what you need if you enjoy using fsi and easily reference external libraries from script files.

nickspag 8y ago

I started ML work in python coming from a .NET background. To echo one of the other comments here- I believe one main reason is due to the iterative nature of data science. You have to sort of re-build completely every time you make a change in .NET. There isn't much of a notebook concept in .NET- a running engine you can query with additional commands/cells/etc, which is beneficial. As well, real-world data feels easier to work with in a more forgiving environment (non-static typing, etc) like Python. However ML.NET is a cool endeavor and as the .NET data prep libraries get more mature we may find some more production benefits from the very typing/compiling system that, while making it difficult to iterate in, provide more stability in the wild.

tluyben2 8y ago
A proper repl seems the first step to start building such interactive/iterative tooling; F# has one so maybe that is a better start than C# for this? C#.NET Core suffers (as far as I know) from this; https://stackoverflow.com/questions/47394231/csharpscript-uses-incompatible-internal-type-representations-at-every-external-c https://stackoverflow.com/questions/47394231/csharpscript-us... I think Mono has a solution, but as far as I know .NET Core basically is unusable for scripting because of this for C# which makes the kind of scenario you want basically impossible from the get go. It is a bit weird not more attention is given to it given so many people ache for an interactive C# environment, especially for this purpose, but also for faster iteration with Xamarin on iOS/Android etc (all of my colleagues would save a lot of time on development with this for Xamarin mobile dev especially). If that has been fixed, there is still the lack of libraries but that and the rest of what is needed is at least not basically impossible to solve by mere mortals.
- pjmlp 8y ago
  Visual Studio has a C# REPL sice version 2015. Then there are Xamarin Workbooks, like Swift playgrounds but for .NET.
  - tluyben2 8y ago
    That's not the same thing though; to be able to actually REPL a complete app or stick in an interpreter where you need it. I tried workbooks and it's more a toy (like Swift playgrounds by the way). It helps for sure, but when we are working on bigger existing apps, we want to plug in REPLs in particular places and iterate. Or generally iterate; both do not fit with the VS REPL or Workbooks.

n-gatedotcom 8y ago

In enterprises, its used a lot. A lot of HN is a niche so don't get your used/not used stats from here

yenwel 8y ago

Check out fslab libraries. Deedle is what you want I think. https://fslab.org/Deedle/ https://fslab.org/Deedle/ Also: (Diffsharp is AD like pytorch) https://fsharp.org/guides/math-and-statistics/ https://fsharp.org/guides/math-and-statistics/