19 ms·
Announcing .NET Jupyter Notebooks
- Lucasoato 7y agoJupyter is such a cool tool, not only for data scientists! Give it a try, guys :D
- sgillen 7y agoThis is great news. Jupyter has become my default tool for prototyping code. I keep trying other platforms that should theoretically have the same features, but I just find Jupyter much more pleasant to use.
- reportgunner 7y agoHow do you prevent jupyter getting lost in a sea of browser tabs ? Do you use it in a separate one ?
- sgillen 7y agoExactly, one window for Jupyter and sometimes Tensorboard, one for normal browsing.
- zwieback 7y agoEven though I love Python I don't really like Jupyter, seems like with a dev environment like VS or VSCode I can't quite find the right use case. With C# it might be a different story, since the development cycle is fundamentally edit-compile-run an environment like Jupyter might give me an extra platform for in-between testing. I already use the Interactive built into VS quite a bit to figure out, for example, just the right format string for Datetime.Now.ToString() or stuff like that.
- Fr0styMatt88 7y agoThe thing that I find really daunting with Jupyter is the set of APIs that it seems I need to learn in order to get it to do anything cool, like for example visualizing data or a function, or making an interactive widget that does the same. I don’t remember the same kind of pain when playing around with Mathematica. I really want to like it and learn it — I really like interactive programming. It just never makes it high enough on my brain space list to seem worth it. Since C# is my daily language, this is pretty cool though. I just really wish that I had a use-case strong enough for it to overcome the initial pains of learning it and the ongoing mental maintenance of staying usefully fluent in yet another set of libraries.
- EForEndeavour 7y agoAgreed on the bewildering number of disparate APIs that you need to bash together in a Jupyter notebook. Data wrangling, plotting, machine-learning, logging, writing to file/DB... It doesn't help that there are often multiple competing packages that solve basically the same problem in different ways, ensuring that whatever choice you make will be fraught with anxiety about not "doing it right." That being said, I've found the only way I can get interactive programming to stick in my mind is to pick a sample dataset, wonder some open-ended questions about it, and then answer those questions in a Jupyter notebook with the help of Google, documenting your thought process as you go. A great starting point to find real-world sample data is the UCI dataset repository: https://archive.ics.uci.edu/ml/datasets.php https://archive.ics.uci.edu/ml/datasets.php
- bransonf 7y agoThe benefit of Jupyter notebooks really is about being able to share chunks of code and write narrative around it. It isn’t really aiming to replace a dev environment like VSCode. We’re seeing a rise of notebooks in particular because they encourage writing a lot of plain text around your code, and that’s really good both for teaching, and promoting good documentation skills for learners.
- hos234 7y ago>share chunks of code and write narrative around it. >encourage writing a lot of plain text around your code Is this targeted at non techies? Teachers? Can someone provide some good example of this? Not having used it much trying to understand what the advantages are over good quality documentation/tutorial pages on say msdn.
- someguy101010 7y agoThis explains the advantages very well imo https://en.wikipedia.org/wiki/Literate_programming#Contrast_with_documentation_generation https://en.wikipedia.org/wiki/Literate_programming#Contrast_...
- nek4life 7y agoIt solves the problem of scientists that have done experiments, but have no way to share and have their experiments and have them validated using the same procedures and programs. At least that's the way I understand it. You can even have a notebook run in a cluster. http://ipython.org/ipython-doc/dev/parallel/ http://ipython.org/ipython-doc/dev/parallel/
- Avalaxy 7y ago> You can even have a notebook run in a cluster. Well, that's exactly what Databricks does.
- unoti 7y ago> Can someone provide some good example of this? Not having used it much trying to understand what the advantages are... Sure! Here are some ways I use notebooks: Imagine any scenario in which you'd rather make a video to demonstrate how something works. For a lot of those cases, a notebook is a great way to show how the code works and how it all comes together. Another set of use cases is you've done some research, need to call a few api's, post process the results, and share that with your team or have it ready for later to do something similar. You could think of this as a "Super Postman". For example, you need to run some api's in a loop, filter the results, and accumulate some things and print totals at the end. Another use case is you're troubleshooting an issue, and want to keep notes of what you've tried, what the results were, and be ready to go back and run things again.
- jaked89 7y agoC# has supported interactive scripting for years now since Roslyn came about. Even VS has a C# Interactive window.
- noblethrasher 7y agoActually, C# had a REPL even before Roslyn: The Mono project shipped one about 4 years prior.
- pjmlp 7y agoC# had a pseudo REPL already back then, the immediate window available during debugging sessions.
- noblethrasher 7y agoTrue, but what made the Mono REPL particularly remarkable was that it launched with much of the functionality of a notebook, with things like inline graphics.
- nomel 7y agoMy use case is algorithm prototyping/development. It lets me mostly put the code aside and concentrate and quickly iterate on the algorithm. It's just a fancy, persistent, reproducible, REPL.
- whb07 7y agoTry F# feels like Python in most regards but ends up having type inference. Edit: if you think “oh that’s nice, just like Java/C++”, HECK NO. It goes beyond this, where complex modeling can easily be represented and things like pattern matching and other features not known in other langs
- Rapzid 7y agoI really wish F# were the dominant language in the AI glue space instead of Python :| Typed array/tensor shapes? Yes please.
- whb07 7y agoI was thinking of writing the bindings to Pytorch directly to C++ for F# now that would be the killer thing to bring on to .net . It’d be like writing As close as writing Python but order of magnitude improvement for performance. But that’s a lot of work and I don’t do much ml work lol so meh Someone should do it
- pjmlp 7y agoWith the caveat of being Windows only, you could have a look into C++/CLI or C++/WinRT.
- phillipcarter 7y agoHello! We're taking a look at improving these kinds of programming scenarios in F# for the next version of the language. Since F# is first-class in the jupyter experience, one of those hurdles is already done! That said, improvements to the language can only go so far. We're trying to figure out what a good set of libraries for the whole range of ML tasks looks like for F# and .NET.
- tasogare 7y agoContinue the good work :) I use mostly C# out of habit but F# is my favorite in terms of expressiveness. I'm planning a series of educational videos in French about the language because it doesn't have the exposure it merits.
- jcmontx 7y ago> I already use the Interactive built into VS quite a bit to figure out, for example, just the right format string for Datetime.Now.ToString() or stuff like that. This is me_irl
- rb808 7y agoI set up Jupyter for my wider team to use, I thought it would be popular but nearly everyone chose to use PyCharm instead.
- jpalomaki 7y agoThere are approaches to combine the two. See for example this article about Jupyter and vscode [1]. And to ”close the loop”, you can of course access vscode using browser with Visual Studio Online [2]. [1] https://code.visualstudio.com/docs/python/jupyter-support https://code.visualstudio.com/docs/python/jupyter-support [2] https://visualstudio.microsoft.com/services/visual-studio-online/ https://visualstudio.microsoft.com/services/visual-studio-on...
- baq 7y agowhat I end up doing is having both the dev env and jupyter open at the same time - doing dev the normal way and then using a jupyter notebook for q&d manual validation and plots. like others said already, jupyter isn't a development environment and trying to use it like one is asking for trouble, but it is a great REPL on some awesome steroids.
- Avalaxy 7y agoI love .NET and I love Jupyter. I don't know how well they will combine though. I feel like the lack of Pandas and flexible typing of Python will make it a lot less useful.
- cm2187 7y agoType inference should do most of the work
- teh_klev 7y agoAnd dynamic types.
- mumblemumble 7y agoI haven't tried it, but what I'm envisioning is LinqPad with cells, including Markdown cells. And that sounds like a nice tool to me. Even better if it works well with Deedle and FSharp.Charting. But I'm not going to demand everything on release day.
- jimmcslim 7y agoI feel like Deedle is dead unfortunately, although I see there has been a bit of commit action recently. There are a couple of efforts to build .NET wrappers around Pandas, or to reimplement Pandas in .NET; see http://scisharp.github.io http://scisharp.github.io
- phillipcarter 7y agoHello! Deedle is actively maintained, as is the XPlot library and a few others under the fslaborg organization on GitHub. However, it's not without its flaws and addressing them in the best way possible would be a breaking change. Additionally, the .NET team is working on a core DataFrame API that could service these needs here.
- zyzhu 7y agoHello. I am the current maintainer of Deedle. Recently, I have been focused on use cases of MathNet interop. Deedle.Math was released last week. pandas-like matrix dot functions are available now. Lots of linear algebra, stats and finance related functions are added too. More finance domain functions and docs improvements are on my to-do list. I am really excited about the new Data Frame from corefxlab to complement what Deedle lacks. Since Deedle's original core engineering team has left and moved on, some core technical debt are beyond my capability to address. But Deedle is still the best data frame library in .Net ecosystem for now if you are able to get around some of its odd edges.
- nick_ 7y agoIntegration of ML.NET and https://github.com/SciSharp https://github.com/SciSharp would be amazing!
- 52-6F-62 7y agoProbably a good time to mention that for the JS-minded there is Bostock’s Observable Notebooks as well. I think the only real drawback is no self hosting. https://observablehq.com/ https://observablehq.com/ A pretty healthy example: https://observablehq.com/@rreusser/2d-n-body-gravity-with-poissons-equation https://observablehq.com/@rreusser/2d-n-body-gravity-with-po...
- alexcnwy 7y agoMike Bostock is a beast - this d3.js example from 2013 still blows my mind: https://archive.nytimes.com/www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-proposal-graphic.html https://archive.nytimes.com/www.nytimes.com/interactive/2012... I haven’t gotten properly stuck into observable - in what ways is it better than using a jupyter notebook?
- 52-6F-62 7y agoI don't know about "better" as I don't have a lot of experience with either—I've only dabbled or used them to prototype something small. The major differentiator is the JS implementation and ability to import libraries from sibling notebooks and other sources. But yeah I'd have to agree that Bostock is a beast.
- merlincorey 7y ago> and ability to import libraries from sibling notebooks The Jupyter documentation includes an example for importing other notebooks here: https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Importing%20Notebooks.html https://jupyter-notebook.readthedocs.io/en/stable/examples/N...
- lmeyerov 7y agoI like Observable for sharing JS tutorials - was just updating https://observablehq.com/@lmeyerov/manipulating-flat-arrays-arrow-style https://observablehq.com/@lmeyerov/manipulating-flat-arrays-... today. Observable does some things quite well for that use case: -- Easier dependencies: no need for 'npm install', just 'require(...)' -- URL publishing -- Collaborative merge flow However, Jupyter made some good decisions that make it win over Observable for our day-to-day data work: -- Manual reexecution vs. automatic: when working with big data, outside APIs, etc., Observable's automatic reexecution is a non-starter -- Fully open source, embeddable, and successful history of non-vc funding: Jupyter is organized and provided in a way companies ( who aren't Amazon ;-)) can rally around, evidenced by contributions by Bloomberg etc. -- Access to underlying unix/windows env and multiple environments (multiple Python versions, ...). As much as I wish Observable's choice of JS could provide a viable data environment, and I've personally invested in making it so and the path to making it first-class is clear, we need way more gov/google/nvidia/etc. support. Ultimately, for not-too-sensitive collaborative data work, we use Google Colab, then offline and sensitive commercial work via Jupyter (Graphistry ships with it preloaded for GPU dataframe & GPU visual graph analytics goodness!), and if we did more in JS tutorial land, Observable would be my first choice.
- alkonaut 7y agoI’d love to see some nontrivial side by side examples of F# notebooks compared to python.
- phillipcarter 7y agoHey there, I work on F# at Microsoft and helped put together the initial set of samples for F#. You can access them here: https://mybinder.org/v2/gh/dotnet/try/master?urlpath=lab https://mybinder.org/v2/gh/dotnet/try/master?urlpath=lab What kinds of nontrivial samples would you be interested in seeing?
- fulafel 7y agoNot the OP but I enjoyed the "Introduction to F#" notebook there. Interactivity let me see what the error messages are like for eaxmple replacing "negate" with "42" in the pipeline example. (hint to Jupyter newbies: alt-enter re-evaluates the code block after editing it)
- LandR 7y agoOn the introduction to F# sample, there is a comment The best way to work with DUs is pattern matching. Using the previously-defined type definitions, we can model withdrawing money from an account: Then a sample that has nothing to do with withdrawing money from an account.
- phillipcarter 7y agoThanks, fixing that here: https://github.com/dotnet/try/pull/638 https://github.com/dotnet/try/pull/638
- ngcc_hk 7y agoNot my cup of tea as I am more multi platform. Windows only I suppose. F# interesting but still.
- herrschnapps 7y ago.net core is cross platform, so this should work on Linux and Mac.
- npo9 7y agoNot only that, .net core is now essentially the only active branch of .net. The windows only .net framework is becoming legacy.
- pjmlp 7y agoKind of. .NET Framework will be the XP of .NET until Microsoft provides an actual migration path to many of the APIs and frameworks that haven't made it into .NET Core. Plus some of those APIs that made the cut into .NET Core, like the UI ones, are Windows only.
- bonesss 7y agoThere are also some critical tooling gaps that mean .Net Core is still broken for scenarios that worked a decade ago (FSI package management with F#, specifically).
- phillipcarter 7y agoFSI has never had package management. Are you referring to something else?
- bonesss 7y agoSorry for the loose term. Not package management a la NuGet, but how packages and their references are handled (ie managed), so that they're not available and operative in FSI on .Net Core in VS. Things that work when compiled, worked about a decade ago, but die on FSI preventing multiple data interaction & scripting scenarios. Ref: https://github.com/dotnet/fsharp/issues/3309 https://github.com/dotnet/fsharp/issues/3309 , https://github.com/dotnet/fsharp/pull/5850 https://github.com/dotnet/fsharp/pull/5850 , https://github.com/fsprojects/IfSharp/issues/206 https://github.com/fsprojects/IfSharp/issues/206 , etc etc Specifically, referencing a library that uses an FSharp.Data type provider, and calling that library in FSI in Visual Studio. Updated to the latest and greatest, 5 y.o. code looks like this when we execute it: > System.MissingMethodException: Method not found: 'System.String FSharp.Data.Http.RequestString(System.String ....
- pietroppeter 7y agoWhat about vb.net?
- codeape 7y agoAre there any plans for implementing code completion/intellisense?
- jonsequitur 7y agoYes, absolutely.
- growlist 7y agoNow if only .NET had arguably the world's greatest IDE!