5 ms·

I don’t want to repeat the old and tired JavaScript hate, but this just isn’t a great idea. I’d suggest that there are 3 important primitives for data science:

• RedShift1 6y ago
  There is decimal.js but yes it's not going to be fast.
• 11235813213455 6y ago
  I don't see JS as less powerful than Python for data science, it's faster than Python, or can use bindings just like Python. JS is maybe less commonly used than Python in data science nowadays, but I wouldn't be surprised if this changes in next years. There are equivalent libs like tensorflow-core, there are native features like BigInt, and there are libs for 64bits floats (decimal.js, big.js). I'd be glad to spend some time converting Scikit-learn into JS and also show you how expressive JS actually is, if you show me some Python code, I'll translate it
  • uryga 6y ago
    > libs for 64bits floats (decimal.js, big.js) both of those libraries are for arbitrary precision decimals, not floats.
    • __jem 6y ago
      If it's arbitrary precision, what's the difference, besides slightly more bookkeeping on your end?
  • danpalmer 6y ago
    The fact that the number support isn’t part of the language is Linda the problem though. When you’re writing data science code, the value is in the answer more than the process of getting to that answer. Anything that complicates that gets in the way. This is why things like Pandas are so popular despite having some questionable engineering. Using a library for big number support, having to get that to play nicely with other libraries, it all goes against the aims. Now for data engineering it’s very different. I wouldn’t choose JS myself, but it’s a much more reasonable choice. For engineering the process by which you get the answer matters far more - is it scalable, testable, repeatable, etc. Having to use a library for big number support is fine. It’s two very different ways of working and I’m still fairly convinced that JS is not conducive to the former.
  • tyingq 6y ago
    >it's faster than Python Is that generally true for data science type tasks, though, where the "fast" in python is really numpy, pandas, etc? >or can use bindings just like Python But there's not really anything like numpy/pandas for it to bind to at the moment, is there? Meaning anything as broad in functionality, fast, mature, etc.
• nosianu 6y ago
  > Numbers are 64bit floats only - no integers, no big numbers. That is not true. BigInt has been available for a bit already. MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... Availability: https://caniuse.com/bigint https://caniuse.com/bigint I don't want to argue for or against using JS for "data science" (I myself used R for that but I use JS a lot for other things), just a clarification on this one concrete claim.
  • mschuetz 6y ago
    > That is not true. BigInt has been available for a bit already. performance-wise, BigInts are terrible. Tried to use them, made things about a hundred times slower. What JS needs are 64 bit integer types, and some form of typing system that allows differentiating between various number types.
    • felixfbecker 6y ago
      Genuine question: I imagine most data science things involve arrays of numbers, not just single numbers. JS has UInt8Array, i.e. it does kinda have integers if you want them in an array anyway. Can that speed things up?
      • deleted 6y ago
        [deleted]
      • mschuetz 6y ago
        In my case, it did involved reading binary files from ArrayBuffers where some attributes were stored as int64. I've been using DataView to read BigInt64s and then used those frequently. Was terribly slow. I've then reverted to alternatives wherever possible, e.g. immediately converting the BigInt64s to Numbers/Doubles where 53 bit integers were sufficient, or splitting down to two 32 bit integers and do some more involved bit magic on them. Much faster, but overall I wasn't happy with the whole process and complexity. I'd rather have native 64bit integer values in JS.
    • 6y ago
• v8dev123 6y ago
  Well, nowadays you can use WASM with JS to access libraries at near native speed.
• RobinL 6y ago
  You can get a long way nowadays with Arquero[0] and Observable[1]. Arquero allows columnar based data storage and processing, with a grammar of data processing verbs similar to e.g. dplyr. Not as fast as vectorized computations in e.g. Python or R, but faster than has previously been possible. I'm not suggesting these are the first tools you'd reach for for data science in production, but I've found them extremely useful for prototyping, experimenting with algorithms, and visualization. I think it's got to the stage they should be seriously considered for some types of relatively simple data processing work due to their ease of deployment. [0]https://github.com/uwdata/arquero https://github.com/uwdata/arquero [1]https://observablehq.com/ https://observablehq.com/
• spion 6y ago
  Other than the fact we have BigInts now, we also have * tensorflowjs, which runs on GPUs https://www.tensorflow.org/js https://www.tensorflow.org/js and * danfo, which aims to be a pandas equivalent for JS: https://danfo.jsdata.org/ https://danfo.jsdata.org/ Given the powerful interactive visualisation capabilities available in JS, its only a matter of time until JS becomes a serious contender IMO.
  • mschuetz 6y ago
    > Other than the fact we have BigInts now performance-wise, BigInts are terrible. Tried to use them, made things about a hundred times slower.
    • spion 6y ago
      That's typical with most JS features, it takes some time for engine performance optimizations to catch up with them. In this particular case I suppose things are moving slower than expected, but with demand increasing prioritization will take place.
• slver 6y ago
  JavaScript has plenty of libraries covering the basics. Here a few: https://github.com/nicolaspanel/numjs https://github.com/nicolaspanel/numjs https://www.npmjs.com/package/fast-math https://www.npmjs.com/package/fast-math https://smartbear.com/de/blog/2013/four-serious-math-libraries-for-javascript/ https://smartbear.com/de/blog/2013/four-serious-math-librari... That's not the problem. The problem is mindshare and network effects. When analyzing why Python is used one way and JS another we're tempted to retroactively rationalize this with something fundamental about the language. There's nothing fundamental about it. It's just happenstance. Python was around longer as a general purpose script, and it filled that niche. JS is relatively new as a script outside the browser.
  • brylie 6y ago
    The first repo has one core contribitor who hasn't been active since June 2018. https://github.com/nicolaspanel/numjs/graphs/contributors https://github.com/nicolaspanel/numjs/graphs/contributors I sincerely believe it is possible for JavaScript to be a viable language ecosystem, but there is dire need for cohesion, collaboration, and longevity. As it stands, there are so many potentially viable projects strewn across the NPM landscape like old, discarded toys. I'm not aware of an initiative, let alone ethos, in the JS community that comes anywhere close to something like NumFocus. https://numfocus.org/ https://numfocus.org/
    • brylie 6y ago
      It is worth mentioning the Danfo project from a sibling comment: https://danfo.jsdata.org/ https://danfo.jsdata.org/
• nsonha 6y ago
  the only thing that used to be a problem is the number type. Libraries are ecosystem problem, not inherent to the language.
• bryanrasmussen 6y ago
  the reason to force data science is that same as the reason to develop libraries in languages for tasks which that language might otherwise seem not well suited to, that there is a large userbase of the language who know how to use it and would like to explore using that language for doing other things than it is normally used for. You may of course suggest that they should just learn a new language, but the history of computing shows that solutions for using languages to new purposes they might not seem suited for happens whenever such a purpose arises.
• heresie-dabord 6y ago
  > repeat the old and tired JavaScript hate, but this just isn’t a great idea. There is absolutely nothing wrong with coders/analysts/scientists building solutions in any language. The "hate" that you mention -- and then proceed to echo -- is a narrow way of asserting the superiority of $mylanguage and the inferiority of $yourlanguage. > flexible numeric types, fast math/algorithm libraries, and data manipulation Your point b) is usually written in a performant, compiled language, and your point c) can be built from robust primitives in any language. However, I will add a point d) about speed and memory usage. I do data analysis with the simplest set of performant tools: sqlite, bash-awk-sed-grep, Perl, Python, C++, SVG, and a browser to render. Any kind of glorified REPL beyond a terminal creates fragile complexity and dependency Hell. My kit doesn't include Node.js or ECMAscript but I'm willing to open my mind enough to think it might, one day. The current tooling for data analysis (or "data science" if we want to be faddish) is a mess and I look forward to better tools in the future.
• jwilber 6y ago
  On point 3 - I had to implement a logistic regression model in js recently and implementing all of the required math methods (eg dot product, transpose, vectorized addition, etc.) were actually super easy with js’s functional array utilities.
  • clircle 6y ago
    js doesn't have a glm library?
    • jwilber 6y ago
      js does have a glm library.
      • javierluraschi 6y ago
        Which one are you using?
        • jwilber 6y ago
          I’m not using one - as said, the project required I implement the model/all requisite math from scratch.
• lhnz 6y ago
  You should absolutely read "JavaScript and the next decade of data programming" by Ben Schmidt [1] before outright saying that it wouldn't be a great idea. JavaScript does have integers (e.g. `Uint8Array`) and it also has big numbers (e.g. `BigInt`). It's true that there's not yet an equivalent to Numpy/Pandas/Scikit yet, but POCs show that it will be possible to create such a thing and that we will be able to use the WebGPU API to access higher performance than is available using Python [2]. I'm not saying that it will definitely happen, but why not? [1] http://benschmidt.org/post/2020-01-15/2020-01-15-webgpu/ http://benschmidt.org/post/2020-01-15/2020-01-15-webgpu/ [2] https://github.com/milhidaka/webgpu-blas https://github.com/milhidaka/webgpu-blas
• ojosilva 6y ago
  We're using NodeJS for data science with Tensorflow JS and it's excellent, at least for our use case, which is 90% discrete data, mostly classification tasks. NodeJS evented architecture is great for multitasking training (and prediction) jobs. I use Node Streams to extract and process data flows out of several data sources on my Macbook Air M1 using the new Neural Engine 16-core chip to train CNN models with excellent results. Data prediction then runs on a ReactJS app, which gives my users a way to model, transform and visualize data on a browser. Everything is in Typescript, which reduces cognitive overload for our programmers and produces good end-to-end duck-type coherence and easy, integrated testing. Now, most data science libs for Javascript are either on their infancy, are proof-of-concept or just abandoned, but TFJS is solid and if you know what, why and to which extent you're using JS for data science, then it's absolutely fine.