3 ms·

How does JuliaDB compare to DataFrame.jl? I know JuliaDB has row indices, which is a very nice feature that DataFrame lacks. Is there any reason I shouldn't use

by smabie 6y ago

How does JuliaDB compare to DataFrame.jl? I know JuliaDB has row indices, which is a very nice feature that DataFrame lacks. Is there any reason I shouldn't use JuliaDB even if I don't need persistence. I wish the Julia ecosystem was a little more integrated: there are a lot of different competing libraries that ostensibly do the same thing. Python has the advantage that it's obvious what you should use: numpy, Pandas, scipy, statsmodels, matplotlib, etc. With Julia, it's less clear. Though I think part of the reason is that actually releasing a new scientific computing Python library is incredibly difficult and requires a lot of expertise. Julia makes it pretty trivial for anyone to contribute a model that has excellent performance. This fragmentation is a common problem among expressive languages.

elalaouifaris 6y ago
I think that this is changing when you see how the community is organized on github. There is a Julia something (or specific name like sciml) for all major applications. I have rarely seen this level of integration in a language community.
xiaodai 6y ago
Partly, it is because Julia is newish and partly because of the Lisp curse that Julia suffers from. But nonsustsined efforts will fall by the way side and true gems will emerge as the clear front runner like DataFrames.jl
- porker 6y ago
  > and partly because of the Lisp curse that Julia suffers from. The Lisp curse?
  - jonniedie 6y ago
    It’s pronounced “curth”
  - oxinabox 6y ago
    http://winestockwebdesign.com/Essays/Lisp_Curse.html http://winestockwebdesign.com/Essays/Lisp_Curse.html
akdor1154 6y ago
I find the IndexedTables (the underlying dataframe-like structure) model to be nicer to work with DataFrames.jl . It's much narrower API but it's well designed enough that this is a good thing, imo. The codebase is also a fair bit smaller. It also uses strongly typed tables (e.g. Table<int, string> etc), whereas DataFrames is loosely typed. Again I think this is a good decision (though it does grate with the Julia JIT's property of "being slow" the first time you run a function on a new type) Finally its split into IndexedTables and NDSparse is again a good design decision that I have not seen replicated in any other dataframe library. It just seems all around better designed. On the other hand it is verging on being unmaintained.
jabl 6y ago
Before numpy became the standard there was Numeric and Numarray, with painful rewrites for whoever had committed to those. And there was Scientific Python building upon those (I think it started using Numeric, then later on was ported on top of numpy), similar to scipy, but that is more or less dead nowadays. And before matplotlib became the standard, there was a bunch of older plotting libs; I recall using something that was a wrapper around gnuplot, IIRC. So I think it all mostly has to do with the fact that Julia is much newer and many even foundational areas are still in a state of flux. In a few decades the dust will have mostly settled. Of course, in the areas where people are pushing the envelope, it's good that a lot of approaches are tried out before the world settles on a winner.

by smabie 6y ago

elalaouifaris 6y ago

I think that this is changing when you see how the community is organized on github. There is a Julia something (or specific name like sciml) for all major applications. I have rarely seen this level of integration in a language community.

xiaodai 6y ago

Partly, it is because Julia is newish and partly because of the Lisp curse that Julia suffers from. But nonsustsined efforts will fall by the way side and true gems will emerge as the clear front runner like DataFrames.jl

porker 6y ago
> and partly because of the Lisp curse that Julia suffers from. The Lisp curse?
- jonniedie 6y ago
  It’s pronounced “curth”
- oxinabox 6y ago
  http://winestockwebdesign.com/Essays/Lisp_Curse.html http://winestockwebdesign.com/Essays/Lisp_Curse.html

akdor1154 6y ago

I find the IndexedTables (the underlying dataframe-like structure) model to be nicer to work with DataFrames.jl . It's much narrower API but it's well designed enough that this is a good thing, imo. The codebase is also a fair bit smaller. It also uses strongly typed tables (e.g. Table<int, string> etc), whereas DataFrames is loosely typed. Again I think this is a good decision (though it does grate with the Julia JIT's property of "being slow" the first time you run a function on a new type) Finally its split into IndexedTables and NDSparse is again a good design decision that I have not seen replicated in any other dataframe library. It just seems all around better designed. On the other hand it is verging on being unmaintained.

jabl 6y ago

Before numpy became the standard there was Numeric and Numarray, with painful rewrites for whoever had committed to those. And there was Scientific Python building upon those (I think it started using Numeric, then later on was ported on top of numpy), similar to scipy, but that is more or less dead nowadays. And before matplotlib became the standard, there was a bunch of older plotting libs; I recall using something that was a wrapper around gnuplot, IIRC. So I think it all mostly has to do with the fact that Julia is much newer and many even foundational areas are still in a state of flux. In a few decades the dust will have mostly settled. Of course, in the areas where people are pushing the envelope, it's good that a lot of approaches are tried out before the world settles on a winner.