3 ms·

That's great but what about nd4j's data structures resemble numpy arrays? Without making connection to the original most this comment just reads like typical sp

That's great but what about nd4j's data structures resemble numpy arrays? Without making connection to the original most this comment just reads like typical spam.

jnbiche 10y ago
In my experience, it's perfectly acceptable on HN to plug a related project in a comment, as long as it's not done repeatedly for totally unrelated projects (in 5+ years here, I can only think of 1 or 2 examples of this actually happening on a frequent basis). Given that these are open source and not even commercial projects, and that this is presumably an enthusiastic developer for those projects just sharing his/her work with a potentially interested audience, I hardly think the "spam" label is appropriate. And to answer your question specifically, based on what I've just read, ND4J offers an optimized n-dimensional array intended for numeric/scientific computing, very much in the manner of numpy. This is presumably what the comment's "our attempt at this for the jvm" refers to.
- agibsonccc 10y ago
  Yes it's a port of the associated concepts to the jvm. FWIW: we actually do use this commercially ;). It's apache though so do what you want with it etc. Apologies if it came off like that but the main intent was to show an alternative implementation. If you actually look at the nd4j.org link you'll also notice a simpler/more concise description of the same concepts. Eg: linear buffers,shapes, strides, offsets, dimensions
  - jnbiche 10y ago
    Apologies for assuming it wasn't commercial just because it was open source, my mistake. In any case, in my experience, comments regarding related non-commercial and commercial projects are quite welcome on HN. It also looks like you have some support for GPU acceleration, which is cool since as far as I know (barring recent developments), numpy doesn't have this natively.
    - agibsonccc 10y ago
      Oh haha not at all! You're right most of the time. Yes we put a lot of work in to multi gpu and the like. Send me an email if you're interested in any details! Emails in profile.
      - jnbiche 10y ago
        I'll go ahead and ask a question here so others can benefit: do you have any plans to support OpenCL, or will you focus on Cuda?
        agibsonccc 10y ago
        We want to. Time/bandwidth is our problem right now. No one uses opencl in deep learning but we aren't against it if someone is willing to work on it. Otherwise I'd have to pay someone a salary to do this..at the end of the day nd4j is part of what keeps us going.
doug1001 10y ago
your question caused me to spend a little time comparing the two (NumPy, which i know quite well, and nd4j, which i don't know at all). so based on comparison, it's clear the nd4j uses the key structural features of numpy arrays--linear buffers, strides, bit-wise and integer indexing, etc.--and the coincidence in the two APIs is also very high (eg, comparing the methods for a 2D NumPy array versus those for a 2D nd4j array). in my view, this is an impressive project, both w/r/t the work done to date and w/r/t the impact on java and scala open-source communities. In Scala land for instance, the most widely used matrix computation library is Breeze, which i've never found easy to use. nd4j, in particular, the Scala bindings, nd4s, seem to me to be a substantial improvement both in architecture and in the api.

That's great but what about nd4j's data structures resemble numpy arrays? Without making connection to the original most this comment just reads like typical spam.

jnbiche 10y ago

In my experience, it's perfectly acceptable on HN to plug a related project in a comment, as long as it's not done repeatedly for totally unrelated projects (in 5+ years here, I can only think of 1 or 2 examples of this actually happening on a frequent basis). Given that these are open source and not even commercial projects, and that this is presumably an enthusiastic developer for those projects just sharing his/her work with a potentially interested audience, I hardly think the "spam" label is appropriate. And to answer your question specifically, based on what I've just read, ND4J offers an optimized n-dimensional array intended for numeric/scientific computing, very much in the manner of numpy. This is presumably what the comment's "our attempt at this for the jvm" refers to.

agibsonccc 10y ago
Yes it's a port of the associated concepts to the jvm. FWIW: we actually do use this commercially ;). It's apache though so do what you want with it etc. Apologies if it came off like that but the main intent was to show an alternative implementation. If you actually look at the nd4j.org link you'll also notice a simpler/more concise description of the same concepts. Eg: linear buffers,shapes, strides, offsets, dimensions
- jnbiche 10y ago
  Apologies for assuming it wasn't commercial just because it was open source, my mistake. In any case, in my experience, comments regarding related non-commercial and commercial projects are quite welcome on HN. It also looks like you have some support for GPU acceleration, which is cool since as far as I know (barring recent developments), numpy doesn't have this natively.
  - agibsonccc 10y ago
    Oh haha not at all! You're right most of the time. Yes we put a lot of work in to multi gpu and the like. Send me an email if you're interested in any details! Emails in profile.
    - jnbiche 10y ago
      I'll go ahead and ask a question here so others can benefit: do you have any plans to support OpenCL, or will you focus on Cuda?
      - agibsonccc 10y ago
        We want to. Time/bandwidth is our problem right now. No one uses opencl in deep learning but we aren't against it if someone is willing to work on it. Otherwise I'd have to pay someone a salary to do this..at the end of the day nd4j is part of what keeps us going.

doug1001 10y ago

your question caused me to spend a little time comparing the two (NumPy, which i know quite well, and nd4j, which i don't know at all). so based on comparison, it's clear the nd4j uses the key structural features of numpy arrays--linear buffers, strides, bit-wise and integer indexing, etc.--and the coincidence in the two APIs is also very high (eg, comparing the methods for a 2D NumPy array versus those for a 2D nd4j array). in my view, this is an impressive project, both w/r/t the work done to date and w/r/t the impact on java and scala open-source communities. In Scala land for instance, the most widely used matrix computation library is Breeze, which i've never found easy to use. nd4j, in particular, the Scala bindings, nd4s, seem to me to be a substantial improvement both in architecture and in the api.