5 ms·

> I would suggest you don't make dramatic claims If you could point out some stuff from ZProc's page, that would be nice! > mpi is the grandfather of message

by devxpy 8y ago

> I would suggest you don't make dramatic claims If you could point out some stuff from ZProc's page, that would be nice! > mpi is the grandfather of message passing libraries Never heard of it before, but just a simple google search reveals that it _might_ be more performant than zmq, but not as fault-tolerant and flexible. It really looks like a niche thing, from this comment by peter hintjens > Why smart cloud builders are betting everything on 0MQ. In detail, compare to the alternatives. Hand-rolling your own TCP stack is insane. Using any broker-based product won't scale. Buying licenses from IBM or TIBCO would eat up your capital. Supercomputing products like MPI aren't designed for this scale. There is literally no alternative. (http://zeromq.org/docs:the-ten-minute-talk http://zeromq.org/docs:the-ten-minute-talk) > Don't get me wrong, message-passing has some advantages, but they certainly aren't that it 'solves' parallelism. Doesn't it? (For most people) --- I can't believe I'm hearing words against zmq on HN, its wierd. Even the guys over at Dask settled on ZMQ over anything - https://github.com/dask/distributed/issues/776 https://github.com/dask/distributed/issues/776 P.S. Seems like you know quite a lot about this topic. Do you have any projects of your own that I can see? Bottom line, I think most people would be happy doing message passing parallelism in the real world. Sure, it doesn't look that good in theory but works damn good in practicality.

TickleSteve 8y ago
Nothing against zeromq, its good s/w, but like all tools it must be used appropriately. ...also, nanomsg is the 'improved' successor. Also, MPI isn't a 'niche' thing, its the way that a large proportion of high-performance applications have been implemented for a few decades (think Crays & weather prediction). Zeromq has a few simple web-apps using it (I exagerate slightly).
- devxpy 8y ago
  Seems like you know quite a lot about this topic. Do you have any projects of your own that I can see? Or do you only work for your employer or something?
  - adw 8y ago
    MPI is extremely well-known: https://en.wikipedia.org/wiki/Message_Passing_Interface https://en.wikipedia.org/wiki/Message_Passing_Interface Anyone who has done any scientific or technical computing is highly likely to be familiar with it – it's been around in some form for over 25 years.
    - devxpy 8y ago
      Do you think using this over zmq in my thingy would really bring much improvement? I use it everyday for some of my home baked apps, and would love if this really made a difference.
      - adw 8y ago
        That depends on your workload. 0MQ is fine software, but they solve different problems. The problem here is the claims you're making. You've written some utility classes around 0MQ for some applications, which is a real thing, so I'd rewrite your GitHub readme to just demonstrate what problems you've solved with it (and at what kind of scale). Making big, sweeping claims gets you into these kinds of threads, because extraordinary claims require extraordinary evidence :-)
        devxpy 8y ago
        Updated readme to be a little better https://github.com/pycampers/zproc https://github.com/pycampers/zproc I use it in a couple of my own stuff which I have open sourced yet. I do plan to release them, and I hope I can prove the usefulness of the library using those... professional (corporate) stuff would be a far fetch for me, obviously.
  - TickleSteve 8y ago
    Like the vast majority of software engineers, my work is not open source. But since you seem like arguments from authority, I've got around 25 years experience in software ranging from hard-real-time embedded defence software to safety-critical train braking systems. I've been software architect on systems selling 10's of millions of products, currently working in the IoT space. I've architected and implemented software on servers, desktops, embedded and mobile platforms. But no, you aren't likely to find my stuff on GitHub.
- devxpy 8y ago
  Is "Crays and weather prediction" really a part of everyday compute?

by devxpy 8y ago

TickleSteve 8y ago

Nothing against zeromq, its good s/w, but like all tools it must be used appropriately. ...also, nanomsg is the 'improved' successor. Also, MPI isn't a 'niche' thing, its the way that a large proportion of high-performance applications have been implemented for a few decades (think Crays & weather prediction). Zeromq has a few simple web-apps using it (I exagerate slightly).

devxpy 8y ago
Seems like you know quite a lot about this topic. Do you have any projects of your own that I can see? Or do you only work for your employer or something?
- adw 8y ago
  MPI is extremely well-known: https://en.wikipedia.org/wiki/Message_Passing_Interface https://en.wikipedia.org/wiki/Message_Passing_Interface Anyone who has done any scientific or technical computing is highly likely to be familiar with it – it's been around in some form for over 25 years.
  - devxpy 8y ago
    Do you think using this over zmq in my thingy would really bring much improvement? I use it everyday for some of my home baked apps, and would love if this really made a difference.
    - adw 8y ago
      That depends on your workload. 0MQ is fine software, but they solve different problems. The problem here is the claims you're making. You've written some utility classes around 0MQ for some applications, which is a real thing, so I'd rewrite your GitHub readme to just demonstrate what problems you've solved with it (and at what kind of scale). Making big, sweeping claims gets you into these kinds of threads, because extraordinary claims require extraordinary evidence :-)
      - devxpy 8y ago
        Updated readme to be a little better https://github.com/pycampers/zproc https://github.com/pycampers/zproc I use it in a couple of my own stuff which I have open sourced yet. I do plan to release them, and I hope I can prove the usefulness of the library using those... professional (corporate) stuff would be a far fetch for me, obviously.
- TickleSteve 8y ago
  Like the vast majority of software engineers, my work is not open source. But since you seem like arguments from authority, I've got around 25 years experience in software ranging from hard-real-time embedded defence software to safety-critical train braking systems. I've been software architect on systems selling 10's of millions of products, currently working in the IoT space. I've architected and implemented software on servers, desktops, embedded and mobile platforms. But no, you aren't likely to find my stuff on GitHub.
devxpy 8y ago
Is "Crays and weather prediction" really a part of everyday compute?