6 ms·

> "high performance" I never claimed it to be performant! "Above all, ZProc is written for safety and the ease of use." (Read here - https://github.com/pycam

by devxpy 8y ago

> "high performance" I never claimed it to be performant! "Above all, ZProc is written for safety and the ease of use." (Read here - https://github.com/pycampers/zproc?files=1#faq https://github.com/pycampers/zproc?files=1#faq) > It's not a revolution I totally agree. It's just a better way of doing things zmq already perfected. Like, tell me if you've ever seen a python object that has a `dict` API, but does message passing in the background. > central (pubsub?) server. Central server, yes. It uses PUB-SUB for state watching and REQ-REP for everything else. > you've just "discovered" message-passing Guess you're right? 2 years is a peanut on the time scale... P.S. Thanks for all the feedback, I've been dying to hear something for a while now.

TickleSteve 8y ago
I would suggest you don't make dramatic claims for a subject that has decades of theory behind it with a huge amount of nuance depending on the exact workload and characteristics of the machines in question. Don't get me wrong, message-passing has some advantages, but they certainly aren't that it 'solves' parallelism. If you wish to know more, investigate: - Smalltalk and Erlang (for message passing languages). - QNX (for a message-passing OS) - mpiPY (for a message-passing Python library, mpi is the grandfather of message passing libraries that runs everywhere). - Occam & the transputer for an example of a hardware-mp implementation (actually its Communicating Sequential Processes, but for your purposes it would be enlightening). - golang for a modern-day implementation of CSP. - Python implementation of CSP (https://github.com/futurecore/python-csp https://github.com/futurecore/python-csp) - Discussion about MP (http://wiki.c2.com/?MessagePassingConcurrency http://wiki.c2.com/?MessagePassingConcurrency, for more just google it) Basically, its great that you want to learn about concurrency & parallelism, but you've come to a gun fight with a blunt butter knife.
- devxpy 8y ago
  > I would suggest you don't make dramatic claims If you could point out some stuff from ZProc's page, that would be nice! > mpi is the grandfather of message passing libraries Never heard of it before, but just a simple google search reveals that it _might_ be more performant than zmq, but not as fault-tolerant and flexible. It really looks like a niche thing, from this comment by peter hintjens > Why smart cloud builders are betting everything on 0MQ. In detail, compare to the alternatives. Hand-rolling your own TCP stack is insane. Using any broker-based product won't scale. Buying licenses from IBM or TIBCO would eat up your capital. Supercomputing products like MPI aren't designed for this scale. There is literally no alternative. (http://zeromq.org/docs:the-ten-minute-talk http://zeromq.org/docs:the-ten-minute-talk) > Don't get me wrong, message-passing has some advantages, but they certainly aren't that it 'solves' parallelism. Doesn't it? (For most people) --- I can't believe I'm hearing words against zmq on HN, its wierd. Even the guys over at Dask settled on ZMQ over anything - https://github.com/dask/distributed/issues/776 https://github.com/dask/distributed/issues/776 P.S. Seems like you know quite a lot about this topic. Do you have any projects of your own that I can see? Bottom line, I think most people would be happy doing message passing parallelism in the real world. Sure, it doesn't look that good in theory but works damn good in practicality.
  - TickleSteve 8y ago
    Nothing against zeromq, its good s/w, but like all tools it must be used appropriately. ...also, nanomsg is the 'improved' successor. Also, MPI isn't a 'niche' thing, its the way that a large proportion of high-performance applications have been implemented for a few decades (think Crays & weather prediction). Zeromq has a few simple web-apps using it (I exagerate slightly).
    - devxpy 8y ago
      Seems like you know quite a lot about this topic. Do you have any projects of your own that I can see? Or do you only work for your employer or something?
      - adw 8y ago
        MPI is extremely well-known: https://en.wikipedia.org/wiki/Message_Passing_Interface https://en.wikipedia.org/wiki/Message_Passing_Interface Anyone who has done any scientific or technical computing is highly likely to be familiar with it – it's been around in some form for over 25 years.
        devxpy 8y ago
        Do you think using this over zmq in my thingy would really bring much improvement? I use it everyday for some of my home baked apps, and would love if this really made a difference.
        adw 8y ago
        That depends on your workload. 0MQ is fine software, but they solve different problems. The problem here is the claims you're making. You've written some utility classes around 0MQ for some applications, which is a real thing, so I'd rewrite your GitHub readme to just demonstrate what problems you've solved with it (and at what kind of scale). Making big, sweeping claims gets you into these kinds of threads, because extraordinary claims require extraordinary evidence :-)
- bushin 8y ago
  HN comment section shouldn't be a gun fight.
  - jstarfish 8y ago
    It isn't, it was more a metaphor for the fact that he's making arguments way out of his league of understanding.

by devxpy 8y ago

TickleSteve 8y ago

I would suggest you don't make dramatic claims for a subject that has decades of theory behind it with a huge amount of nuance depending on the exact workload and characteristics of the machines in question. Don't get me wrong, message-passing has some advantages, but they certainly aren't that it 'solves' parallelism. If you wish to know more, investigate: - Smalltalk and Erlang (for message passing languages). - QNX (for a message-passing OS) - mpiPY (for a message-passing Python library, mpi is the grandfather of message passing libraries that runs everywhere). - Occam & the transputer for an example of a hardware-mp implementation (actually its Communicating Sequential Processes, but for your purposes it would be enlightening). - golang for a modern-day implementation of CSP. - Python implementation of CSP (https://github.com/futurecore/python-csp https://github.com/futurecore/python-csp) - Discussion about MP (http://wiki.c2.com/?MessagePassingConcurrency http://wiki.c2.com/?MessagePassingConcurrency, for more just google it) Basically, its great that you want to learn about concurrency & parallelism, but you've come to a gun fight with a blunt butter knife.

devxpy 8y ago
> I would suggest you don't make dramatic claims If you could point out some stuff from ZProc's page, that would be nice! > mpi is the grandfather of message passing libraries Never heard of it before, but just a simple google search reveals that it _might_ be more performant than zmq, but not as fault-tolerant and flexible. It really looks like a niche thing, from this comment by peter hintjens > Why smart cloud builders are betting everything on 0MQ. In detail, compare to the alternatives. Hand-rolling your own TCP stack is insane. Using any broker-based product won't scale. Buying licenses from IBM or TIBCO would eat up your capital. Supercomputing products like MPI aren't designed for this scale. There is literally no alternative. (http://zeromq.org/docs:the-ten-minute-talk http://zeromq.org/docs:the-ten-minute-talk) > Don't get me wrong, message-passing has some advantages, but they certainly aren't that it 'solves' parallelism. Doesn't it? (For most people) --- I can't believe I'm hearing words against zmq on HN, its wierd. Even the guys over at Dask settled on ZMQ over anything - https://github.com/dask/distributed/issues/776 https://github.com/dask/distributed/issues/776 P.S. Seems like you know quite a lot about this topic. Do you have any projects of your own that I can see? Bottom line, I think most people would be happy doing message passing parallelism in the real world. Sure, it doesn't look that good in theory but works damn good in practicality.
- TickleSteve 8y ago
  Nothing against zeromq, its good s/w, but like all tools it must be used appropriately. ...also, nanomsg is the 'improved' successor. Also, MPI isn't a 'niche' thing, its the way that a large proportion of high-performance applications have been implemented for a few decades (think Crays & weather prediction). Zeromq has a few simple web-apps using it (I exagerate slightly).
  - devxpy 8y ago
    Seems like you know quite a lot about this topic. Do you have any projects of your own that I can see? Or do you only work for your employer or something?
    - adw 8y ago
      MPI is extremely well-known: https://en.wikipedia.org/wiki/Message_Passing_Interface https://en.wikipedia.org/wiki/Message_Passing_Interface Anyone who has done any scientific or technical computing is highly likely to be familiar with it – it's been around in some form for over 25 years.
      - devxpy 8y ago
        Do you think using this over zmq in my thingy would really bring much improvement? I use it everyday for some of my home baked apps, and would love if this really made a difference.
        adw 8y ago
        That depends on your workload. 0MQ is fine software, but they solve different problems. The problem here is the claims you're making. You've written some utility classes around 0MQ for some applications, which is a real thing, so I'd rewrite your GitHub readme to just demonstrate what problems you've solved with it (and at what kind of scale). Making big, sweeping claims gets you into these kinds of threads, because extraordinary claims require extraordinary evidence :-)
bushin 8y ago
HN comment section shouldn't be a gun fight.
- jstarfish 8y ago
  It isn't, it was more a metaphor for the fact that he's making arguments way out of his league of understanding.