5 ms·

> So, basically, you allow network programmer to not understand network programming, which is considered a major (main?) benefit, but at the expense of operator

> So, basically, you allow network programmer to not understand network programming, which is considered a major (main?) benefit, but at the expense of operators needing one more service to maintain? That's true. My ultimate goal is to have a way to reconcile parts of a stack written in different languages to interop, without the burden of having to learn network programming (ie strict RPC). While it's true that Redis is yet another service, it is precisely that which I hope would appeal to some people if they use Redis anyway. > How many programmers need to write a client Well, when implementations in other languages come out (PHP and Node are in the works), programmers don't even have to 'write a client' at all. The point of the project is to be able to swap out existing functions with a Shisetsu-wrapped remote executables for easy integration. > how many users would need to run an additional service? Redis clearly doesn't pay off here. See my first answer. If they don't want to run an additional service, they can use existing RPC protocols instead (ZeroRPC, Thrift, ZeroC Ice) at the cost of additional development to integrate those.

dozzie 10y ago
> While it's true that Redis is yet another service, it is precisely that which I hope would appeal to some people if they use Redis anyway. It doesn't make sense. Instead of a system with one service one now has a system with two services, each of which can fail in its own unique way. More moving parts for absolutely no reason at all. This way you could add there ZeroMQ or AMQP, just to appeal to people who have these two already deployed. >> How many programmers need to write a client > when implementations in other languages come out [...], programmers don't even have to 'write a client' at all. So the main benefit of "not needing to use network sockets to write a client" drowns down the toilet. Have you ever needed to implement your own XML-RPC client? I only needed that once, and only because I insisted on using a runtime without a satisfactory one. This is what libraries are for, not sprinkling a database over the smallest things. > The point of the project is to be able to swap out existing functions with a Shisetsu-wrapped remote executables for easy integration. This is the whole point of RPC, not putting Redis there. > If [users] don't want to run an additional service, they can use existing RPC protocols instead (ZeroRPC, Thrift, ZeroC Ice) at the cost of additional development to integrate those. So you say Redis acts as a magic faerie dust that eases integration process? How?
- kixpanganiban 10y ago
  Thanks for your insights. Clearly, I should have put more thought about the whole project before showing it to the world. You're absolutely right on most points, this introduces more moving parts, another unnecessary service, and defeats the purpose of an RPC. In fact, I have had to implement my own XML-RPC client (and an RPC-over-HTTP client too) on a certain occasion, and I tried to remember my gripes on that occasion and avoid them with this approach. Regarding your last line: > So you say Redis acts as a magic faerie dust that eases integration process? How? I guess what I said was too vague. It should have been "it makes an RPC-like system easier to integrate when you already have Redis". Shisetsu is not a magic bullet. It won't make things magically easier for everyone. All it is really is a niche project that, given you already use Python and Redis, is something for you to consider. If not, then you can move along, there's absolutely nothing to see here. :)
  - dozzie 10y ago
    >> So you say Redis acts as a magic faerie dust that eases integration process? How? > I guess what I said was too vague. It should have been "it makes an RPC-like system easier to integrate when you already have Redis". OK, let's work with this: in what way is Redis helping here over XML-RPC or JSON-RPC? What kind of problems does it solve that are pain in the ass with the other RPC protocols? (And now don't take my word that it doesn't, try to justify your choice, and next time start your README with that.) I'm not against custom protocols or adding daemons to a system (I've recently(ish) written a set of RPC daemons with their own protocols; the daemons are replacing Celery task queue, which itself is a widely tested software), they just have a decent reason to be introduced. Additional complexity just needs to pay for itself somehow.
    - kixpanganiban 10y ago
      Thanks man, I really appreciate you giving me points and guiding me here. > OK, let's work with this: in what way is Redis helping here over XML-RPC or JSON-RPC? What kind of problems does it solve that are pain in the ass with the other RPC protocols? Redis vs XML-RPC/JSON-RPC is a mismatch. It's more of a msgpack-RPC with Redis as broker vs XML-RPC or JSON-RPC: 1. Most brokerless RPC protocols make it difficult to inspect the message on the fly. With Redis, you can open up redis-cli and subscribe to the appropriate channels to extract the raw messages. For most users this might not mean anything, but for others this could mean easier debugging, measuring performance, and so on. 2. msgpack's serialized format is small. This makes it faster to send it across a network especially when the message is considerably large (vs, say, JSON or XML). That, and msgpack itself performs serialization/deserialization quickly on its own. see: http://webcache.googleusercontent.com/search?q=cache:hYaRwEUpbdwJ:maxondev.com/serialization-performance-comparison-c-net-formats-frameworks-xmldatacontractserializer-xmlserializer-binaryformatter-json-newtonsoft-servicestack-text/+&cd=2&hl=en&ct=clnk&gl=ph http://webcache.googleusercontent.com/search?q=cache:hYaRwEU... (I'm using a Google Cached link because main link seems to be down) > I'm not against custom protocols or adding daemons to a system (I've recently(ish) written a set of RPC daemons with their own protocols; the daemons are replacing Celery task queue, which itself is a widely tested software), they just have a decent reason to be introduced. Additional complexity just needs to pay for itself somehow I totally agree. However, the idea of introducing Redis into the equation to use as the broker didn't seem like additional complexity to me because a lot of other network messaging protocols rely on a broker anyway. For example, the widely popular Web App Messaging Protocol (WAMP) (http://wamp-proto.org/why/ http://wamp-proto.org/why/) relies on brokers to decouple callee/caller on an RPC operation. ZeroRPC, another popular RPC choice, uses ZMQ. For Shisetsu, what I see is the main advantage of Redis is that I can delegate multiplexing work to it instead of having to code myself, making the protocol and its implementation easier to manage, just by relying on Redis' PubSub. The client and server don't even have to know each other's address and port, they just have to know which Redis instance to connect to and assume that there exists one of the other to work with. Redis also provides the channel buffer and the queue instead of me having to write those as part of the client/server code, which means anyone else looking to implement the protocol in another language wouldn't have to deal with those as well.

dozzie 10y ago

> While it's true that Redis is yet another service, it is precisely that which I hope would appeal to some people if they use Redis anyway. It doesn't make sense. Instead of a system with one service one now has a system with two services, each of which can fail in its own unique way. More moving parts for absolutely no reason at all. This way you could add there ZeroMQ or AMQP, just to appeal to people who have these two already deployed. >> How many programmers need to write a client > when implementations in other languages come out [...], programmers don't even have to 'write a client' at all. So the main benefit of "not needing to use network sockets to write a client" drowns down the toilet. Have you ever needed to implement your own XML-RPC client? I only needed that once, and only because I insisted on using a runtime without a satisfactory one. This is what libraries are for, not sprinkling a database over the smallest things. > The point of the project is to be able to swap out existing functions with a Shisetsu-wrapped remote executables for easy integration. This is the whole point of RPC, not putting Redis there. > If [users] don't want to run an additional service, they can use existing RPC protocols instead (ZeroRPC, Thrift, ZeroC Ice) at the cost of additional development to integrate those. So you say Redis acts as a magic faerie dust that eases integration process? How?

kixpanganiban 10y ago
Thanks for your insights. Clearly, I should have put more thought about the whole project before showing it to the world. You're absolutely right on most points, this introduces more moving parts, another unnecessary service, and defeats the purpose of an RPC. In fact, I have had to implement my own XML-RPC client (and an RPC-over-HTTP client too) on a certain occasion, and I tried to remember my gripes on that occasion and avoid them with this approach. Regarding your last line: > So you say Redis acts as a magic faerie dust that eases integration process? How? I guess what I said was too vague. It should have been "it makes an RPC-like system easier to integrate when you already have Redis". Shisetsu is not a magic bullet. It won't make things magically easier for everyone. All it is really is a niche project that, given you already use Python and Redis, is something for you to consider. If not, then you can move along, there's absolutely nothing to see here. :)
- dozzie 10y ago
  >> So you say Redis acts as a magic faerie dust that eases integration process? How? > I guess what I said was too vague. It should have been "it makes an RPC-like system easier to integrate when you already have Redis". OK, let's work with this: in what way is Redis helping here over XML-RPC or JSON-RPC? What kind of problems does it solve that are pain in the ass with the other RPC protocols? (And now don't take my word that it doesn't, try to justify your choice, and next time start your README with that.) I'm not against custom protocols or adding daemons to a system (I've recently(ish) written a set of RPC daemons with their own protocols; the daemons are replacing Celery task queue, which itself is a widely tested software), they just have a decent reason to be introduced. Additional complexity just needs to pay for itself somehow.
  - kixpanganiban 10y ago
    Thanks man, I really appreciate you giving me points and guiding me here. > OK, let's work with this: in what way is Redis helping here over XML-RPC or JSON-RPC? What kind of problems does it solve that are pain in the ass with the other RPC protocols? Redis vs XML-RPC/JSON-RPC is a mismatch. It's more of a msgpack-RPC with Redis as broker vs XML-RPC or JSON-RPC: 1. Most brokerless RPC protocols make it difficult to inspect the message on the fly. With Redis, you can open up redis-cli and subscribe to the appropriate channels to extract the raw messages. For most users this might not mean anything, but for others this could mean easier debugging, measuring performance, and so on. 2. msgpack's serialized format is small. This makes it faster to send it across a network especially when the message is considerably large (vs, say, JSON or XML). That, and msgpack itself performs serialization/deserialization quickly on its own. see: http://webcache.googleusercontent.com/search?q=cache:hYaRwEUpbdwJ:maxondev.com/serialization-performance-comparison-c-net-formats-frameworks-xmldatacontractserializer-xmlserializer-binaryformatter-json-newtonsoft-servicestack-text/+&cd=2&hl=en&ct=clnk&gl=ph http://webcache.googleusercontent.com/search?q=cache:hYaRwEU... (I'm using a Google Cached link because main link seems to be down) > I'm not against custom protocols or adding daemons to a system (I've recently(ish) written a set of RPC daemons with their own protocols; the daemons are replacing Celery task queue, which itself is a widely tested software), they just have a decent reason to be introduced. Additional complexity just needs to pay for itself somehow I totally agree. However, the idea of introducing Redis into the equation to use as the broker didn't seem like additional complexity to me because a lot of other network messaging protocols rely on a broker anyway. For example, the widely popular Web App Messaging Protocol (WAMP) (http://wamp-proto.org/why/ http://wamp-proto.org/why/) relies on brokers to decouple callee/caller on an RPC operation. ZeroRPC, another popular RPC choice, uses ZMQ. For Shisetsu, what I see is the main advantage of Redis is that I can delegate multiplexing work to it instead of having to code myself, making the protocol and its implementation easier to manage, just by relying on Redis' PubSub. The client and server don't even have to know each other's address and port, they just have to know which Redis instance to connect to and assume that there exists one of the other to work with. Redis also provides the channel buffer and the queue instead of me having to write those as part of the client/server code, which means anyone else looking to implement the protocol in another language wouldn't have to deal with those as well.