5 ms·

It would be nice if such standard supported Publisher-Subscriber style of messaging. For example alarm could subscribe to events from motion sensors and securit

by mr__y 7y ago

It would be nice if such standard supported Publisher-Subscriber style of messaging. For example alarm could subscribe to events from motion sensors and security cams (a camera in addition to providing a stream could push events like motion detection) or heating controller could subscribe to temperature change events. To make security of that realistic this would need some form of ACL, probably centrally[0] controlled. [0] central as in "home router" not as "somewhere in the cloud"

armagon 7y ago
I expect that MQTT is that standard. ( http://mqtt.org/ http://mqtt.org/ ) I have just been playing around with IoT devices and was just trying to set it up.
- mr__y 7y ago
  Thanks, it looks really interesting and seems to do what I just described. The thing it lacks though is - in my opinion - RPC-style messages. For example an alarm controller could subscribe to motion events, but when such an event occur it should be able to turn on sound-device immediately or shut some locks. The one-to-one communication as described here: https://github.com/mqtt/mqtt.github.io/wiki/one_to_one https://github.com/mqtt/mqtt.github.io/wiki/one_to_one seems more like a hack than a proper solution.
- icebraining 7y ago
  MQTT is a nice transport layer, but to be really useful this needs standard message templates and topics.
zackmorris 7y ago
Another thing I'd like to see is a distributed shared state standard (not just for IoT): https://en.wikipedia.org/wiki/Distributed_shared_memory https://en.wikipedia.org/wiki/Distributed_shared_memory https://en.wikipedia.org/wiki/Software_transactional_memory https://en.wikipedia.org/wiki/Software_transactional_memory https://en.wikipedia.org/wiki/Consensus_algorithm https://en.wikipedia.org/wiki/Consensus_algorithm https://en.wikipedia.org/wiki/Raft_(computer_science) https://en.wikipedia.org/wiki/Raft_(computer_science) https://en.wikipedia.org/wiki/Distributed_hash_table https://en.wikipedia.org/wiki/Distributed_hash_table The current trend of using async logic via message passing to maintain state is extremely error-prone and I personally consider it nondeterministic if it uses timeouts (which is pretty much the entire internet currently). A much better system works more like Firebase, where clients can subscribe to change events but also have access to a single tree that holds the latest agreed-upon state. I'd recommend something like a distributed hash table (DHT) so that the entire state doesn't have to be downloaded by each client. The state itself should probably be binary JSON to start with, but the standard should be loose and allow for XML or anything else. This would take us to something more UNIX-like, where devices can be thought of as processes interacting via a single shared memory, and abstract away the tedious/brittle/insecure interconnect layer. It should also support unreliable message passing like UDP for realtime support, which is required for simulations and gaming (there is a higher than 50% risk of the protocol omitting this or getting it wrong). After writing this out, I just realized that what I really want is an Actor model for IoT, where each device is completely isolated but communicates over channels like Elixir/Erlang or Go. I've come to reject both async and concurrent shared mutable memory because they each have major drawbacks. But I think that synchronous functional programming with primarily immutable variables and no shared mutable memory, building up a last known-good state inspectable by anyone with privileges is the future. The state could/should have something like an access control lists (ACL) and/or a permissions system. Ideally this should be hierarchical just like Firebase, and probably use a similar rule-based logic, to facilitate assigning roles to each client. It's trivial to demonstrate that once you have this, you can build any other seemingly complex abstraction above it like a database or game server. We have been missing this abstraction for several decades so we keep witnessing company after company reinvent the wheel. As far as I know, the only attempt to do this is RethinkDB: https://en.wikipedia.org/wiki/RethinkDB https://en.wikipedia.org/wiki/RethinkDB Unfortunately, I'm not sure if it's still supported: https://rethinkdb.com/blog/rethinkdb-shutdown/ https://rethinkdb.com/blog/rethinkdb-shutdown/ https://lwn.net/Articles/713716/ https://lwn.net/Articles/713716/ As usual, I would gladly work on a project like this, but there is currently no funding mechanism to support foundational open source projects like this with market-rate pay.
- mr__y 7y ago
  How would you solve access permissions in DHT? What if a node in DHT holds data that it is not granted to read? This would imply encryption of the data to prevent unauthorized reads - this does not really seem as a complicated problem - the data could be encrypted with only the nodes granted the read access having the decryption key(s). What seems harder to me is write permissions - if the data is encrypted/signed, only those nodes that have been granted permissions (and thus have copies of keys) would be able to overwrite/change that data. So the node that holds the data would not be able to overwrite it if it does not have r/w access to that piece of data. Nothing prevents from removing this data though. (I'm assuming the possibility of some nodes being malicious)
  - nybble41 7y ago
    > What seems harder to me is write permissions The usual solution for this is that the data must be signed by a specific private key; possession of that key grants write permission. A generation count or timestamp is embedded in the signed data to ensure that newer data cannot be replaced with obsolete data. Malicious nodes can still block updates from being distributed, though, so you need more than one path to seed the data into the network.
  - zackmorris 7y ago
    I'm not sure for DHT. But in Firebase the permissions are handled as user access rules written in Javascript, returning boolean true or false if the user has the ability to read/write the node and if the data being written passes a validation check: https://firebase.google.com/docs/rules https://firebase.google.com/docs/rules https://firebase.google.com/docs/rules/rules-language https://firebase.google.com/docs/rules/rules-language So Firebase permissions really have more to do with the identity layer than keeping the data secret. I think that probably everything should be encrypted anyway. This was a largely solved problem even in the 90s, and embedded devices are about as powerful as the 386 and 68030 computers popular then. We really need a distributed SSL identity layer though. Something like letsencrypt.org except the client's identity would come from the social proof of its peers instead of a central organization. Maybe this is already a solved problem in OpenSSL? I only understand symmetric key encryption, but not the man in the middle protections that SSL certificates provide. To me, p2p identity and encryption should have been a foundational pillar of the internet, not some controversial afterthought like we think of it today.

by mr__y 7y ago

armagon 7y ago

I expect that MQTT is that standard. ( http://mqtt.org/ http://mqtt.org/ ) I have just been playing around with IoT devices and was just trying to set it up.

mr__y 7y ago
Thanks, it looks really interesting and seems to do what I just described. The thing it lacks though is - in my opinion - RPC-style messages. For example an alarm controller could subscribe to motion events, but when such an event occur it should be able to turn on sound-device immediately or shut some locks. The one-to-one communication as described here: https://github.com/mqtt/mqtt.github.io/wiki/one_to_one https://github.com/mqtt/mqtt.github.io/wiki/one_to_one seems more like a hack than a proper solution.
icebraining 7y ago
MQTT is a nice transport layer, but to be really useful this needs standard message templates and topics.

zackmorris 7y ago

Another thing I'd like to see is a distributed shared state standard (not just for IoT): https://en.wikipedia.org/wiki/Distributed_shared_memory https://en.wikipedia.org/wiki/Distributed_shared_memory https://en.wikipedia.org/wiki/Software_transactional_memory https://en.wikipedia.org/wiki/Software_transactional_memory https://en.wikipedia.org/wiki/Consensus_algorithm https://en.wikipedia.org/wiki/Consensus_algorithm https://en.wikipedia.org/wiki/Raft_(computer_science) https://en.wikipedia.org/wiki/Raft_(computer_science) https://en.wikipedia.org/wiki/Distributed_hash_table https://en.wikipedia.org/wiki/Distributed_hash_table The current trend of using async logic via message passing to maintain state is extremely error-prone and I personally consider it nondeterministic if it uses timeouts (which is pretty much the entire internet currently). A much better system works more like Firebase, where clients can subscribe to change events but also have access to a single tree that holds the latest agreed-upon state. I'd recommend something like a distributed hash table (DHT) so that the entire state doesn't have to be downloaded by each client. The state itself should probably be binary JSON to start with, but the standard should be loose and allow for XML or anything else. This would take us to something more UNIX-like, where devices can be thought of as processes interacting via a single shared memory, and abstract away the tedious/brittle/insecure interconnect layer. It should also support unreliable message passing like UDP for realtime support, which is required for simulations and gaming (there is a higher than 50% risk of the protocol omitting this or getting it wrong). After writing this out, I just realized that what I really want is an Actor model for IoT, where each device is completely isolated but communicates over channels like Elixir/Erlang or Go. I've come to reject both async and concurrent shared mutable memory because they each have major drawbacks. But I think that synchronous functional programming with primarily immutable variables and no shared mutable memory, building up a last known-good state inspectable by anyone with privileges is the future. The state could/should have something like an access control lists (ACL) and/or a permissions system. Ideally this should be hierarchical just like Firebase, and probably use a similar rule-based logic, to facilitate assigning roles to each client. It's trivial to demonstrate that once you have this, you can build any other seemingly complex abstraction above it like a database or game server. We have been missing this abstraction for several decades so we keep witnessing company after company reinvent the wheel. As far as I know, the only attempt to do this is RethinkDB: https://en.wikipedia.org/wiki/RethinkDB https://en.wikipedia.org/wiki/RethinkDB Unfortunately, I'm not sure if it's still supported: https://rethinkdb.com/blog/rethinkdb-shutdown/ https://rethinkdb.com/blog/rethinkdb-shutdown/ https://lwn.net/Articles/713716/ https://lwn.net/Articles/713716/ As usual, I would gladly work on a project like this, but there is currently no funding mechanism to support foundational open source projects like this with market-rate pay.

mr__y 7y ago
How would you solve access permissions in DHT? What if a node in DHT holds data that it is not granted to read? This would imply encryption of the data to prevent unauthorized reads - this does not really seem as a complicated problem - the data could be encrypted with only the nodes granted the read access having the decryption key(s). What seems harder to me is write permissions - if the data is encrypted/signed, only those nodes that have been granted permissions (and thus have copies of keys) would be able to overwrite/change that data. So the node that holds the data would not be able to overwrite it if it does not have r/w access to that piece of data. Nothing prevents from removing this data though. (I'm assuming the possibility of some nodes being malicious)
- nybble41 7y ago
  > What seems harder to me is write permissions The usual solution for this is that the data must be signed by a specific private key; possession of that key grants write permission. A generation count or timestamp is embedded in the signed data to ensure that newer data cannot be replaced with obsolete data. Malicious nodes can still block updates from being distributed, though, so you need more than one path to seed the data into the network.
- zackmorris 7y ago
  I'm not sure for DHT. But in Firebase the permissions are handled as user access rules written in Javascript, returning boolean true or false if the user has the ability to read/write the node and if the data being written passes a validation check: https://firebase.google.com/docs/rules https://firebase.google.com/docs/rules https://firebase.google.com/docs/rules/rules-language https://firebase.google.com/docs/rules/rules-language So Firebase permissions really have more to do with the identity layer than keeping the data secret. I think that probably everything should be encrypted anyway. This was a largely solved problem even in the 90s, and embedded devices are about as powerful as the 386 and 68030 computers popular then. We really need a distributed SSL identity layer though. Something like letsencrypt.org except the client's identity would come from the social proof of its peers instead of a central organization. Maybe this is already a solved problem in OpenSSL? I only understand symmetric key encryption, but not the man in the middle protections that SSL certificates provide. To me, p2p identity and encryption should have been a foundational pillar of the internet, not some controversial afterthought like we think of it today.