4 ms·

Toward an open Internet of Things

45 points by Garbage 12y ago

huntaub 12y ago
I 100% agree that there needs to be a simple protocol (much like email) that can help interconnect the many devices that we are bringing online. I think that the emphasis this time around, however, will be on security and flexibility. We need a protocol that we can trust and adapt to changing times. PLUG: This is exactly the sort of thing that we are attempting to build at [AirDispatch](http://airdispatch.org http://airdispatch.org). You should check it out and let me know what you think.
feralmoan 12y ago
"Unfortunately, we’re still in a world where a manufacturer’s first reaction is to lock things down, to make devices incompatible, in an attempt to extract as much profit as possible from the system. If this is your mindset, you’re not going to release an API (as Philips did), let alone encourage your competition to build on the same API (as Philips hasn’t)." Yeah, IoT protocols are a warzone, hell, few companies/coders are even able to agree on general web API architectures (REST/RPC/SOAP - and everything in between. puke). Add MQTT, CoAP, XMPP or whatever other pet vendor protocol or semantic layer into the mix... total nightmare. What's left are a bunch of consumers hung out to dry because a these big old guard companies haven't learnt how to collaborate and deliver broad spectrum value to their customers. A lot of hand-rolled integrations for many years to come! Whoever can alleviate that pain will likely end up dominating
simpsond 12y ago
The devices need to leverage existing standards like TCP or UDP. Then the battle of IoT protocols happens between MQTT, CoAP, and existing things like HTTP. For the extremely tiny devices with low power budgets, these may be too heavy.. so there needs to be some sort of gateway. The problem then is the same as the internet of now: There is another layer on top of that which is specific to an application. I hope there is a battle for different standards and the best one wins.
fidotron 12y ago
This is a core idea I've been thinking about for a while. What I settled on is a system where I have different sensors around which just broadcast (really multicast) a trivial binary encoding of an object (in the JSON sense) whenever they feel like it. The idea is if you standardise the fields in the object then anything listening can decide which messages it wants and which it doesn't. The problem is inventing or maintaining a standard in this space is going to be a completely thankless task. We've seen things like Android where the theory of Intent interoperability is really good, but in practice it just isn't used that much, or is deliberately broken, such as with Facebook (in order to promote their SDK). The other problem is that there is a lot of feeling that any standards are being created in the hope that a problem emerges, and it's not like we have any really compelling problems with which to guide the standards. The reality is 90% of the IoT fuss is cloud infrastructure providers trying to come up with a use for their systems, and anything which doesn't rely on them will be told it's not part of the IoT, and is just a traditional hardware effort. EDIT: started fleshing this out on github a while back ( https://github.com/atomirex/chirp https://github.com/atomirex/chirp ), but never got round to uploading the actual implementations thanks to their terribly embarrassing state! This might persuade me to get round to resolving that.
- doublerebel 12y ago
  Protobuf is not bad, but relies on a fixed schema. Blink protocol transmits the schema and seems much better suited. But then there is the question of message security and authenticity. Hawk encoding seems to be the best option outside of TLS transport. There are enough existing formats/protocols/standards, let's work with the existing ones and continue to improve them.
  - fidotron 12y ago
    My conclusion was obscuring message contents isn't really too important, but certainly authenticating them is, and preventing replay attacks is where it gets hard. (So a challenge/response system would be needed in some cases). This becomes nasty at the device configuration stage though, since you need to distribute/share keys somewhere. There simply are not protocols around that deal with buying a device from a store, taking it home and magically making it work, but that is the point it needs to get to. The industry seems to be moving towards "solving" that through embedding cellular modems in everything and getting the stuff to phone home. The actual binary format is beside the point really, but it does need to be as close to self describing as possible.
11thEarlOfMar 12y ago
There is a lot to learn yet. The Internet of Things is a new paradigm for interacting with the world. Anyone who has developed applications that aggregate the control of disparate devices and tried to make them work together knows what the pain points are. For starters, there are many reasons why a Thing on a network might not do what you tell it to do. Error handling is a huge PITA.
- eldavido 12y ago
  +1. It's going to be a while before this all shakes out. I hope the standards bodies wait a bit and cherry-pick the best parts of the various vendor implementations, rather than trying to jam something incomplete and poorly specified into the market now. With all this noise about "standards" though, it's important to remember how much we (computer people) take for granted. I, as well as the majority of people here, I'd guess, wouldn't consider building a new device that doesn't use TCP/UDP and IPv4/6. There'll be a battle on the layer-1/2 front (radios, electrical stuff) and messaging formats (OSI layer 5+: application, session, presentation), but let's not forget how far we've come.
doublerebel 12y ago
participants should send data that obeys the specifications, but they should be willing to accept data that doesn’t. This is what I believe to be the next logical step in device interoperability -- the "un-API". Inferring API-call intention from the provided data. It will require more than one datapoint and a feedback loop, but an API only has to be categorized correctly once -- rather than every implementation being created manually or relying on custom independent library. Zapier has touched the tip of the iceberg on this. I am currently building a SaaS product to this end, please get in touch if this interests you.
- jmspring 12y ago
  This is nothing more than the Robustness Principle - "Be conservative in what you do, be liberal in what you accept from others" http://en.wikipedia.org/wiki/Robustness_principle http://en.wikipedia.org/wiki/Robustness_principle
  - doublerebel 12y ago
    Yes, the article states that directly before the line I quoted -- I assumed it would be recognized. In practice very few APIs are tolerant of unexpected fields, values, or protocols. Forget the protocol war that other commenters are expecting, our devices should interact regardless of protocol. Only then will producers be forced to compete on useful features.
- ttflee 12y ago
  > Inferring API-call intention from the provided data Correct me if I was wrong. By 'Inferring', did you mean that a generalized broker/translator/proxy could be built in between of unlimited APIs, like the intermediate representation of a compiler?
deleted 12y ago
[deleted]
anentropic 12y ago
I don't understand why anyone wants this
- pessimizer 12y ago
  Adolescent power fantasies.
  - pessimizer 12y ago
    Again, adolescent power fantasies.
pessimizer 12y ago
Money quote: "The assumption behind any 'lock-in' strategy is ultimately that your product is poor and that customers will gladly switch vendors if given a chance."
- deleted 12y ago
  [deleted]
- hershel 12y ago
  That's wrong. The assumption is "even great products can get copied, and than profits go to hell.".Lock-in is one of the rare strategies that work.
  - pessimizer 12y ago
    No one ever leaves your first mover product to go to a copy unless they have a reason to.
    - hershel 12y ago
      Isn't lower cost reason enough ? and what about new customers(and the side effects they carry for marketing and other stuff) ?
    - gress 12y ago
      The reason can be someone else's better advertising budget.
dharmadhikariv 12y ago
Qualcomm and Linux Foundation has taken great initiative in developing common platform for IoT through Allseen Alliance. Many device manufacturers are already supporting this effort in standardizing hardware & software stack for IoT.
zokier 12y ago
The main attraction in openess in IoT as a consumer is that it would hopefully enable the market for robust, hopefully open-source, turn-key software solutions for HW manufacturers, minimizing the amount of (inevitably broken) software the HW mfgs need to write themselves. I think we have repeatedly seen that HW guys just can not be relied to write good enough software to be exposed on the internet.
carsonreinke 12y ago
Hopefully "Internet of Connected Things"
jamesbritt 12y ago
I've been poking around with using OSC (Open Sound Control, but hardly limited to just sound) mixed with some sort of DNS-like or DNS-based registration and discovery. Devices that can peer can just exchange OSC messages directly. Otherwise they might use some sort of proxying service (to arrange for OSC over TCP).
- stelonix 12y ago
  This is interesting. I've been toying with the idea of using OSC to control general playback/playlists across different programs. The idea is to have an unified playlist, queue and volume control, but the end goal is to be able to synchronize different applications using OSC (think controlling Youtube playback and a general purpose audio player with OSC and having each other respect the current playback status).
Thiz 12y ago
Things connected to the internet should expose an API defining what they are, what they do, and what they need to perform their magic, so consumers can act accordingly. name: thermostat task: control room temperature icon: [base64] functions: - turn on - turn off - set temperature (temp:num) With that information my cell phone can interact with all the things around it. If I click on the thermostat icon I get the info shown and three butons with the actions I can perform on it. If I want to control the temperature I just enter a number and done. If a TV is around, I can easily turn it on/off, switch channels, volume, etc. No need for specialized apps, they're universal APIs every device can access, provided some level of security or trust. Is that the IoT we want?
- bradfa 12y ago
  Basically, yes. But how do you authenticate that the thing telling the TV to turn off actually has the authority to tell it to turn off? This is a whole nother level of complexity which consumers will need to navigate after the API for all the things are standardized and many companies are going to screw up the security way worse than they are currently screwing up the interoperability. The real problem with IoT isn't the standardization of the transfer medium, it's the security implications of having your things on an internet.
- bergie 12y ago
  That is actually pretty close to how we define components in the FBP protocol. You'll just want to add a bit more information about the inputs, and a possible set of outputs too (for example, a temperature reading?) http://noflojs.org/documentation/protocol/#component http://noflojs.org/documentation/protocol/#component For IoT use the default WebSocket transport isn't desirable, of course. But maybe something like MQTT? A subset of the protocol already works with serial-over-USB when talking to microcontrollers
jandrewrogers 12y ago
As someone working on standards development in this area, I would point out that the proliferation of proprietary APIs and protocols are not caused by a desire to create lock-in in many cases. More often than not, the interfaces are proprietary because good standards simply don't exist to do what is required. A large part of the problem often overlooked by the tech community is that none of the existing web standards are useful for the Internet of Things. It needs a new set of standards designed for the purpose so that applications can support the operations necessary at the throughput and operation rates required by IoT. For example, something that comes up a lot at the standards meetings is representation of data. Formats like JSON and XML were fine for the web, but when you have to continuously parse a billion records every second the implied computational cost of just parsing using most existing standard formats is extraordinary. It shifts the economics such that computational efficiency becomes totally worth the implementation cost. Unfortunately, there are not a lot of existing standards that reflect the tradeoffs you see with IoT implementations.
- Gobitron 12y ago
  Can you provide an example of an alternative that would be more efficient? And are you talking about communication over something other than HTTP?
- jallmann 12y ago
  What would be a good data representation at that scale? Is a typed/schema-driven approach preferable? For ingesting such massive amounts of data, it seems a sensible serialization would have minimal divergence between the wire and memory representation, such as Cap'n Proto. Handling composite types seems tricky too -- eg, for prefixing lengths to strings or lists, it appears the extra CPU time for variable-sized integers would be preferable to the I/O overhead of billions of wasted bytes that come with a fixed-size prefix. Assuming explicit begin/end delimiters aren't even an option here.
  - jandrewrogers 12y ago
    Cap'n'Proto is actually pretty good example of a modern, high-performance serialization. I've been using it as a template for IoT wire representation discussions. Fast wire encodings are almost universally TLV ("tag-length-value") style serializations. Delimiter scanning is inefficient and also means the parser has little ability to predict what will be coming over the wire so that it can optimize processing. While older serializations tend to be byte oriented, newer formats use word-sized "frames" (even if not aligned) to enable nearly branchless, parallel processing of the bytes in the stream using bit-twiddling techniques or vector instructions.
zwieback 12y ago
Seems a little early to complain about lack of protocol standards when the underlying transport and topologies aren't even converging yet. It seems easy enough for anything that supports conventional protocols like IP but a lot of future networked devices will be using NFC-like things or mesh networks.
viggity 12y ago
I work/consult for a startup doing Power Over Ethernet Lighting (www.igor-tech.com). We use UPnP (universal plug-n-play) to communicate with lights as there is already an open spec. FWIW, we're also releasing an open reference design for hardware.
legulere 12y ago
The alternative usually is design by committee, which leads to huge ugly standards. I'm also not so sure wether the robustness principle really helps. With HTML we had lots of problems because everybody parsed non-standard files differently.

45 points by Garbage 12y ago

I 100% agree that there needs to be a simple protocol (much like email) that can help interconnect the many devices that we are bringing online. I think that the emphasis this time around, however, will be on security and flexibility. We need a protocol that we can trust and adapt to changing times. PLUG: This is exactly the sort of thing that we are attempting to build at [AirDispatch](http://airdispatch.org http://airdispatch.org). You should check it out and let me know what you think.

feralmoan 12y ago

"Unfortunately, we’re still in a world where a manufacturer’s first reaction is to lock things down, to make devices incompatible, in an attempt to extract as much profit as possible from the system. If this is your mindset, you’re not going to release an API (as Philips did), let alone encourage your competition to build on the same API (as Philips hasn’t)." Yeah, IoT protocols are a warzone, hell, few companies/coders are even able to agree on general web API architectures (REST/RPC/SOAP - and everything in between. puke). Add MQTT, CoAP, XMPP or whatever other pet vendor protocol or semantic layer into the mix... total nightmare. What's left are a bunch of consumers hung out to dry because a these big old guard companies haven't learnt how to collaborate and deliver broad spectrum value to their customers. A lot of hand-rolled integrations for many years to come! Whoever can alleviate that pain will likely end up dominating

simpsond 12y ago

The devices need to leverage existing standards like TCP or UDP. Then the battle of IoT protocols happens between MQTT, CoAP, and existing things like HTTP. For the extremely tiny devices with low power budgets, these may be too heavy.. so there needs to be some sort of gateway. The problem then is the same as the internet of now: There is another layer on top of that which is specific to an application. I hope there is a battle for different standards and the best one wins.

fidotron 12y ago

This is a core idea I've been thinking about for a while. What I settled on is a system where I have different sensors around which just broadcast (really multicast) a trivial binary encoding of an object (in the JSON sense) whenever they feel like it. The idea is if you standardise the fields in the object then anything listening can decide which messages it wants and which it doesn't. The problem is inventing or maintaining a standard in this space is going to be a completely thankless task. We've seen things like Android where the theory of Intent interoperability is really good, but in practice it just isn't used that much, or is deliberately broken, such as with Facebook (in order to promote their SDK). The other problem is that there is a lot of feeling that any standards are being created in the hope that a problem emerges, and it's not like we have any really compelling problems with which to guide the standards. The reality is 90% of the IoT fuss is cloud infrastructure providers trying to come up with a use for their systems, and anything which doesn't rely on them will be told it's not part of the IoT, and is just a traditional hardware effort. EDIT: started fleshing this out on github a while back ( https://github.com/atomirex/chirp https://github.com/atomirex/chirp ), but never got round to uploading the actual implementations thanks to their terribly embarrassing state! This might persuade me to get round to resolving that.

doublerebel 12y ago
Protobuf is not bad, but relies on a fixed schema. Blink protocol transmits the schema and seems much better suited. But then there is the question of message security and authenticity. Hawk encoding seems to be the best option outside of TLS transport. There are enough existing formats/protocols/standards, let's work with the existing ones and continue to improve them.
- fidotron 12y ago
  My conclusion was obscuring message contents isn't really too important, but certainly authenticating them is, and preventing replay attacks is where it gets hard. (So a challenge/response system would be needed in some cases). This becomes nasty at the device configuration stage though, since you need to distribute/share keys somewhere. There simply are not protocols around that deal with buying a device from a store, taking it home and magically making it work, but that is the point it needs to get to. The industry seems to be moving towards "solving" that through embedding cellular modems in everything and getting the stuff to phone home. The actual binary format is beside the point really, but it does need to be as close to self describing as possible.

11thEarlOfMar 12y ago

There is a lot to learn yet. The Internet of Things is a new paradigm for interacting with the world. Anyone who has developed applications that aggregate the control of disparate devices and tried to make them work together knows what the pain points are. For starters, there are many reasons why a Thing on a network might not do what you tell it to do. Error handling is a huge PITA.

eldavido 12y ago
+1. It's going to be a while before this all shakes out. I hope the standards bodies wait a bit and cherry-pick the best parts of the various vendor implementations, rather than trying to jam something incomplete and poorly specified into the market now. With all this noise about "standards" though, it's important to remember how much we (computer people) take for granted. I, as well as the majority of people here, I'd guess, wouldn't consider building a new device that doesn't use TCP/UDP and IPv4/6. There'll be a battle on the layer-1/2 front (radios, electrical stuff) and messaging formats (OSI layer 5+: application, session, presentation), but let's not forget how far we've come.

doublerebel 12y ago

participants should send data that obeys the specifications, but they should be willing to accept data that doesn’t. This is what I believe to be the next logical step in device interoperability -- the "un-API". Inferring API-call intention from the provided data. It will require more than one datapoint and a feedback loop, but an API only has to be categorized correctly once -- rather than every implementation being created manually or relying on custom independent library. Zapier has touched the tip of the iceberg on this. I am currently building a SaaS product to this end, please get in touch if this interests you.

jmspring 12y ago
This is nothing more than the Robustness Principle - "Be conservative in what you do, be liberal in what you accept from others" http://en.wikipedia.org/wiki/Robustness_principle http://en.wikipedia.org/wiki/Robustness_principle
- doublerebel 12y ago
  Yes, the article states that directly before the line I quoted -- I assumed it would be recognized. In practice very few APIs are tolerant of unexpected fields, values, or protocols. Forget the protocol war that other commenters are expecting, our devices should interact regardless of protocol. Only then will producers be forced to compete on useful features.
ttflee 12y ago
> Inferring API-call intention from the provided data Correct me if I was wrong. By 'Inferring', did you mean that a generalized broker/translator/proxy could be built in between of unlimited APIs, like the intermediate representation of a compiler?

deleted 12y ago

[deleted]

anentropic 12y ago

I don't understand why anyone wants this

pessimizer 12y ago
Adolescent power fantasies.
- pessimizer 12y ago
  Again, adolescent power fantasies.

pessimizer 12y ago

Money quote: "The assumption behind any 'lock-in' strategy is ultimately that your product is poor and that customers will gladly switch vendors if given a chance."

deleted 12y ago
[deleted]
hershel 12y ago
That's wrong. The assumption is "even great products can get copied, and than profits go to hell.".Lock-in is one of the rare strategies that work.
- pessimizer 12y ago
  No one ever leaves your first mover product to go to a copy unless they have a reason to.
  - hershel 12y ago
    Isn't lower cost reason enough ? and what about new customers(and the side effects they carry for marketing and other stuff) ?
  - gress 12y ago
    The reason can be someone else's better advertising budget.

dharmadhikariv 12y ago

Qualcomm and Linux Foundation has taken great initiative in developing common platform for IoT through Allseen Alliance. Many device manufacturers are already supporting this effort in standardizing hardware & software stack for IoT.

zokier 12y ago

The main attraction in openess in IoT as a consumer is that it would hopefully enable the market for robust, hopefully open-source, turn-key software solutions for HW manufacturers, minimizing the amount of (inevitably broken) software the HW mfgs need to write themselves. I think we have repeatedly seen that HW guys just can not be relied to write good enough software to be exposed on the internet.

carsonreinke 12y ago

Hopefully "Internet of Connected Things"

jamesbritt 12y ago

I've been poking around with using OSC (Open Sound Control, but hardly limited to just sound) mixed with some sort of DNS-like or DNS-based registration and discovery. Devices that can peer can just exchange OSC messages directly. Otherwise they might use some sort of proxying service (to arrange for OSC over TCP).

stelonix 12y ago
This is interesting. I've been toying with the idea of using OSC to control general playback/playlists across different programs. The idea is to have an unified playlist, queue and volume control, but the end goal is to be able to synchronize different applications using OSC (think controlling Youtube playback and a general purpose audio player with OSC and having each other respect the current playback status).

Thiz 12y ago

Things connected to the internet should expose an API defining what they are, what they do, and what they need to perform their magic, so consumers can act accordingly. name: thermostat task: control room temperature icon: [base64] functions: - turn on - turn off - set temperature (temp:num) With that information my cell phone can interact with all the things around it. If I click on the thermostat icon I get the info shown and three butons with the actions I can perform on it. If I want to control the temperature I just enter a number and done. If a TV is around, I can easily turn it on/off, switch channels, volume, etc. No need for specialized apps, they're universal APIs every device can access, provided some level of security or trust. Is that the IoT we want?

bradfa 12y ago
Basically, yes. But how do you authenticate that the thing telling the TV to turn off actually has the authority to tell it to turn off? This is a whole nother level of complexity which consumers will need to navigate after the API for all the things are standardized and many companies are going to screw up the security way worse than they are currently screwing up the interoperability. The real problem with IoT isn't the standardization of the transfer medium, it's the security implications of having your things on an internet.
bergie 12y ago
That is actually pretty close to how we define components in the FBP protocol. You'll just want to add a bit more information about the inputs, and a possible set of outputs too (for example, a temperature reading?) http://noflojs.org/documentation/protocol/#component http://noflojs.org/documentation/protocol/#component For IoT use the default WebSocket transport isn't desirable, of course. But maybe something like MQTT? A subset of the protocol already works with serial-over-USB when talking to microcontrollers

jandrewrogers 12y ago

As someone working on standards development in this area, I would point out that the proliferation of proprietary APIs and protocols are not caused by a desire to create lock-in in many cases. More often than not, the interfaces are proprietary because good standards simply don't exist to do what is required. A large part of the problem often overlooked by the tech community is that none of the existing web standards are useful for the Internet of Things. It needs a new set of standards designed for the purpose so that applications can support the operations necessary at the throughput and operation rates required by IoT. For example, something that comes up a lot at the standards meetings is representation of data. Formats like JSON and XML were fine for the web, but when you have to continuously parse a billion records every second the implied computational cost of just parsing using most existing standard formats is extraordinary. It shifts the economics such that computational efficiency becomes totally worth the implementation cost. Unfortunately, there are not a lot of existing standards that reflect the tradeoffs you see with IoT implementations.

Gobitron 12y ago
Can you provide an example of an alternative that would be more efficient? And are you talking about communication over something other than HTTP?
jallmann 12y ago
What would be a good data representation at that scale? Is a typed/schema-driven approach preferable? For ingesting such massive amounts of data, it seems a sensible serialization would have minimal divergence between the wire and memory representation, such as Cap'n Proto. Handling composite types seems tricky too -- eg, for prefixing lengths to strings or lists, it appears the extra CPU time for variable-sized integers would be preferable to the I/O overhead of billions of wasted bytes that come with a fixed-size prefix. Assuming explicit begin/end delimiters aren't even an option here.
- jandrewrogers 12y ago
  Cap'n'Proto is actually pretty good example of a modern, high-performance serialization. I've been using it as a template for IoT wire representation discussions. Fast wire encodings are almost universally TLV ("tag-length-value") style serializations. Delimiter scanning is inefficient and also means the parser has little ability to predict what will be coming over the wire so that it can optimize processing. While older serializations tend to be byte oriented, newer formats use word-sized "frames" (even if not aligned) to enable nearly branchless, parallel processing of the bytes in the stream using bit-twiddling techniques or vector instructions.

zwieback 12y ago

Seems a little early to complain about lack of protocol standards when the underlying transport and topologies aren't even converging yet. It seems easy enough for anything that supports conventional protocols like IP but a lot of future networked devices will be using NFC-like things or mesh networks.

viggity 12y ago

I work/consult for a startup doing Power Over Ethernet Lighting (www.igor-tech.com). We use UPnP (universal plug-n-play) to communicate with lights as there is already an open spec. FWIW, we're also releasing an open reference design for hardware.

legulere 12y ago

The alternative usually is design by committee, which leads to huge ugly standards. I'm also not so sure wether the robustness principle really helps. With HTML we had lots of problems because everybody parsed non-standard files differently.