21 ms·

Reality Editor – MIT Media Lab

201 points by piyushmakhija 11y ago

volaski 11y ago
Someone please enlighten me, maybe I'm missing something, but I don't get why this augmented reality interface is better than having a centralized web page where I can manage them using the good old textual interface. It feels like a very convoluted rube goldberg machine approach to me. For example, the guy says something like "the car knows because I connected my chair to my car." Imagining how he would have done this makes me doubt the convenience. I can imagine he would have started dragging when he left the office and kept his thumb on the screen until he got to the car and finally connected it to the knob. He could have just opened a web page and connected these two devices without having to go through all this trouble.
- hackuser 11y ago
  The challenge of innovation is to see the possibilities. That's what makes it hard; anyone can see potential flaws - if that's what it took everyone would do it. I'd much prefer that we focus on the former.
  - volaski 11y ago
    What new possibility does this bring to the table? Connected devices concept has been around like forever. Sure it looks cool (like flying cars, or those impractical UIs from minority report) but how does this make things more convenient? I just was put off by how this guy makes it sound like he came up with all these concepts. "Imagine you could ... " ==> all the things he mentioned with this phrase are nothing new. The only thing that's novel in the video is the UI and I was pointing out how he talks as if it's his new UI that makes all these imaginations possible, but in fact in my opinion it's actually worse than existing solutions (just looks cooler).
    - hackuser 11y ago
      I don't see what value that adds to the discussion; why do I care?
    - thetmkay 11y ago
      The UI is the pitch - technology people can't use is useless to people (tautology). It's a commercial-like video to attract attention - but fair enough if it doesn't impress you. It's hard for me to judge UI without using it myself. It's cool that they're at least trying to solve the problem of a better UI because I believe that's a problem that exists. It reminds me of one of Brett Victor's talks.
- roymurdock 11y ago
  This is important because it represents the next step in allowing regular, non-technical (but increasingly tech-familiar) consumers to set up and maintain their own networks and systems. Setting up a webpage to control a network of lights, monitors, garage doors, thermostats, cameras, etc. might seem simple to you and to most of HN, because many here are familiar with the hardware, the languages, the ISA's, the basic device networking principles and best-practices that make these things work. You are in the very small minority of the 300m people in the US and the 7bn people in the world. For everyone else, visually dragging and dropping will be a much easier way to visualize and set up networks. This product is not for you - it's for the vastly larger market outside of computer scientists/embedded engineers. When you can demonstrate this value prop to the mass consumer market, under the MIT brand nonetheless, big companies like Audi will start to pay attention because they see an avenue to market an increasingly heterogeneous product (luxury sedan) to the rich, older folks who buy these things based on cool new features - automated avoidance, tying volume into acceleration, "Intelligent Drive" [1], projected HUD [2], etc. As we see from the video, Audi has granted this researcher access to APIs that would be completely inaccessible otherwise. This is another key - API access. Could you control the features in your car through a webpage? No, because you don't have access to your windows or your stereo, unless you do some intensive and illegal hacking. M2M and IoT has been contained mostly to the realm of industrial/auto applications so far. So once big companies start to see the consumer market opening up through a simpler, more intuitive interface, they might start opening up APIs to new machines and products. Of course, allowing access to these APIs opens up a whole new can of safety worms...but that's going to be an increasingly large issue that we will have to deal with as we continue to integrate software and connectivity into new products. [1] https://www.mbusa.com/mercedes/technology/videos/detail/title-safety/videoId-fc0835ab8d127410VgnVCM100000ccec1e35RCRD https://www.mbusa.com/mercedes/technology/videos/detail/titl... [2] https://www.youtube.com/watch?v=sg3mmfVILRA https://www.youtube.com/watch?v=sg3mmfVILRA
  - volaski 11y ago
    I think you're assuming a lot of things about mainstream users. If AR was that intuitive, it would have already been mainstream. Many apps tried to port existing good old text based interface into AR but none have yet to gain traction. You need to remember the mainstream users do know how to use computer and do know how to access websites and enter forms. But they do not know how this geeky augmented reality technology works. Also my point here was not criticizing AR itself, I'm just saying his approach makes things needlessly convoluted for the sake of making it look cooler. Read my comment about how someone would drag and drop his chair in his office to his car.
- hanniabu 11y ago
  It's so you can use the 'codes' presented on the objects and create these connections live. There's less steps and it's much more fluid than taking a picture of this code, saving that object, clicking on a features dropdown, and selecting another objects feature you want this to be attached to.
  - volaski 11y ago
    I would argue this was on the wrong track the moment it started out with QR code. "Why don't we bring two geekiest technologies--QR code and Augmented Reality--together? Sure! That will make it mainstream!"
- wangii 11y ago
  I think the AR UI in the video is pretty cool. The trouble is, AR itself hasn't taken off. My theory is people really dislike reality. Reality is ugly, it's dirty, and when people operate upon information, they(we) do want abstraction over reality, want simplicity over complexity, desire focus over distractions.
  - thetmkay 11y ago
    What AR technology exists, and how good is it really? I think the big thing for AR is hands-free/HUD devices. There are niche use cases for phones, but they're described as CV problems rather than AR.
- blhack 11y ago
  I actually agree with you, I think that this is really interesting, but I don't really think this is much more useful than: http://nodered.org/ http://nodered.org/ However, the reason that this is interesting to people is that it makes device discovery easy, and effectively makes the node-red interface builder available as an AR thing.
veli_joza 11y ago
Did they invent that labyrinth pattern just for this, or is it an existing technology?
- jlrubin 11y ago
  I'm pretty sure Valentin also invented it, but perhaps as a separate project and is using it for this.
  - mkeeter 11y ago
    The homepage for the QR-ish codes is here: http://hrqr.org/ http://hrqr.org/ It was an earlier project by the same researcher.
- valentinheun 11y ago
  We use natural feature tracking. The pattern is really just because of a design choice. It can be very kind of visual structure.
RobertoG 11y ago
A knob that changes functions.. I feel old fashioned saying this but.. I think I prefer my objects stateless.
kapad 11y ago
I think with connected devices, a central machine that figures out the patterns for me and others in my house and then automagically creates the patters (the drawing of lines on the app) would be way cooler.
hackuser 11y ago
Do I understand this correctly? It's exposing the functionalites of IoT devices as advertised services (IoTaaS?), allowing them to be linked together (e.g. the timer from the microwave and the lightswitch), and provides an intuitive interface that, hopefully, end users might utilize. Exposing the IoT functions as services seems brilliant to me (has it been discussed before?). What will we end up with? A overly redundent mess of IoT functions (e.g., how many devices have clocks)? Or maybe an IoT server on each home/office/car/phone (or more likely, in the cloud) that provides a central source for basic functions like timers, monitors, etc.?
- pidg 11y ago
  This desperately needs someone who can write in plain English to actually explain it without the MIT imagineering jargon. The video helps somewhat, but I suspect if someone described this in plain English it'd seem a lot less exciting than they want it to be.
  - thetmkay 11y ago
    Imagine you had a switch next to your bed. Most likely, the switch controls the lights, which is perfect when you want to turn them off and go to sleep. But maybe when you wake up, you'd actually want it to open the windows, or start the coffee maker. And if someone else comes in to your room, maybe they'd prefer it if it started the audio system. If we could manipulate all the physical objects around us with software, can we produce an environment which we can manipulate not just physically, but also electronically, to map our intents and needs as we have them? And if we could, how would we go about "reprogramming" the world around us? It's the ultimate vision of AR and "IOT".
- valentinheun 11y ago
  It goes beyond that. The interesting thing is that in the moment you have a visual representation for an object, you can break it down into all its components. From that moment on the abstract object is represented by numbers only. You can read more about it here: http://openhybrid.org/how-to-connect-everything.html http://openhybrid.org/how-to-connect-everything.html The interfaces them selfs are web pages.
dfar1 11y ago
The idea that everything can be integrated via AR is fantastic... it's the next step from IFTT. The challenge is replacing everything you have with IOT versions that can be connected to these apps and hoping those things won't go obsolete in a couple of years. Technology is moving so fast, not everyone can catch up, thus there are no standards.
- valentinheun 11y ago
  We designed the platform behind the Reality Editor in such a way that it can adopt. You can read more about it here: http://openhybrid.org/how-to-connect-everything.html http://openhybrid.org/how-to-connect-everything.html If you can run nodejs in your object, you will be able to make it in to a Hybrid Object the Reality Editor can talk to. The inspiration is the World Wide Web. It was created with a clear simple and open foundation. You can still read the first webpages ever made. You should be able to still interact with the first Hybrid Objects ever made.
wangii 11y ago
I'm not sold. The whole IoT thing fails miserably in story-telling. Until now, I have yet to read/hear/watch one compelling example. Who really cares if my chair is connected with the light so that I can get the best light shedding angle? who cares if my refrigerator is connect with my micro-wave to de-freeze meat according to my online menu? Maybe I'm too old for this. There are millions of important things to be solved by computer and internet, yet big-names like Google and MIT push this typical high cost, low return technology, for what? to sell chips and standards to factories?
- valentinheun 11y ago
  I agree. At this point it really makes no sense to use this. But hey people said that about the internet back then. ;-)
  - wangii 11y ago
    The big deal of the internet is it connects people. Before AI takes over the planet, or invasion of alien, the most powerful thing is the communication amongst human beings, its implication we have not fully explored.
  - 20centuryboys 11y ago
    Comparing that to the internet? Don't even go there!
    - valentin_heun 11y ago
      Why not? The Reality Editor is an equivalent of a Web Browser for the Physical space. We are spatial oriented creatures operating the world around us with muscle memory and physical tools. The real power of technology comes when we combine digital and physical tools seamlessly. What is in front of us will change the World around you even more then the Internet has done it up to this point. How do you want to engage with all of the self driving cars and digital/physical content and objects around you? Browse through 1000 apps with all the drop down menus? Who wants to remember all of these physical<->digital links? My question for that future now is: What will be the digital equivalent of a screwdriver in that future? Thats what the Reality Editor is! A powerful tool.
valentinheun 11y ago
Hi its Valentin, the guy in the video. Let me know if you have any questions. :-)
- Arrygoo 11y ago
  As most people have pointed out, this doesn't seem like anything new. If all of those devices are IoT, you can have an app that controls them. Seems pretty pointless to have to use your camera and drag between objects. You could achieve this much easier with icons in an app. Do you personally think this a good product?
  - drcode 11y ago
    It's useful for the same reasons that a 4yo can use the touch screen on an iPad but can't use the track pad on a computer: It removes one additional step of indirection and thereby reduces cognitive load.
  - valentin_heun 11y ago
    How would you connect two objects in physical space with your app centric interface? This is not a trivial task. Its very complex and has a lot of abstract steps. Steps you wold not be able to explain a 8 year old child in 5 minutes. We gave the Reality Editor in to the hands of a 8 year old child and we did not even need to explain how to do it. He just ran with it.
    - deleted 11y ago
      [deleted]
    - xyproto 11y ago
      Try to make him do it while watching a 5 year old and an 8 year old while entering a car. It's a more realistic scenario for when you need a knob to work as expected.
      - valentinheun 11y ago
        That is a good scenario. What you see in the video and on open hybrid is really as much as we can put in to code at this point and I believe that it is a very good beginning. There is so much more that we want to implement and some of it would give answers for your situation. I think the Reality Editor will grow incremental and every version will be a bit better then the version before. In some years, when your car actually has this functionality, your scenario will be solved.
microcolonel 11y ago
I think the more valuable contribution is not Reality Editor, but Open Hybrid. Right now there are many crappy home automation standards, and none/few outside the home. Also, HRQR just looks awesome.
- valentinheun 11y ago
  True and not true. Open Hybrid is not useful without the visual representation that breaks down the object in to its components. Short write up about it here: http://openhybrid.org/how-to-connect-everything.html http://openhybrid.org/how-to-connect-everything.html The problem of all other standards is that they have to think about complex abstract standards.
cushychicken 11y ago
Valentin's idea has a lot of interesting components to it. The ability to stitch together a few different physical interfaces ad hoc using the camera + smartphone combination is particularly brilliant. I like where his head's at: integrating a bunch of discrete IoT devices should be simple, simple, simple. My main beef? Frankly, it's those stickers that the camera needs to recognize the device. Ugly, ugly, ugly.
- valentin_heun 11y ago
  Computers have been huge (ugly?) machines before now they are sexy and small. Screens have been heavy, now its all in your pocket. All of it has been in the Marked and sold wonderfully. Visual Markers (Stickers) for Augmented Reality are a design challenge that can lead to new looks. Its Natural Feature tracking, so everything can be a "sticker". This can actually help companies to differentiate from each other. Its something new to think about. I would not say that a radio with just a touch screen is particular beautiful, but it sells too. ;-) Anyhow, one thing is sure we will get rid of the marker. There is so much technological development going on in that field.
  - cushychicken 11y ago
    Thanks for answering! I really like the comparison to a "digital screwdriver". I think way too many companies set the setup bar for their IoT devices way too high. If you can make it as easy as Reality Editor, you should!
bikamonki 11y ago
Maybe I did not get the example? I would not need to get up to turn off the light b/c I can use a trigger like touching something on my night table. But if that something and the light switch are somehow networked, can't I just use my phone and some app to directly turn it off? Same thing with the timer example: isn't my phone already capable of running all sorts of abstractions? Why would I need to borrow the timer function from the tv? Finally, as far as discovering objects and its capabilities, wifi + some sort of IoT protocol should do, right? No need to point a camera, this gadgets will always be broadcasting its API (and ACLs I hope). Sorry, maybe I did not get the product at all.
leebz 11y ago
Are you going to build The Reality Editor for Android any time soon?
johann28 11y ago
Most people don't want to reprogram stuff, they don't want to customize stuff. They want things to "just work", they don't want to choose which button controls the car windows and which knob controls the bass. This is the job of the designer. Most people want a finished product. People don't want to "rewire" their products, they don't want to hack on this kind of thing. Except for geeks and technical people, who can also handle normal apps and interfaces. The analogy with the Internet breaks down because the Internet is mainly successful because it connects people. That's what people care about, other people. The telephone was a success because it lets you talk to other people. Unless this thing helps people deal with other people, it's too complex and uninteresting for the masses and probably too simple and tedious for technical people.
- valentinheun 11y ago
  I disagree! The internet is successful because authoring and publishing became as easy as consuming. The first web browser was an editor and a browser in the same time. The most successful webpages are those where people are authors. People want tools that empower them to engage with their environment. To say people don't want to "rewire" their products is like saying that you don't want to connect your Electric Guitar with a wire to the Amplifier.
  - johann28 11y ago
    > The most successful webpages are those where people are authors. Exactly. But not because they enjoy tweaking some website settings or enjoy the control over the layout of a page. No, the reason is connection with other people. They write a blog so that they get feedback from people. They post pictures to Instagram and Facebook to get likes and show off. Connecting the electric guitar and repairing things with screwdrivers is seen by most people as a chore. Sure, techie, geeky tinkerers enjoy messing around with things, the same ones who enjoy configuring linux, for example. I just haven't seen a single use case that would make sense in an everyday setting. Having a knob that controls the lamp? Well, lamps already have that, without being radio controlled. If a car dashboard is designed well, then there's no need to remap the buttons. Very few people like to customize their stuff that much. If there is a trend in software (and hardware) it's that they remove options. We have less features and customizations than 10 years ago. Developers realized that people just mess their settings up and they call support. Also, the more options there are, the more bugs there will be. So now instead of supporting a myriad of settings, they just go with a default and leave it at that. Why don't we have the option of vertical browser tab arrangement in Chrome? Well, because they decided it's too complex and unnecessary for most users. People just want to get their things done. They want to get business things done with other business people and they want to have fun with their friends. The success of the Internet is all about human communication, sharing experiences (mainly through photos), and gossip.
xyproto 11y ago
"We change, but interfaces haven't, therefore we need to change the interfaces" is a flawed argument. Change has a cost. The interfaces may be good enough. The new interfaces may be worse. Change may result in many standards where there used to be one. Change is fresh and nice in other ways, but to me, the foundation for this research seems to be more along the lines of "Glorified knobs for everyone!", which is cool in its own right.
giancarlostoro 11y ago
Thanks for an interesting project shared. I am sharing this at my college, some share concerns about adaptability, but I feel like as this project expands it will feel seamless. I am glad it is so readily available and already showing support for newer boards like the Raspberry Pi Zero. Thank you for your work, I hope to see this and similar projects expand!
Puriney 11y ago
Tune a .vimrc of my own life.
plg 11y ago
Why don't you just manipulate the physical objects with your hands? They're right in front of you. I think I am missing something.
lhh 11y ago
Seems like a lot of people aren't really convinced of the value of this kind of tech (and maybe of IoT in general), and I'll admit that I'm a little skeptical of parts of it as well. I don't really see the value in linking devices to each other (eg I don't need my shower to automatically turn on after my alarm goes off), and I don't need to be able to read HN on my fridge, but I do think there's a lot of value in making the smart phone a remote control for the physical world. I think it'd be awesome if none of my devices (thermostat, TV, TV peripherals, lights, door lock, sound system, etc) needed their own physical interfaces but I could just control them all with my phone. My phone is already my remote control for a bunch of other things, like transportation (Uber), food (Seamless), money (Venmo), shipping stuff (Shyp), love (Tinder - just kidding?), etc, so why not have it control my physical devices too?
- jkestner 11y ago
  > I think it'd be awesome if none of my devices (thermostat, TV, TV peripherals, lights, > door lock, sound system, etc) needed their own physical interfaces but I could just > control them all with my phone. Awesome, maybe, but why? This is IoT in a nutshell. Let's add connectivity where there is articulable value, and where it actually makes the core task simpler. Usually, adding a remote interface makes things more complex even as it adds functionality. So have a good reason.
- onion2k 11y ago
  Agreed. I really don't understand people who are against this sort of research. A light switch will still work as a light switch does now if you don't program it. There's another layer of complexity in the product design to go wrong, and add cost, but so long as when things break they still work in "dumb" mode and the cost added is minimal then a smart switch that you can program is a win for those people who want it and has practically no effect on those who don't. Being anti-IoT seems like sour grapes; thinking "I don't want this in my life so no one should have it" is completely irrational.
  - kapad 11y ago
    I agree. This is just research, and not a market ready product. And research is really necessary in IoT, not just to figure out what is useful and what isn't, but also to figure out security and privacy concerns with everything being connected.
intrasight 11y ago
I see object with beautiful tactile affordances being turned into tactile-free and afforance-free touch interfaces. What's the point of that?
devin 11y ago
Anyone else reminded of http://reactable.com/ http://reactable.com/ (Reactable) at all?
pnathan 11y ago
I'm a nerd. What's the command line API for this? I really don't want to be screwing with remote 2.0; i'd rather write my own programs and call the actions on these objects.
politician 11y ago
This is an interesting project that practically begs for the availability of smart, pluggable, physical control surfaces able to be reprogrammed using this technique. Integration with Nest-style devices seems like an obvious first choice for a consumer product. On maintenance: It seems like there'd be a bit of difficulty around managing a nest of "wires" between devices that are only visible under a 5" smartphone glass. So, you'd want something head-mounted (glasses), preferably with zoom. Microsoft HoloLens? That said, you'd definitely need an abstract schematic view to manage a dense network. It would be a really curious development if, in addition to plumbers and electricians, new construction required "reality editors" to wire up behaviors between such interfaces. "This won't do; your reality isn't up to code." Final thought: lose the digital tattoos and integrate Bluetooth/Zigby beacons for device advertisement. There's no way that those patterns are going to become fashionable. Or, use something else that's easy to print but is outside of our visual range.
adamredwoods 11y ago
Wow, imagine this applied to manufacturing or assembly lines!
las3rprint3r 11y ago
This is awesome. Being a coder I can see myself writing IoT services without a problem (just simple REST apis). And who cares about the masses? Code or be coded is what I say. However, tried using the iPhone app and couldn't get it to work. Made me feel dumb :( Be nice if there was a help me in the app or a tutorial on how to hook phone up to other devices.

201 points by piyushmakhija 11y ago

Someone please enlighten me, maybe I'm missing something, but I don't get why this augmented reality interface is better than having a centralized web page where I can manage them using the good old textual interface. It feels like a very convoluted rube goldberg machine approach to me. For example, the guy says something like "the car knows because I connected my chair to my car." Imagining how he would have done this makes me doubt the convenience. I can imagine he would have started dragging when he left the office and kept his thumb on the screen until he got to the car and finally connected it to the knob. He could have just opened a web page and connected these two devices without having to go through all this trouble.

hackuser 11y ago
The challenge of innovation is to see the possibilities. That's what makes it hard; anyone can see potential flaws - if that's what it took everyone would do it. I'd much prefer that we focus on the former.
- volaski 11y ago
  What new possibility does this bring to the table? Connected devices concept has been around like forever. Sure it looks cool (like flying cars, or those impractical UIs from minority report) but how does this make things more convenient? I just was put off by how this guy makes it sound like he came up with all these concepts. "Imagine you could ... " ==> all the things he mentioned with this phrase are nothing new. The only thing that's novel in the video is the UI and I was pointing out how he talks as if it's his new UI that makes all these imaginations possible, but in fact in my opinion it's actually worse than existing solutions (just looks cooler).
  - hackuser 11y ago
    I don't see what value that adds to the discussion; why do I care?
  - thetmkay 11y ago
    The UI is the pitch - technology people can't use is useless to people (tautology). It's a commercial-like video to attract attention - but fair enough if it doesn't impress you. It's hard for me to judge UI without using it myself. It's cool that they're at least trying to solve the problem of a better UI because I believe that's a problem that exists. It reminds me of one of Brett Victor's talks.
roymurdock 11y ago
This is important because it represents the next step in allowing regular, non-technical (but increasingly tech-familiar) consumers to set up and maintain their own networks and systems. Setting up a webpage to control a network of lights, monitors, garage doors, thermostats, cameras, etc. might seem simple to you and to most of HN, because many here are familiar with the hardware, the languages, the ISA's, the basic device networking principles and best-practices that make these things work. You are in the very small minority of the 300m people in the US and the 7bn people in the world. For everyone else, visually dragging and dropping will be a much easier way to visualize and set up networks. This product is not for you - it's for the vastly larger market outside of computer scientists/embedded engineers. When you can demonstrate this value prop to the mass consumer market, under the MIT brand nonetheless, big companies like Audi will start to pay attention because they see an avenue to market an increasingly heterogeneous product (luxury sedan) to the rich, older folks who buy these things based on cool new features - automated avoidance, tying volume into acceleration, "Intelligent Drive" [1], projected HUD [2], etc. As we see from the video, Audi has granted this researcher access to APIs that would be completely inaccessible otherwise. This is another key - API access. Could you control the features in your car through a webpage? No, because you don't have access to your windows or your stereo, unless you do some intensive and illegal hacking. M2M and IoT has been contained mostly to the realm of industrial/auto applications so far. So once big companies start to see the consumer market opening up through a simpler, more intuitive interface, they might start opening up APIs to new machines and products. Of course, allowing access to these APIs opens up a whole new can of safety worms...but that's going to be an increasingly large issue that we will have to deal with as we continue to integrate software and connectivity into new products. [1] https://www.mbusa.com/mercedes/technology/videos/detail/title-safety/videoId-fc0835ab8d127410VgnVCM100000ccec1e35RCRD https://www.mbusa.com/mercedes/technology/videos/detail/titl... [2] https://www.youtube.com/watch?v=sg3mmfVILRA https://www.youtube.com/watch?v=sg3mmfVILRA
- volaski 11y ago
  I think you're assuming a lot of things about mainstream users. If AR was that intuitive, it would have already been mainstream. Many apps tried to port existing good old text based interface into AR but none have yet to gain traction. You need to remember the mainstream users do know how to use computer and do know how to access websites and enter forms. But they do not know how this geeky augmented reality technology works. Also my point here was not criticizing AR itself, I'm just saying his approach makes things needlessly convoluted for the sake of making it look cooler. Read my comment about how someone would drag and drop his chair in his office to his car.
hanniabu 11y ago
It's so you can use the 'codes' presented on the objects and create these connections live. There's less steps and it's much more fluid than taking a picture of this code, saving that object, clicking on a features dropdown, and selecting another objects feature you want this to be attached to.
- volaski 11y ago
  I would argue this was on the wrong track the moment it started out with QR code. "Why don't we bring two geekiest technologies--QR code and Augmented Reality--together? Sure! That will make it mainstream!"
wangii 11y ago
I think the AR UI in the video is pretty cool. The trouble is, AR itself hasn't taken off. My theory is people really dislike reality. Reality is ugly, it's dirty, and when people operate upon information, they(we) do want abstraction over reality, want simplicity over complexity, desire focus over distractions.
- thetmkay 11y ago
  What AR technology exists, and how good is it really? I think the big thing for AR is hands-free/HUD devices. There are niche use cases for phones, but they're described as CV problems rather than AR.
blhack 11y ago
I actually agree with you, I think that this is really interesting, but I don't really think this is much more useful than: http://nodered.org/ http://nodered.org/ However, the reason that this is interesting to people is that it makes device discovery easy, and effectively makes the node-red interface builder available as an AR thing.

veli_joza 11y ago

Did they invent that labyrinth pattern just for this, or is it an existing technology?

jlrubin 11y ago
I'm pretty sure Valentin also invented it, but perhaps as a separate project and is using it for this.
- mkeeter 11y ago
  The homepage for the QR-ish codes is here: http://hrqr.org/ http://hrqr.org/ It was an earlier project by the same researcher.
valentinheun 11y ago
We use natural feature tracking. The pattern is really just because of a design choice. It can be very kind of visual structure.

RobertoG 11y ago

A knob that changes functions.. I feel old fashioned saying this but.. I think I prefer my objects stateless.

kapad 11y ago

I think with connected devices, a central machine that figures out the patterns for me and others in my house and then automagically creates the patters (the drawing of lines on the app) would be way cooler.

hackuser 11y ago

Do I understand this correctly? It's exposing the functionalites of IoT devices as advertised services (IoTaaS?), allowing them to be linked together (e.g. the timer from the microwave and the lightswitch), and provides an intuitive interface that, hopefully, end users might utilize. Exposing the IoT functions as services seems brilliant to me (has it been discussed before?). What will we end up with? A overly redundent mess of IoT functions (e.g., how many devices have clocks)? Or maybe an IoT server on each home/office/car/phone (or more likely, in the cloud) that provides a central source for basic functions like timers, monitors, etc.?

pidg 11y ago
This desperately needs someone who can write in plain English to actually explain it without the MIT imagineering jargon. The video helps somewhat, but I suspect if someone described this in plain English it'd seem a lot less exciting than they want it to be.
- thetmkay 11y ago
  Imagine you had a switch next to your bed. Most likely, the switch controls the lights, which is perfect when you want to turn them off and go to sleep. But maybe when you wake up, you'd actually want it to open the windows, or start the coffee maker. And if someone else comes in to your room, maybe they'd prefer it if it started the audio system. If we could manipulate all the physical objects around us with software, can we produce an environment which we can manipulate not just physically, but also electronically, to map our intents and needs as we have them? And if we could, how would we go about "reprogramming" the world around us? It's the ultimate vision of AR and "IOT".
valentinheun 11y ago
It goes beyond that. The interesting thing is that in the moment you have a visual representation for an object, you can break it down into all its components. From that moment on the abstract object is represented by numbers only. You can read more about it here: http://openhybrid.org/how-to-connect-everything.html http://openhybrid.org/how-to-connect-everything.html The interfaces them selfs are web pages.

dfar1 11y ago

The idea that everything can be integrated via AR is fantastic... it's the next step from IFTT. The challenge is replacing everything you have with IOT versions that can be connected to these apps and hoping those things won't go obsolete in a couple of years. Technology is moving so fast, not everyone can catch up, thus there are no standards.

valentinheun 11y ago
We designed the platform behind the Reality Editor in such a way that it can adopt. You can read more about it here: http://openhybrid.org/how-to-connect-everything.html http://openhybrid.org/how-to-connect-everything.html If you can run nodejs in your object, you will be able to make it in to a Hybrid Object the Reality Editor can talk to. The inspiration is the World Wide Web. It was created with a clear simple and open foundation. You can still read the first webpages ever made. You should be able to still interact with the first Hybrid Objects ever made.

wangii 11y ago

I'm not sold. The whole IoT thing fails miserably in story-telling. Until now, I have yet to read/hear/watch one compelling example. Who really cares if my chair is connected with the light so that I can get the best light shedding angle? who cares if my refrigerator is connect with my micro-wave to de-freeze meat according to my online menu? Maybe I'm too old for this. There are millions of important things to be solved by computer and internet, yet big-names like Google and MIT push this typical high cost, low return technology, for what? to sell chips and standards to factories?

valentinheun 11y ago
I agree. At this point it really makes no sense to use this. But hey people said that about the internet back then. ;-)
- wangii 11y ago
  The big deal of the internet is it connects people. Before AI takes over the planet, or invasion of alien, the most powerful thing is the communication amongst human beings, its implication we have not fully explored.
- 20centuryboys 11y ago
  Comparing that to the internet? Don't even go there!
  - valentin_heun 11y ago
    Why not? The Reality Editor is an equivalent of a Web Browser for the Physical space. We are spatial oriented creatures operating the world around us with muscle memory and physical tools. The real power of technology comes when we combine digital and physical tools seamlessly. What is in front of us will change the World around you even more then the Internet has done it up to this point. How do you want to engage with all of the self driving cars and digital/physical content and objects around you? Browse through 1000 apps with all the drop down menus? Who wants to remember all of these physical<->digital links? My question for that future now is: What will be the digital equivalent of a screwdriver in that future? Thats what the Reality Editor is! A powerful tool.

valentinheun 11y ago

Hi its Valentin, the guy in the video. Let me know if you have any questions. :-)

Arrygoo 11y ago
As most people have pointed out, this doesn't seem like anything new. If all of those devices are IoT, you can have an app that controls them. Seems pretty pointless to have to use your camera and drag between objects. You could achieve this much easier with icons in an app. Do you personally think this a good product?
- drcode 11y ago
  It's useful for the same reasons that a 4yo can use the touch screen on an iPad but can't use the track pad on a computer: It removes one additional step of indirection and thereby reduces cognitive load.
- valentin_heun 11y ago
  How would you connect two objects in physical space with your app centric interface? This is not a trivial task. Its very complex and has a lot of abstract steps. Steps you wold not be able to explain a 8 year old child in 5 minutes. We gave the Reality Editor in to the hands of a 8 year old child and we did not even need to explain how to do it. He just ran with it.
  - deleted 11y ago
    [deleted]
  - xyproto 11y ago
    Try to make him do it while watching a 5 year old and an 8 year old while entering a car. It's a more realistic scenario for when you need a knob to work as expected.
    - valentinheun 11y ago
      That is a good scenario. What you see in the video and on open hybrid is really as much as we can put in to code at this point and I believe that it is a very good beginning. There is so much more that we want to implement and some of it would give answers for your situation. I think the Reality Editor will grow incremental and every version will be a bit better then the version before. In some years, when your car actually has this functionality, your scenario will be solved.

microcolonel 11y ago

I think the more valuable contribution is not Reality Editor, but Open Hybrid. Right now there are many crappy home automation standards, and none/few outside the home. Also, HRQR just looks awesome.

valentinheun 11y ago
True and not true. Open Hybrid is not useful without the visual representation that breaks down the object in to its components. Short write up about it here: http://openhybrid.org/how-to-connect-everything.html http://openhybrid.org/how-to-connect-everything.html The problem of all other standards is that they have to think about complex abstract standards.

cushychicken 11y ago

Valentin's idea has a lot of interesting components to it. The ability to stitch together a few different physical interfaces ad hoc using the camera + smartphone combination is particularly brilliant. I like where his head's at: integrating a bunch of discrete IoT devices should be simple, simple, simple. My main beef? Frankly, it's those stickers that the camera needs to recognize the device. Ugly, ugly, ugly.

valentin_heun 11y ago
Computers have been huge (ugly?) machines before now they are sexy and small. Screens have been heavy, now its all in your pocket. All of it has been in the Marked and sold wonderfully. Visual Markers (Stickers) for Augmented Reality are a design challenge that can lead to new looks. Its Natural Feature tracking, so everything can be a "sticker". This can actually help companies to differentiate from each other. Its something new to think about. I would not say that a radio with just a touch screen is particular beautiful, but it sells too. ;-) Anyhow, one thing is sure we will get rid of the marker. There is so much technological development going on in that field.
- cushychicken 11y ago
  Thanks for answering! I really like the comparison to a "digital screwdriver". I think way too many companies set the setup bar for their IoT devices way too high. If you can make it as easy as Reality Editor, you should!

bikamonki 11y ago

Maybe I did not get the example? I would not need to get up to turn off the light b/c I can use a trigger like touching something on my night table. But if that something and the light switch are somehow networked, can't I just use my phone and some app to directly turn it off? Same thing with the timer example: isn't my phone already capable of running all sorts of abstractions? Why would I need to borrow the timer function from the tv? Finally, as far as discovering objects and its capabilities, wifi + some sort of IoT protocol should do, right? No need to point a camera, this gadgets will always be broadcasting its API (and ACLs I hope). Sorry, maybe I did not get the product at all.

leebz 11y ago

Are you going to build The Reality Editor for Android any time soon?

johann28 11y ago

Most people don't want to reprogram stuff, they don't want to customize stuff. They want things to "just work", they don't want to choose which button controls the car windows and which knob controls the bass. This is the job of the designer. Most people want a finished product. People don't want to "rewire" their products, they don't want to hack on this kind of thing. Except for geeks and technical people, who can also handle normal apps and interfaces. The analogy with the Internet breaks down because the Internet is mainly successful because it connects people. That's what people care about, other people. The telephone was a success because it lets you talk to other people. Unless this thing helps people deal with other people, it's too complex and uninteresting for the masses and probably too simple and tedious for technical people.

valentinheun 11y ago
I disagree! The internet is successful because authoring and publishing became as easy as consuming. The first web browser was an editor and a browser in the same time. The most successful webpages are those where people are authors. People want tools that empower them to engage with their environment. To say people don't want to "rewire" their products is like saying that you don't want to connect your Electric Guitar with a wire to the Amplifier.
- johann28 11y ago
  > The most successful webpages are those where people are authors. Exactly. But not because they enjoy tweaking some website settings or enjoy the control over the layout of a page. No, the reason is connection with other people. They write a blog so that they get feedback from people. They post pictures to Instagram and Facebook to get likes and show off. Connecting the electric guitar and repairing things with screwdrivers is seen by most people as a chore. Sure, techie, geeky tinkerers enjoy messing around with things, the same ones who enjoy configuring linux, for example. I just haven't seen a single use case that would make sense in an everyday setting. Having a knob that controls the lamp? Well, lamps already have that, without being radio controlled. If a car dashboard is designed well, then there's no need to remap the buttons. Very few people like to customize their stuff that much. If there is a trend in software (and hardware) it's that they remove options. We have less features and customizations than 10 years ago. Developers realized that people just mess their settings up and they call support. Also, the more options there are, the more bugs there will be. So now instead of supporting a myriad of settings, they just go with a default and leave it at that. Why don't we have the option of vertical browser tab arrangement in Chrome? Well, because they decided it's too complex and unnecessary for most users. People just want to get their things done. They want to get business things done with other business people and they want to have fun with their friends. The success of the Internet is all about human communication, sharing experiences (mainly through photos), and gossip.

xyproto 11y ago

"We change, but interfaces haven't, therefore we need to change the interfaces" is a flawed argument. Change has a cost. The interfaces may be good enough. The new interfaces may be worse. Change may result in many standards where there used to be one. Change is fresh and nice in other ways, but to me, the foundation for this research seems to be more along the lines of "Glorified knobs for everyone!", which is cool in its own right.

giancarlostoro 11y ago

Thanks for an interesting project shared. I am sharing this at my college, some share concerns about adaptability, but I feel like as this project expands it will feel seamless. I am glad it is so readily available and already showing support for newer boards like the Raspberry Pi Zero. Thank you for your work, I hope to see this and similar projects expand!

Puriney 11y ago

Tune a .vimrc of my own life.

plg 11y ago

Why don't you just manipulate the physical objects with your hands? They're right in front of you. I think I am missing something.

lhh 11y ago

Seems like a lot of people aren't really convinced of the value of this kind of tech (and maybe of IoT in general), and I'll admit that I'm a little skeptical of parts of it as well. I don't really see the value in linking devices to each other (eg I don't need my shower to automatically turn on after my alarm goes off), and I don't need to be able to read HN on my fridge, but I do think there's a lot of value in making the smart phone a remote control for the physical world. I think it'd be awesome if none of my devices (thermostat, TV, TV peripherals, lights, door lock, sound system, etc) needed their own physical interfaces but I could just control them all with my phone. My phone is already my remote control for a bunch of other things, like transportation (Uber), food (Seamless), money (Venmo), shipping stuff (Shyp), love (Tinder - just kidding?), etc, so why not have it control my physical devices too?

jkestner 11y ago
> I think it'd be awesome if none of my devices (thermostat, TV, TV peripherals, lights, > door lock, sound system, etc) needed their own physical interfaces but I could just > control them all with my phone. Awesome, maybe, but why? This is IoT in a nutshell. Let's add connectivity where there is articulable value, and where it actually makes the core task simpler. Usually, adding a remote interface makes things more complex even as it adds functionality. So have a good reason.
onion2k 11y ago
Agreed. I really don't understand people who are against this sort of research. A light switch will still work as a light switch does now if you don't program it. There's another layer of complexity in the product design to go wrong, and add cost, but so long as when things break they still work in "dumb" mode and the cost added is minimal then a smart switch that you can program is a win for those people who want it and has practically no effect on those who don't. Being anti-IoT seems like sour grapes; thinking "I don't want this in my life so no one should have it" is completely irrational.
- kapad 11y ago
  I agree. This is just research, and not a market ready product. And research is really necessary in IoT, not just to figure out what is useful and what isn't, but also to figure out security and privacy concerns with everything being connected.

intrasight 11y ago

I see object with beautiful tactile affordances being turned into tactile-free and afforance-free touch interfaces. What's the point of that?

devin 11y ago

Anyone else reminded of http://reactable.com/ http://reactable.com/ (Reactable) at all?

pnathan 11y ago

I'm a nerd. What's the command line API for this? I really don't want to be screwing with remote 2.0; i'd rather write my own programs and call the actions on these objects.

politician 11y ago

This is an interesting project that practically begs for the availability of smart, pluggable, physical control surfaces able to be reprogrammed using this technique. Integration with Nest-style devices seems like an obvious first choice for a consumer product. On maintenance: It seems like there'd be a bit of difficulty around managing a nest of "wires" between devices that are only visible under a 5" smartphone glass. So, you'd want something head-mounted (glasses), preferably with zoom. Microsoft HoloLens? That said, you'd definitely need an abstract schematic view to manage a dense network. It would be a really curious development if, in addition to plumbers and electricians, new construction required "reality editors" to wire up behaviors between such interfaces. "This won't do; your reality isn't up to code." Final thought: lose the digital tattoos and integrate Bluetooth/Zigby beacons for device advertisement. There's no way that those patterns are going to become fashionable. Or, use something else that's easy to print but is outside of our visual range.

adamredwoods 11y ago

Wow, imagine this applied to manufacturing or assembly lines!

las3rprint3r 11y ago

This is awesome. Being a coder I can see myself writing IoT services without a problem (just simple REST apis). And who cares about the masses? Code or be coded is what I say. However, tried using the iPhone app and couldn't get it to work. Made me feel dumb :( Be nice if there was a help me in the app or a tutorial on how to hook phone up to other devices.