4 ms·

I love Bret Victor's talks, blog posts, etc. They're super inspiring. 2 things came to mind though. 1. It seems, possibly, the exact wrong time to make rooms

I love Bret Victor's talks, blog posts, etc. They're super inspiring. 2 things came to mind though. 1. It seems, possibly, the exact wrong time to make rooms with giant displays. With things like Google Glass and Oculus Rift as first gen (2nd?) VR/AR you could project all of that info virtually and cheaply and be able to have all the visualization he describes wherever you are, not just at a makerspace that only a few people can use at a time. 2. I'm always super inspired by the Bret's visualizations but when I actually try to figure out how they'd be implemented I'm clearly not smart enough to figure out how that would happen. In this example in particular, he shows graph toward the end where the system tries every setting and graphs the results so it's easy to pick out the best setting. How would that happen? How does the system know what "good" is? It seems to me it can't know that. You'd have to program it which in itself would be pretty hard. Worse, most system have not just one adjustment but many. Just a few and there'd be tens of thousands of variations/combinations to try to figure out "best". I'm not saying we can't get there. Maybe the first step is building a framework that would make it easy to make systems like that with various kinds of visualizers, analysers, time-recording, searching features etc, and maybe somewhere along the way we'd figure out how to automate more of it. I'd love to help work on such a system.

anko 12y ago
I just wanted to make a couple of comments; 1. I think that when something is new, like Google Glass or occulus rift, we (as nerds) tend to think it's the future because it hasn't been possible before before we have quite worked out when it's most applicable. Giant displays are awesome when you're brainstorming a topic with people in the room, which is what maker spaces seem to be about. I can see glass being good for things like surgery, where you need to concentrate your attention on something, but would like some extra data to be easily available. VR might be a substitute for live brainstorming sessions, and it might allow you to visualise in new ways. But latency is going to be killer across remote locations, and that's a big challenge in terms of infrastructure. 2. I'd imagine as these types of techniques become commonplace, we'll have to learn more stats. You could also apply clustering and neural nets to a lot of data sets, but yeah - maybe we'll all have to become statisticians.
vanderZwan 12y ago
> 1. It seems, possibly, the exact wrong time to make rooms with giant displays. With things like Google Glass and Oculus Rift as first gen (2nd?) VR/AR you could project all of that info virtually and cheaply and be able to have all the visualization he describes wherever you are, not just at a makerspace that only a few people can use at a time. I think the reason he wants giant displays is the reason we have giant shared displays in control rooms to: so you can see what others are looking at. Don't forget maker spaces are supposed to be communal spaces where you share ideas. You can learn a lot just by watching someone else use a tool the right way. And then there's of course the collaboration, where shared displays are even more important.
yorak 12y ago
> the system tries every setting and graphs the results so it's easy to pick out the best setting. How would that happen? How does the system know what "good" is? In a research field of Parameter Tuning we try to answer that question. The field is more focused to optimizing algorithm parameters, but it could be applied to physical world of robots etc., if it is feasible to automatically repeat the experiment few dozen to around hundred times. If we would do like Bret proposed, by logging each experiment we would already have done some "probing" on the parameter space. This, in turn, would allow us to build a statistical model of the phenomena and then minimize/maximize on that. The resulting parameter configuration would then be evaluated, the model updated and the process repeated until satisfactory level of performance was reached. See for example the recent works from Hutter et al. [1], where they use Random Forests with parameter tuning to make parameter "goodness" predictions (in order to reduce the actual experiments on the target algoritm/robot/whatever). [1] http://www.cs.ubc.ca/labs/beta/Projects/SMAC/ http://www.cs.ubc.ca/labs/beta/Projects/SMAC/
knowaveragejoe 12y ago
I think display are, for now, the better option. AR and VR are still far too in their infancy for them to be as intuitively useful as a room full of displays is to NASA or a power grid operations team(per his example). That will change with time of course, but for something we can do now and over the next few years I think displays and projectors are the way to go. In a way, the projectors over the worktable he shows are(in effect) a limited form of augmented reality.
- greggman 12y ago
  I guess the question is how many years? How many years until we have the libraries, servers, sensors, visualizers, and other systems in place for make this a reality? Assuming it's 5 years (probably over estimating) where will fb oculus be? If it happens to have become fb oculus glass, something you wear that can augment the world this system of Bret's will come to fruition just as there displays are no longer needed I guess that's irrelevant though as switching displays will be trivial. The real work is in the software
rtkwe 12y ago
There are two big problems that I see with the current generation of VR goggles for applications like this. The first is bandwidth, the Oculus's screen had too low of a resolution for densely packed information and text, this may have been solved in the latest but I haven't seen it in person. Google Glass has the problem because of it's really small size, trying to access and manipulate much information on that interface would be extremely slow and cumbersome. The second is with full VR goggles. Right now there's not a really good way to be able to use them and interact with anything outside the VR environment. For your other point about the graph the system doesn't have to pick out the best on its own. Just having the possible values and the results displayed the user could pick which one achieves their goals best. For his light following robot though there's a pretty easy way to evaluate the parameters, how far is the robot from the light, that is a simple function and could easily be programmed. The integration of the whole room and how you get the robot to automatically do many runs with different parameters, for the viewing across possibilities, is where I think a lot of the difficulty lies. To do that with a small robot you have to have a lot of things automated: robot repositioning, light movement, data collection on the robot, etc.
- endergen 12y ago
  The new Oculus and the consumer version are high enough resolution for text rendering. Though obviously artifacts of rendering them in 3D if anything but screen aligned and pixel snapped. Additionally there are already plans for a front facing camera (that could do pass throug), because of the being blind part and the potential for hand/peripheral tracking in addition to augmented reality. Bonus: I hope they just go all out and make/purchase custom hardware tech for structure.io/project tango structured light real time 3d scanning. Would be so cool!
joshdance 12y ago
I think it would be some sort of teaching the system what good it. Let's say you are making his little robot. You want it to follow the light. So you pick a variable (light sensor reading) and optimize for a certain setting or direction (this reading as high as possible). The tool track that reading over the course of each try and picks the "best" one. This will have complexities of course. Do you want the highest single reading, or the highest mean or median over the course of the test? etc. But definitely solvable as he states. We managed to build 3d printers and space shuttles, we can build tools like this.
dcre 12y ago
I don't think he was saying that the computer itself decides what the best setting is, but rather that it tests a range of options (given by the user) and presents the results in aggregate (using reductions specified by the user) so that the user can more easily make the decision. This specification of the ranges also solves the problem of optimizing multiple parameters simultaneously — people usually begin with hunches about what the optima are, which significantly limits the number of possibilities the system will have to test.
tripzilch 12y ago
The bigger problem I wondered about, is that he glossed over how to "reset" the robot, to start out in a set initial condition for each run. He had some fade-to-black effect in his presentation, after which the robot magically appears at its starting point ... yeah. You're basically going to need a robot arm if you want to automate that bit.

anko 12y ago

I just wanted to make a couple of comments; 1. I think that when something is new, like Google Glass or occulus rift, we (as nerds) tend to think it's the future because it hasn't been possible before before we have quite worked out when it's most applicable. Giant displays are awesome when you're brainstorming a topic with people in the room, which is what maker spaces seem to be about. I can see glass being good for things like surgery, where you need to concentrate your attention on something, but would like some extra data to be easily available. VR might be a substitute for live brainstorming sessions, and it might allow you to visualise in new ways. But latency is going to be killer across remote locations, and that's a big challenge in terms of infrastructure. 2. I'd imagine as these types of techniques become commonplace, we'll have to learn more stats. You could also apply clustering and neural nets to a lot of data sets, but yeah - maybe we'll all have to become statisticians.

vanderZwan 12y ago

> 1. It seems, possibly, the exact wrong time to make rooms with giant displays. With things like Google Glass and Oculus Rift as first gen (2nd?) VR/AR you could project all of that info virtually and cheaply and be able to have all the visualization he describes wherever you are, not just at a makerspace that only a few people can use at a time. I think the reason he wants giant displays is the reason we have giant shared displays in control rooms to: so you can see what others are looking at. Don't forget maker spaces are supposed to be communal spaces where you share ideas. You can learn a lot just by watching someone else use a tool the right way. And then there's of course the collaboration, where shared displays are even more important.

yorak 12y ago

> the system tries every setting and graphs the results so it's easy to pick out the best setting. How would that happen? How does the system know what "good" is? In a research field of Parameter Tuning we try to answer that question. The field is more focused to optimizing algorithm parameters, but it could be applied to physical world of robots etc., if it is feasible to automatically repeat the experiment few dozen to around hundred times. If we would do like Bret proposed, by logging each experiment we would already have done some "probing" on the parameter space. This, in turn, would allow us to build a statistical model of the phenomena and then minimize/maximize on that. The resulting parameter configuration would then be evaluated, the model updated and the process repeated until satisfactory level of performance was reached. See for example the recent works from Hutter et al. [1], where they use Random Forests with parameter tuning to make parameter "goodness" predictions (in order to reduce the actual experiments on the target algoritm/robot/whatever). [1] http://www.cs.ubc.ca/labs/beta/Projects/SMAC/ http://www.cs.ubc.ca/labs/beta/Projects/SMAC/

knowaveragejoe 12y ago

I think display are, for now, the better option. AR and VR are still far too in their infancy for them to be as intuitively useful as a room full of displays is to NASA or a power grid operations team(per his example). That will change with time of course, but for something we can do now and over the next few years I think displays and projectors are the way to go. In a way, the projectors over the worktable he shows are(in effect) a limited form of augmented reality.

greggman 12y ago
I guess the question is how many years? How many years until we have the libraries, servers, sensors, visualizers, and other systems in place for make this a reality? Assuming it's 5 years (probably over estimating) where will fb oculus be? If it happens to have become fb oculus glass, something you wear that can augment the world this system of Bret's will come to fruition just as there displays are no longer needed I guess that's irrelevant though as switching displays will be trivial. The real work is in the software

rtkwe 12y ago

There are two big problems that I see with the current generation of VR goggles for applications like this. The first is bandwidth, the Oculus's screen had too low of a resolution for densely packed information and text, this may have been solved in the latest but I haven't seen it in person. Google Glass has the problem because of it's really small size, trying to access and manipulate much information on that interface would be extremely slow and cumbersome. The second is with full VR goggles. Right now there's not a really good way to be able to use them and interact with anything outside the VR environment. For your other point about the graph the system doesn't have to pick out the best on its own. Just having the possible values and the results displayed the user could pick which one achieves their goals best. For his light following robot though there's a pretty easy way to evaluate the parameters, how far is the robot from the light, that is a simple function and could easily be programmed. The integration of the whole room and how you get the robot to automatically do many runs with different parameters, for the viewing across possibilities, is where I think a lot of the difficulty lies. To do that with a small robot you have to have a lot of things automated: robot repositioning, light movement, data collection on the robot, etc.

endergen 12y ago
The new Oculus and the consumer version are high enough resolution for text rendering. Though obviously artifacts of rendering them in 3D if anything but screen aligned and pixel snapped. Additionally there are already plans for a front facing camera (that could do pass throug), because of the being blind part and the potential for hand/peripheral tracking in addition to augmented reality. Bonus: I hope they just go all out and make/purchase custom hardware tech for structure.io/project tango structured light real time 3d scanning. Would be so cool!

joshdance 12y ago

I think it would be some sort of teaching the system what good it. Let's say you are making his little robot. You want it to follow the light. So you pick a variable (light sensor reading) and optimize for a certain setting or direction (this reading as high as possible). The tool track that reading over the course of each try and picks the "best" one. This will have complexities of course. Do you want the highest single reading, or the highest mean or median over the course of the test? etc. But definitely solvable as he states. We managed to build 3d printers and space shuttles, we can build tools like this.

dcre 12y ago

I don't think he was saying that the computer itself decides what the best setting is, but rather that it tests a range of options (given by the user) and presents the results in aggregate (using reductions specified by the user) so that the user can more easily make the decision. This specification of the ranges also solves the problem of optimizing multiple parameters simultaneously — people usually begin with hunches about what the optima are, which significantly limits the number of possibilities the system will have to test.

tripzilch 12y ago

The bigger problem I wondered about, is that he glossed over how to "reset" the robot, to start out in a set initial condition for each run. He had some fade-to-black effect in his presentation, after which the robot magically appears at its starting point ... yeah. You're basically going to need a robot arm if you want to automate that bit.