8 ms·

Bret Victor: Seeing Spaces [video]

302 points by zindlerb 12y ago

mrspeaker 12y ago
Every time I realize I'm guessing about (rather than directly seeing) the behavior of my code I think of Bret's talks. I never actually improve my workflow, but at least now I'm angry about it!
- ColinDabritz 12y ago
  Interesting. I frequently think back to his talks and use it as a framework to guide my approach especially when debugging. I think about what I can see, and what I can't, and try to address what I can't see. This process has broken through many bugs that seem challenging until they are visible.
  - blahber 12y ago
    /optimistic I think debuggers are under appreciated by Open Source languages(as in, it's not THE priority). An approach I am forced to use is heavy log statements and a DEBUG flag, so that I can feel the program show me what it's upto. I think Bret's ideas are worth pondering on. The only issue when dealing with the philosophical idea of time, I find for myself, is that it's an easy rabbit hole to fall into. Nice interesting paper tangentially related to this, http://www.vpri.org/pdf/tr2011001_final_worlds.pdf http://www.vpri.org/pdf/tr2011001_final_worlds.pdf
vidar 12y ago
He just keeps knocking stuff out of the park.
- faceyspacey 12y ago
  no, he's not. this is his weakest presentation yet. he was on, but now he fell off. it's not making any sense that he's gone in this direction, and let last year's software ideas wither away on the vine. He should actually propel forward last year's ideas by at least releasing the stuff he said he'd open source on github. Instead, he hasn't done that, nor has he made his own commercial company (which would be a perfectly understandable route), but instead is giving lower quality talks that regurgitate material about observability that was way more profound when he shared it in relation to software, which by naturel, unlike hardware products, is unobservable.
  - DaniFong 12y ago
    You should hesitate for a second; your arguments favor exploitation rather than exploration. Bret's work aims to move beyond 'the tiny rectangle' of computer screens, and companies are nearly decade scale commitments. Why not continue to produce great research before deciding upon the idea that's 'the one?' I realize this is antithetical to the prevailing theory of lean startups, but, from a deep research perspective, that is exactly why committing to support an open source project or a company is so dangerous. Soon you would have people depending upon you, and the freedom to explore vanishes.
  - anko 12y ago
    I think with things like swift playground and lighttable which have been released due to a direct influence on Brett's work, maybe he's in exactly the right place? By discussion the broader concepts and getting people excited, he's probably getting more done than he could with a single company.
  - cryptnotic 12y ago
    Bret is your typical genius visionary. He's bored by the thing that excited him last year.
  - faceyspacey 12y ago
    nobody said don't do these talks. I'm saying just release the damn code along with it. like, i said, it makes zero sense if he's trying to help us all. The devil is in the details with coding and creation--the details of his partial implementations could help us all.
jayvanguard 12y ago
Great presentation as usual. One fundamental tension I see in much of the work he does is between purpose-built and general-purpose tooling and environments. The challenge in both the maker space as well as much of the visual learning and programming material he has done previously is that each of them is incredibly time consuming to adapt to each new different project. In the real world even similar tasks within projects in the same domain often have enough subtle differences that re-use is not possible or very costly. That isn't to say these are insurmountable but maybe much of the focus needs to be on meta-tooling that can accelerate the work of experts to build these purpose-built environments (as opposed to making generic tooling). Inspiring stuff.
- 3JPLW 12y ago
  > purpose-built and general-purpose tooling and environments That seemed to me to be a big unresolved tension in his comparison between command centers and physical workshops. Command centers still just have lots of general use multi-purpose workstations. TV stations do much better on this point, interestingly. While they all allow for seeing, I wonder what kinds of improvements we could have with purpose-built HID.
- DalekBaldwin 12y ago
  Especially for programming, the tools have to be narrowly tailored to the examples since you're constantly wrestling with the specter of Turing-completeness. Any given program is an instance of an infinite number of more general classes of programs, and it's the tool designer's job to choose which dimensions of the design space are meaningful and important enough to be worth simultaneously visualizing the consequences of possible alternatives. I think it's going to take some very judicious integration of recent work on modularity-enhancing programming paradigms into the design of reflective language implementations before it becomes tractable to build responsive special-purpose reflective tools on top of a generic infrastructure. Or at least that's the strategy I'm trying.
  - ilovecomputers 12y ago
    That strategy lead to quite the run-on sentence. You have much work ahead of you friend from that appearance alone.
cing 12y ago
In other words... a meatspace debugger? Cool idea, but I don't quite buy the comparison to "spaces laden with sensors and visualizations" like the NASA control center, Large Hadron Collider, etc.. All of those spaces revolve around monitoring, not the design/making process. In a similar vein, in my field of computational science, heaps of money has been invested in spaces for data exploration/visualization [1], unfortunately, they are essentially useless for the scientific process. [1]: http://en.wikipedia.org/wiki/Cave_automatic_virtual_environment http://en.wikipedia.org/wiki/Cave_automatic_virtual_environm...
- hcarvalhoalves 12y ago
  > All of those spaces revolve around monitoring It's not only monitoring because it's not passive, they interact with the system and make changes on the fly, so it's making in a sense too.
- wolfv 12y ago
  I agree. The big-screen control-center examples he gave are for monitoring a mission in real time. The Hadron Collider scientists analyze the data later on ordinary-size monitors. Its the systematic data collection that is important.
deleted 12y ago
[deleted]
hartror 12y ago
Some of the software shown in the first minute: http://vimeo.com/66085662 http://vimeo.com/66085662 Some pretty tools in there.
agumonkey 12y ago
Augmented Breality
tsunamifury 12y ago
I think software engineer and the fundamentals of coding have always had a bias towards those who can conceptualize ideas in the abstract, then build with the assumptions that those concepts are happening regardless of their ability to see them. This is fine, except that it limits those who need to tinker in order to find out how those concepts work. When the elements are visually recognizable and physically manipulatable, you can tinker without having to hold the entire chain on concepts in your mind. It reduces the load and increases the likelyhood of 'playing around'. I hope some day more of Victor's ideas can be realized through the understanding that visualizing processes allows us to use more of our brain to design and develop or products -- not to mention stumble upon and explore unexpected outcomes.
- ilovecomputers 12y ago
  That bias came from the necessity that engineering work is mostly invisible and you can't completely generalize every concept to fit every project. You need to put in extra work to visualize a graphic that would respond to the state of the system and the concepts you are visualizing might only be relevant to the project at hand. Despite these challenges, when we can visualize certain general concepts, which apply to a wide group of systems, but engineers do a good enough job and rarely are visual designers or UX experts involved. At that point its a cultural issue and that's why I'm grateful Bret Victor is around to advocate better designed tools for engineers.
  - seanmcdirmid 12y ago
    There are UX designers that focus on high investment tools....at AutoDesk for example. Also, high investment tools are more difficult to change given...well...the investments made in them by the user base. Also, be careful to distinguish between visual and UX, those aren't often the same people outside of the web world. Heck, data visualization people often aren't visual designers.
- darkmighty 12y ago
  I can't tell if everyone one is like that (I assume not), but I am extremely reliant on visualization, and I suppose Bret is also. I believe profoundly on it's power partly because it's the only way I can really get things done properly, which is why those tools resonate so much to me. I have a friend however that shudders every time I mention making programming more visual. I mean, If I'm not visualizing something, I can perhaps find some solution in a logical way following some guidelines -- much like following a recipe to solve a equation or doing trial an error to solve an algebraic problem. But critically, I can't create this way. I may stumble unto something useful, but it's an entirely different process from creativity -- it's efficiency is so much lower that it's qualitatively different. But my difficulty at that is not fundamental, I think. I believe you can be creative purely 'algebraically', but I have no experience with it -- but I do believe that some people have the same efficiency that I have when I think visually and perhaps stumble a little more when trying to visualize things themselves. I do think this diversity is quite good, but that's missing the point. My point is that Bret's tools shouldn't be universally essential, but are probably universally beneficial. And for some people, like me, they'd be simply enabling. To give an exaple: when I see a system of linear equations, I don't think of a bunch of steps to solve them and the number of cases that the solutions may look like in terms of constraints of variables an so on. I think of the Image hyperplane, which can be abstracted as a plane in 3D, and the Kernel, which may be for example a line not on the Image. Then instead I wonder if the Kernel is non-null, what funny things this matrix is doing to the vectors (are they rotating, contracting, and so on), or what are the invariant subspaces. I can answer most questions one could ask about such system, but it's a distinct way, I'm not sure if more efficient or not (and that may be up to the task).
primigenus 12y ago
It blows my mind that Bret keeps giving talks in public and sharing his ideas for free when pretty much each of them could have been used as a startup pitch in return for likely investment. But I guess he's more interested in inspiring others than just committing to one idea for years. I'm glad we have him around.
- deleted 12y ago
  [deleted]
- 3rd3 12y ago
  This idealism appears to be one reason why he left Apple: http://worrydream.com/Apple/ http://worrydream.com/Apple/
  - loup-vaillant 12y ago
    That's… horrible. It feels like he's under a gag order. I wager this stuff could be useful to all of us, but Apple just clings on this "Intellectual Property" like, like… Well, like any corporation.
    - GuiA 12y ago
      Huh...? He worked for a company out of his own will, and that company paid him in exchange for his work, with the full understanding that all ownership of the work would go to them. I mean, it's not even as if Apple hides that about them- that's one thing they tell you OVER and OVER when you interview with them. He knew exactly what he was getting into when he accepted to work for them (and did so for many years), and he definitely doesn't seem to mind what he got out of it (salary + the ability to call himself an "ex-Apple employee" and extract the social proof/appeal to authority that comes with it). I love Bret's work, but this page on his website is very distasteful and comes across as fairly petty.
      - shasta 12y ago
        Nah, the companies pushing these contracts are the ones exhibiting distasteful behavior. That page is very tasteful.
nickbauman 12y ago
Bret Victor is the Leonardo DaVinci of the age. A curator, assembler and presenter of the great ideas of our time.
seanmcdirmid 12y ago
Some of this is extremely similar to Jun Kato's research. http://junkato.jp http://junkato.jp More specifically see phybots: http://junkato.jp/phybots/ http://junkato.jp/phybots/ Kato leverages the overhead camera trick in this system, though in a bit different way. See "A Toolkit for Easy Development of Mobile Robot Applications with Visual Markers and a Ceiling Camera:" http://junkato.jp/publications/uist2009-sakamoto-andy.pdf http://junkato.jp/publications/uist2009-sakamoto-andy.pdf
- gajomi 12y ago
  Thanks for the excellent link. I only took a quick look, so maybe the information is somewhere on the site, but do you know if this person (or others) are doing similar work for robots in 3d?
vanderZwan 12y ago
BTW, for good reading material on control rooms, look on Google Scholar for papers by Paul Heath and Christian Luff. They're very thorough in their analysis of how people in control rooms communicate and "spontaneously" synchronise their actions.
cma 12y ago
I think spaces like this would do well to incorporate projected augmented reality ala CastAR: https://www.youtube.com/watch?v=GpmKq_qg3Tk https://www.youtube.com/watch?v=GpmKq_qg3Tk You could collaborate, sharing the same view, or each individual could project different views, or mix and match.
greggman 12y ago
I love Bret Victor's talks, blog posts, etc. They're super inspiring. 2 things came to mind though. 1. It seems, possibly, the exact wrong time to make rooms with giant displays. With things like Google Glass and Oculus Rift as first gen (2nd?) VR/AR you could project all of that info virtually and cheaply and be able to have all the visualization he describes wherever you are, not just at a makerspace that only a few people can use at a time. 2. I'm always super inspired by the Bret's visualizations but when I actually try to figure out how they'd be implemented I'm clearly not smart enough to figure out how that would happen. In this example in particular, he shows graph toward the end where the system tries every setting and graphs the results so it's easy to pick out the best setting. How would that happen? How does the system know what "good" is? It seems to me it can't know that. You'd have to program it which in itself would be pretty hard. Worse, most system have not just one adjustment but many. Just a few and there'd be tens of thousands of variations/combinations to try to figure out "best". I'm not saying we can't get there. Maybe the first step is building a framework that would make it easy to make systems like that with various kinds of visualizers, analysers, time-recording, searching features etc, and maybe somewhere along the way we'd figure out how to automate more of it. I'd love to help work on such a system.
- anko 12y ago
  I just wanted to make a couple of comments; 1. I think that when something is new, like Google Glass or occulus rift, we (as nerds) tend to think it's the future because it hasn't been possible before before we have quite worked out when it's most applicable. Giant displays are awesome when you're brainstorming a topic with people in the room, which is what maker spaces seem to be about. I can see glass being good for things like surgery, where you need to concentrate your attention on something, but would like some extra data to be easily available. VR might be a substitute for live brainstorming sessions, and it might allow you to visualise in new ways. But latency is going to be killer across remote locations, and that's a big challenge in terms of infrastructure. 2. I'd imagine as these types of techniques become commonplace, we'll have to learn more stats. You could also apply clustering and neural nets to a lot of data sets, but yeah - maybe we'll all have to become statisticians.
- vanderZwan 12y ago
greggman 12y ago
I love Bret Victor's talks, blog posts, etc. They're super inspiring. 2 things came to mind though. 1. It seems, possibly, the exact wrong time to make rooms with giant displays. With things like Google Glass and Oculus Rift as first gen (2nd?) VR/AR you could project all of that info virtually and cheaply and be able to have all the visualization he describes wherever you are, not just at a makerspace that only a few people can use at a time. 2. I'm always super inspired by the Bret's visualizations but when I actually try to figure out how they'd be implemented I'm clearly not smart enough to figure out how that would happen. In this example in particular, he shows graph toward the end where the system tries every setting and graphs the results so it's easy to pick out the best setting. How would that happen? How does the system know what "good" is? It seems to me it can't know that. You'd have to program it which in itself would be pretty hard. Worse, most system have not just one adjustment but many. Just a few and there'd be tens of thousands of variations/combinations to try to figure out "best". I'm not saying we can't get there. Maybe the first step is building a framework that would make it easy to make systems like that with various kinds of visualizers, analysers, time-recording, searching features etc, and maybe somewhere along the way we'd figure out how to automate more of it. I'd love to help work on such a system.
ohwp 12y ago
I think this is how the NSA became the NSA as we know it today. When your task is to prevent terrorism you need to see. You need to see in time and detect patterns. So you need to store as much data as possible because. So it's good to stick to some boundaries. In the example of the robot: you could measure room temperature, because maybe the sensors are acting to it. Or you could measure the amount of people in the room because sensors could act to it. Heck, maybe the sensors are acting different to different people, so track there faces and store it. Well maybe sensors are sensitive to somebodies smell so track that too. There are limits to what is useful to track.
Sarien 12y ago
I am totally in favor of good tools with good visual representations but those almost always have to be handcrafted for every specific problem. Which is probably why Bret has never delivered anything useful. And if you're going to talk about ideas and inspiration: Lighttable does nothing that emacs didn't do 20 years ago except a little prettier.
pseud0r 12y ago
These kind of things would be really great for science labs also. Let say you're doing some medical research, growing some cell cultures and you add some compounds to the cell cultures to see what happens. Then something weird happened to some of the cell cultures, and you don't know exactly what caused it. Perhaps that thing was really an important scientific discovery waiting to happen, but you missed it, because you didn't have all the data. The process is normally recorded with a lab diary, where you write down everything deemed important. The problem is, you're not going to notice everything, and there is also a lot of things that you can't see without more sensors that just your eyes. The system Bret describes here is basically an automated lab diary. With enough sensors it could record much more data, much more accurately than a person, and it has a way to query the actual data rather than having to either manually browse through pages of text or searching through it with just a basic full-text search engine. A problem with many scientific experiments is that you might a lot of measuring equipment and sensors for the thing you are doing an experiment on, but you don't have the same thing for the experiment itself, to easily be able to debug the process and to see where something went right or wrong. Why was one lab able to reproduce an experiment, but another couldn't? This kind of questions can be very difficult and time consuming to answer.
Htsthbjig 12y ago
I agree with Bret Victor here. I already have something similar of what he is proposing. Not so great, but my prototype is real and works. You can make one of this using "inexpensive" TVs for most of the room. Cheap cameras with HDMI and framegrabbers, a PC with CUDA-OpenCL cards. Arduino sensors work anywhere with all OSes and super easy to use, albeit not very efficient. My experience with years of embedded programming is that NO HUMAN BEING is made for working with the cold, brainless machine or metal if you don't visualize your data. Even the person who tells you she likes doing it, she can't work on it for long periods of time without burning. It is like climbing over 7.000meters of altitude. Humans could survive for some time with those conditions, but depleting internal resources fast.
nchlswu 12y ago
After a while of reading the replies, Iron Man came to mind.
kentpalmer 12y ago
A theory that I have been developing that might be a basis for understanding the possibility of Seeing Spaces is called Schemas Theory. See http://SchemaTheory.net http://SchemaTheory.net for a draft presentation that is still in work. Audios are still in production for the tutorial. Other papers on Schemas Theory are at https://independent.academia.edu/KentPalmer https://independent.academia.edu/KentPalmer and http://emergentdesign.net http://emergentdesign.net and http://archonic.net http://archonic.net A good book on Schemas is Umberto Eco Kant and the Platypus. Basically schemas theory tells us what it is possible to see and also give us the intelligible templates for our designs. http://kdp.me http://kdp.me
kgc 12y ago
I think Oculus technology would allow people to do all of this virtually with the physical portion being merely props. This would be a lot cheaper than doing everything for real.

302 points by zindlerb 12y ago

Every time I realize I'm guessing about (rather than directly seeing) the behavior of my code I think of Bret's talks. I never actually improve my workflow, but at least now I'm angry about it!

ColinDabritz 12y ago
Interesting. I frequently think back to his talks and use it as a framework to guide my approach especially when debugging. I think about what I can see, and what I can't, and try to address what I can't see. This process has broken through many bugs that seem challenging until they are visible.
- blahber 12y ago
  /optimistic I think debuggers are under appreciated by Open Source languages(as in, it's not THE priority). An approach I am forced to use is heavy log statements and a DEBUG flag, so that I can feel the program show me what it's upto. I think Bret's ideas are worth pondering on. The only issue when dealing with the philosophical idea of time, I find for myself, is that it's an easy rabbit hole to fall into. Nice interesting paper tangentially related to this, http://www.vpri.org/pdf/tr2011001_final_worlds.pdf http://www.vpri.org/pdf/tr2011001_final_worlds.pdf

vidar 12y ago

He just keeps knocking stuff out of the park.

faceyspacey 12y ago
no, he's not. this is his weakest presentation yet. he was on, but now he fell off. it's not making any sense that he's gone in this direction, and let last year's software ideas wither away on the vine. He should actually propel forward last year's ideas by at least releasing the stuff he said he'd open source on github. Instead, he hasn't done that, nor has he made his own commercial company (which would be a perfectly understandable route), but instead is giving lower quality talks that regurgitate material about observability that was way more profound when he shared it in relation to software, which by naturel, unlike hardware products, is unobservable.
- DaniFong 12y ago
  You should hesitate for a second; your arguments favor exploitation rather than exploration. Bret's work aims to move beyond 'the tiny rectangle' of computer screens, and companies are nearly decade scale commitments. Why not continue to produce great research before deciding upon the idea that's 'the one?' I realize this is antithetical to the prevailing theory of lean startups, but, from a deep research perspective, that is exactly why committing to support an open source project or a company is so dangerous. Soon you would have people depending upon you, and the freedom to explore vanishes.
- anko 12y ago
  I think with things like swift playground and lighttable which have been released due to a direct influence on Brett's work, maybe he's in exactly the right place? By discussion the broader concepts and getting people excited, he's probably getting more done than he could with a single company.
- cryptnotic 12y ago
  Bret is your typical genius visionary. He's bored by the thing that excited him last year.
- faceyspacey 12y ago
  nobody said don't do these talks. I'm saying just release the damn code along with it. like, i said, it makes zero sense if he's trying to help us all. The devil is in the details with coding and creation--the details of his partial implementations could help us all.

jayvanguard 12y ago

Great presentation as usual. One fundamental tension I see in much of the work he does is between purpose-built and general-purpose tooling and environments. The challenge in both the maker space as well as much of the visual learning and programming material he has done previously is that each of them is incredibly time consuming to adapt to each new different project. In the real world even similar tasks within projects in the same domain often have enough subtle differences that re-use is not possible or very costly. That isn't to say these are insurmountable but maybe much of the focus needs to be on meta-tooling that can accelerate the work of experts to build these purpose-built environments (as opposed to making generic tooling). Inspiring stuff.

3JPLW 12y ago
> purpose-built and general-purpose tooling and environments That seemed to me to be a big unresolved tension in his comparison between command centers and physical workshops. Command centers still just have lots of general use multi-purpose workstations. TV stations do much better on this point, interestingly. While they all allow for seeing, I wonder what kinds of improvements we could have with purpose-built HID.
DalekBaldwin 12y ago
Especially for programming, the tools have to be narrowly tailored to the examples since you're constantly wrestling with the specter of Turing-completeness. Any given program is an instance of an infinite number of more general classes of programs, and it's the tool designer's job to choose which dimensions of the design space are meaningful and important enough to be worth simultaneously visualizing the consequences of possible alternatives. I think it's going to take some very judicious integration of recent work on modularity-enhancing programming paradigms into the design of reflective language implementations before it becomes tractable to build responsive special-purpose reflective tools on top of a generic infrastructure. Or at least that's the strategy I'm trying.
- ilovecomputers 12y ago
  That strategy lead to quite the run-on sentence. You have much work ahead of you friend from that appearance alone.

cing 12y ago

In other words... a meatspace debugger? Cool idea, but I don't quite buy the comparison to "spaces laden with sensors and visualizations" like the NASA control center, Large Hadron Collider, etc.. All of those spaces revolve around monitoring, not the design/making process. In a similar vein, in my field of computational science, heaps of money has been invested in spaces for data exploration/visualization [1], unfortunately, they are essentially useless for the scientific process. [1]: http://en.wikipedia.org/wiki/Cave_automatic_virtual_environment http://en.wikipedia.org/wiki/Cave_automatic_virtual_environm...

hcarvalhoalves 12y ago
> All of those spaces revolve around monitoring It's not only monitoring because it's not passive, they interact with the system and make changes on the fly, so it's making in a sense too.
wolfv 12y ago
I agree. The big-screen control-center examples he gave are for monitoring a mission in real time. The Hadron Collider scientists analyze the data later on ordinary-size monitors. Its the systematic data collection that is important.

deleted 12y ago

[deleted]

hartror 12y ago

Some of the software shown in the first minute: http://vimeo.com/66085662 http://vimeo.com/66085662 Some pretty tools in there.

agumonkey 12y ago

Augmented Breality

tsunamifury 12y ago

I think software engineer and the fundamentals of coding have always had a bias towards those who can conceptualize ideas in the abstract, then build with the assumptions that those concepts are happening regardless of their ability to see them. This is fine, except that it limits those who need to tinker in order to find out how those concepts work. When the elements are visually recognizable and physically manipulatable, you can tinker without having to hold the entire chain on concepts in your mind. It reduces the load and increases the likelyhood of 'playing around'. I hope some day more of Victor's ideas can be realized through the understanding that visualizing processes allows us to use more of our brain to design and develop or products -- not to mention stumble upon and explore unexpected outcomes.

ilovecomputers 12y ago
That bias came from the necessity that engineering work is mostly invisible and you can't completely generalize every concept to fit every project. You need to put in extra work to visualize a graphic that would respond to the state of the system and the concepts you are visualizing might only be relevant to the project at hand. Despite these challenges, when we can visualize certain general concepts, which apply to a wide group of systems, but engineers do a good enough job and rarely are visual designers or UX experts involved. At that point its a cultural issue and that's why I'm grateful Bret Victor is around to advocate better designed tools for engineers.
- seanmcdirmid 12y ago
  There are UX designers that focus on high investment tools....at AutoDesk for example. Also, high investment tools are more difficult to change given...well...the investments made in them by the user base. Also, be careful to distinguish between visual and UX, those aren't often the same people outside of the web world. Heck, data visualization people often aren't visual designers.
darkmighty 12y ago
I can't tell if everyone one is like that (I assume not), but I am extremely reliant on visualization, and I suppose Bret is also. I believe profoundly on it's power partly because it's the only way I can really get things done properly, which is why those tools resonate so much to me. I have a friend however that shudders every time I mention making programming more visual. I mean, If I'm not visualizing something, I can perhaps find some solution in a logical way following some guidelines -- much like following a recipe to solve a equation or doing trial an error to solve an algebraic problem. But critically, I can't create this way. I may stumble unto something useful, but it's an entirely different process from creativity -- it's efficiency is so much lower that it's qualitatively different. But my difficulty at that is not fundamental, I think. I believe you can be creative purely 'algebraically', but I have no experience with it -- but I do believe that some people have the same efficiency that I have when I think visually and perhaps stumble a little more when trying to visualize things themselves. I do think this diversity is quite good, but that's missing the point. My point is that Bret's tools shouldn't be universally essential, but are probably universally beneficial. And for some people, like me, they'd be simply enabling. To give an exaple: when I see a system of linear equations, I don't think of a bunch of steps to solve them and the number of cases that the solutions may look like in terms of constraints of variables an so on. I think of the Image hyperplane, which can be abstracted as a plane in 3D, and the Kernel, which may be for example a line not on the Image. Then instead I wonder if the Kernel is non-null, what funny things this matrix is doing to the vectors (are they rotating, contracting, and so on), or what are the invariant subspaces. I can answer most questions one could ask about such system, but it's a distinct way, I'm not sure if more efficient or not (and that may be up to the task).

primigenus 12y ago

It blows my mind that Bret keeps giving talks in public and sharing his ideas for free when pretty much each of them could have been used as a startup pitch in return for likely investment. But I guess he's more interested in inspiring others than just committing to one idea for years. I'm glad we have him around.

deleted 12y ago
[deleted]
3rd3 12y ago
This idealism appears to be one reason why he left Apple: http://worrydream.com/Apple/ http://worrydream.com/Apple/
- loup-vaillant 12y ago
  That's… horrible. It feels like he's under a gag order. I wager this stuff could be useful to all of us, but Apple just clings on this "Intellectual Property" like, like… Well, like any corporation.
  - GuiA 12y ago
    Huh...? He worked for a company out of his own will, and that company paid him in exchange for his work, with the full understanding that all ownership of the work would go to them. I mean, it's not even as if Apple hides that about them- that's one thing they tell you OVER and OVER when you interview with them. He knew exactly what he was getting into when he accepted to work for them (and did so for many years), and he definitely doesn't seem to mind what he got out of it (salary + the ability to call himself an "ex-Apple employee" and extract the social proof/appeal to authority that comes with it). I love Bret's work, but this page on his website is very distasteful and comes across as fairly petty.
    - shasta 12y ago
      Nah, the companies pushing these contracts are the ones exhibiting distasteful behavior. That page is very tasteful.

nickbauman 12y ago

Bret Victor is the Leonardo DaVinci of the age. A curator, assembler and presenter of the great ideas of our time.

seanmcdirmid 12y ago

Some of this is extremely similar to Jun Kato's research. http://junkato.jp http://junkato.jp More specifically see phybots: http://junkato.jp/phybots/ http://junkato.jp/phybots/ Kato leverages the overhead camera trick in this system, though in a bit different way. See "A Toolkit for Easy Development of Mobile Robot Applications with Visual Markers and a Ceiling Camera:" http://junkato.jp/publications/uist2009-sakamoto-andy.pdf http://junkato.jp/publications/uist2009-sakamoto-andy.pdf

gajomi 12y ago
Thanks for the excellent link. I only took a quick look, so maybe the information is somewhere on the site, but do you know if this person (or others) are doing similar work for robots in 3d?

vanderZwan 12y ago

BTW, for good reading material on control rooms, look on Google Scholar for papers by Paul Heath and Christian Luff. They're very thorough in their analysis of how people in control rooms communicate and "spontaneously" synchronise their actions.

cma 12y ago

I think spaces like this would do well to incorporate projected augmented reality ala CastAR: https://www.youtube.com/watch?v=GpmKq_qg3Tk https://www.youtube.com/watch?v=GpmKq_qg3Tk You could collaborate, sharing the same view, or each individual could project different views, or mix and match.

greggman 12y ago

I love Bret Victor's talks, blog posts, etc. They're super inspiring. 2 things came to mind though. 1. It seems, possibly, the exact wrong time to make rooms with giant displays. With things like Google Glass and Oculus Rift as first gen (2nd?) VR/AR you could project all of that info virtually and cheaply and be able to have all the visualization he describes wherever you are, not just at a makerspace that only a few people can use at a time. 2. I'm always super inspired by the Bret's visualizations but when I actually try to figure out how they'd be implemented I'm clearly not smart enough to figure out how that would happen. In this example in particular, he shows graph toward the end where the system tries every setting and graphs the results so it's easy to pick out the best setting. How would that happen? How does the system know what "good" is? It seems to me it can't know that. You'd have to program it which in itself would be pretty hard. Worse, most system have not just one adjustment but many. Just a few and there'd be tens of thousands of variations/combinations to try to figure out "best". I'm not saying we can't get there. Maybe the first step is building a framework that would make it easy to make systems like that with various kinds of visualizers, analysers, time-recording, searching features etc, and maybe somewhere along the way we'd figure out how to automate more of it. I'd love to help work on such a system.

anko 12y ago
I just wanted to make a couple of comments; 1. I think that when something is new, like Google Glass or occulus rift, we (as nerds) tend to think it's the future because it hasn't been possible before before we have quite worked out when it's most applicable. Giant displays are awesome when you're brainstorming a topic with people in the room, which is what maker spaces seem to be about. I can see glass being good for things like surgery, where you need to concentrate your attention on something, but would like some extra data to be easily available. VR might be a substitute for live brainstorming sessions, and it might allow you to visualise in new ways. But latency is going to be killer across remote locations, and that's a big challenge in terms of infrastructure. 2. I'd imagine as these types of techniques become commonplace, we'll have to learn more stats. You could also apply clustering and neural nets to a lot of data sets, but yeah - maybe we'll all have to become statisticians.
vanderZwan 12y ago

greggman 12y ago

ohwp 12y ago

I think this is how the NSA became the NSA as we know it today. When your task is to prevent terrorism you need to see. You need to see in time and detect patterns. So you need to store as much data as possible because. So it's good to stick to some boundaries. In the example of the robot: you could measure room temperature, because maybe the sensors are acting to it. Or you could measure the amount of people in the room because sensors could act to it. Heck, maybe the sensors are acting different to different people, so track there faces and store it. Well maybe sensors are sensitive to somebodies smell so track that too. There are limits to what is useful to track.

Sarien 12y ago

I am totally in favor of good tools with good visual representations but those almost always have to be handcrafted for every specific problem. Which is probably why Bret has never delivered anything useful. And if you're going to talk about ideas and inspiration: Lighttable does nothing that emacs didn't do 20 years ago except a little prettier.

pseud0r 12y ago

These kind of things would be really great for science labs also. Let say you're doing some medical research, growing some cell cultures and you add some compounds to the cell cultures to see what happens. Then something weird happened to some of the cell cultures, and you don't know exactly what caused it. Perhaps that thing was really an important scientific discovery waiting to happen, but you missed it, because you didn't have all the data. The process is normally recorded with a lab diary, where you write down everything deemed important. The problem is, you're not going to notice everything, and there is also a lot of things that you can't see without more sensors that just your eyes. The system Bret describes here is basically an automated lab diary. With enough sensors it could record much more data, much more accurately than a person, and it has a way to query the actual data rather than having to either manually browse through pages of text or searching through it with just a basic full-text search engine. A problem with many scientific experiments is that you might a lot of measuring equipment and sensors for the thing you are doing an experiment on, but you don't have the same thing for the experiment itself, to easily be able to debug the process and to see where something went right or wrong. Why was one lab able to reproduce an experiment, but another couldn't? This kind of questions can be very difficult and time consuming to answer.

Htsthbjig 12y ago

I agree with Bret Victor here. I already have something similar of what he is proposing. Not so great, but my prototype is real and works. You can make one of this using "inexpensive" TVs for most of the room. Cheap cameras with HDMI and framegrabbers, a PC with CUDA-OpenCL cards. Arduino sensors work anywhere with all OSes and super easy to use, albeit not very efficient. My experience with years of embedded programming is that NO HUMAN BEING is made for working with the cold, brainless machine or metal if you don't visualize your data. Even the person who tells you she likes doing it, she can't work on it for long periods of time without burning. It is like climbing over 7.000meters of altitude. Humans could survive for some time with those conditions, but depleting internal resources fast.

nchlswu 12y ago

After a while of reading the replies, Iron Man came to mind.

kentpalmer 12y ago

A theory that I have been developing that might be a basis for understanding the possibility of Seeing Spaces is called Schemas Theory. See http://SchemaTheory.net http://SchemaTheory.net for a draft presentation that is still in work. Audios are still in production for the tutorial. Other papers on Schemas Theory are at https://independent.academia.edu/KentPalmer https://independent.academia.edu/KentPalmer and http://emergentdesign.net http://emergentdesign.net and http://archonic.net http://archonic.net A good book on Schemas is Umberto Eco Kant and the Platypus. Basically schemas theory tells us what it is possible to see and also give us the intelligible templates for our designs. http://kdp.me http://kdp.me

kgc 12y ago

I think Oculus technology would allow people to do all of this virtually with the physical portion being merely props. This would be a lot cheaper than doing everything for real.