14 ms·

I’ve been following OS UI experiments for some years and the problem with all of them was that they took an approach that sounded cool but wasn’t really practic

by fairpx 9y ago

I’ve been following OS UI experiments for some years and the problem with all of them was that they took an approach that sounded cool but wasn’t really practical. From 3D desktops to strange card based interfaces that wanted to replicate the messy physical desk. I think the key is to redesign the GUI by removing stuff instead of adding fancy new ways of shuffling files.

setr 9y ago
I imagine the big issue is that the underlying model isn't actually being changed to support the new UI (even by a wrapper-layer); so you're just doing shallow edits to the "skin" of the OS. The problem being the underlying model and the WIMP model have been co-evolving, and likely aren't very open to other fashions of interaction. Presumably, the way to beat it out is to figure out how to change both to instantiate an entirely different interaction model, and thus naturally a new UI. Otherwise you can really only commit to incremental improvememts, but nothing particularly interesting. ie smalltalk's requires an entirely different underlying model to support its in-place editable UI model; it cannot naturally be built (but maybe hacked together) if you try to avoid that shift
- mncharity 9y ago
  The big incoming UI challenge is XR. But as with ST/Alto, it's an integrated bootstrap problem of hardware, system design, foundational software, user software, and UI. Absent research labs, we wait on market availability for hardware. For the modern analogues of mice and bitmapped screens. Eye tracking may be cheap next year, but now it's $2k. High-resolution HMDs may be $10k next year, but now they're simply unavailable. Hand tracking is flaky. Haptics are "it buzzes". For software, we're so used to the 2D windows interface, it's easy to forget how much foundational work was needed. Eg, around topography, and Smalltalk, and live coding. Hand gesture recognition, spoken dialog management, 3D constraint-based layout, are just a few modern analogues. And so on. So it looks like we're going to be doing broad-based innovation and infrastructure construction, gated on hardware availability. And plagued by patents. It's not clear to me what can usefully be done to explore UI in the meantime. You can do wireframing in TiltBrush, but not really. It doesn't give you the feel of say hand-attached controls, and of automated collaboration in general. The "oh, it feels good to present this graph as a sphere, and hold it you lap and peel off layers". So... Perhaps do UI design by paper improv? A crew scribbling on and holding up paper, hovering around a "user", who is managing a todo list? :) That could be fun.
  - scroot 9y ago
    The eleVR people have some interesting demos in this regard, where you program from inside the VR system. That might be the best way to try things out and conduct the "foundational work"
    - mncharity 9y ago
      > "foundational work" Hmm, I was ambiguous there. What I had in mind for "foundational" was, for example, constraint engines for GUI layout. Now we have CSS. But there was something like a decade, where we knew constraints were a good way to do UI layout, but there weren't available constraint engines. Each major language had odd bits of abandonware zombies, academic or hobby projects, with code which might or might not be online, or be buildable, or be licensed for use. So we had to wait for market incentives to change, and some non-trivial work to be done, before we had the foundation element of GUI constraint-based layout. An Alto analogy might be bitmapped fonts and display. A current analogy might be 3D dynamic structured graph layout. There's been a lot of research over the years on how to dynamically modify graphs without disorienting viewers. And on how to layout graphs within graphs, and with various spatial constraints. But last time I checked, the github cupboard was bare. So say you want to write a really wizzy todo app with complex dependencies. Well, first you read two decades of graph layout papers; then you write the graph layout library no one else has gotten to yet. Foundational work on our collective todo list.
    - scroot 9y ago
      Ok I found what I was talking about: http://elevr.com/portfolio/future-programming-interfaces/ http://elevr.com/portfolio/future-programming-interfaces/
  - seanmcdirmid 9y ago
    VR has a huge input problem, you can't touch any of the 3D stuff, you are basically left waving hands in the air. Bret Victor's lab is going in the other direction of tangible objects with digital projections.
    - 9y ago
seanmcdirmid 9y ago
This is what killed Microsoft Surface the Table; the NUIs could never demonstrate their worth beyond whimsy. Metro seemed to actually be a reset with all the NUI stuff thrown out.
bloaf 9y ago
I agree to a certain extent, and I think we can see that in the the evolution of video game interfaces. If you look at games like System Shock 2 to Bioshock, you see that the interfaces have evolved in the direction of finding ways of simplifying the information presented to the player and eliminating irrelevant information.

by fairpx 9y ago

setr 9y ago

I imagine the big issue is that the underlying model isn't actually being changed to support the new UI (even by a wrapper-layer); so you're just doing shallow edits to the "skin" of the OS. The problem being the underlying model and the WIMP model have been co-evolving, and likely aren't very open to other fashions of interaction. Presumably, the way to beat it out is to figure out how to change both to instantiate an entirely different interaction model, and thus naturally a new UI. Otherwise you can really only commit to incremental improvememts, but nothing particularly interesting. ie smalltalk's requires an entirely different underlying model to support its in-place editable UI model; it cannot naturally be built (but maybe hacked together) if you try to avoid that shift

mncharity 9y ago
The big incoming UI challenge is XR. But as with ST/Alto, it's an integrated bootstrap problem of hardware, system design, foundational software, user software, and UI. Absent research labs, we wait on market availability for hardware. For the modern analogues of mice and bitmapped screens. Eye tracking may be cheap next year, but now it's $2k. High-resolution HMDs may be $10k next year, but now they're simply unavailable. Hand tracking is flaky. Haptics are "it buzzes". For software, we're so used to the 2D windows interface, it's easy to forget how much foundational work was needed. Eg, around topography, and Smalltalk, and live coding. Hand gesture recognition, spoken dialog management, 3D constraint-based layout, are just a few modern analogues. And so on. So it looks like we're going to be doing broad-based innovation and infrastructure construction, gated on hardware availability. And plagued by patents. It's not clear to me what can usefully be done to explore UI in the meantime. You can do wireframing in TiltBrush, but not really. It doesn't give you the feel of say hand-attached controls, and of automated collaboration in general. The "oh, it feels good to present this graph as a sphere, and hold it you lap and peel off layers". So... Perhaps do UI design by paper improv? A crew scribbling on and holding up paper, hovering around a "user", who is managing a todo list? :) That could be fun.
- scroot 9y ago
  The eleVR people have some interesting demos in this regard, where you program from inside the VR system. That might be the best way to try things out and conduct the "foundational work"
  - mncharity 9y ago
    > "foundational work" Hmm, I was ambiguous there. What I had in mind for "foundational" was, for example, constraint engines for GUI layout. Now we have CSS. But there was something like a decade, where we knew constraints were a good way to do UI layout, but there weren't available constraint engines. Each major language had odd bits of abandonware zombies, academic or hobby projects, with code which might or might not be online, or be buildable, or be licensed for use. So we had to wait for market incentives to change, and some non-trivial work to be done, before we had the foundation element of GUI constraint-based layout. An Alto analogy might be bitmapped fonts and display. A current analogy might be 3D dynamic structured graph layout. There's been a lot of research over the years on how to dynamically modify graphs without disorienting viewers. And on how to layout graphs within graphs, and with various spatial constraints. But last time I checked, the github cupboard was bare. So say you want to write a really wizzy todo app with complex dependencies. Well, first you read two decades of graph layout papers; then you write the graph layout library no one else has gotten to yet. Foundational work on our collective todo list.
  - scroot 9y ago
    Ok I found what I was talking about: http://elevr.com/portfolio/future-programming-interfaces/ http://elevr.com/portfolio/future-programming-interfaces/
- seanmcdirmid 9y ago
  VR has a huge input problem, you can't touch any of the 3D stuff, you are basically left waving hands in the air. Bret Victor's lab is going in the other direction of tangible objects with digital projections.
  - 9y ago

seanmcdirmid 9y ago

This is what killed Microsoft Surface the Table; the NUIs could never demonstrate their worth beyond whimsy. Metro seemed to actually be a reset with all the NUI stuff thrown out.

bloaf 9y ago

I agree to a certain extent, and I think we can see that in the the evolution of video game interfaces. If you look at games like System Shock 2 to Bioshock, you see that the interfaces have evolved in the direction of finding ways of simplifying the information presented to the player and eliminating irrelevant information.