8 ms·

10 finger multi-touch UI designed to replace the mouse

171 points by zain 17y ago

TrevorJ 17y ago
Having a non-focused, non-active window arbitrarily take up 1/3 of my viewable monitor at all times would drive me nuts. It also makes no sense to me to go to the trouble of allowing a user to use all ten fingers, and then restricting application interaction to only two or three fingers. I work in rich, deep applications with tons of controls and buttons, if I'm going to go to the trouble of learning to use ten fingers to navigate a computer, you better allow me to make use of all of them withing the individual programs that I'm using.
- stcredzero 17y ago
  I think you're onto something. OS interaction/application switching should be simple, and the individual app interaction should be more complex. I think 4 app selection modes are almost too much. 3 is perfect. Prediction: I suspect that Apple is taking Exposé in exactly this direction.
  - buro9 17y ago
    Re: the prediction. Doesn't it help to have a documented piece of prior art like the Linux demo in this video when it comes to patent defence and implementation on other platforms? Of course, R. Clayton Miller may have patented too. Or he could see it in the same vein as the safety match and just record it to give it away.
  - TrevorJ 17y ago
    The real logical gap in the thinking here is that they are focusing on replacing the mouse but in reality, they are in fact replacing functions that can be executed simply and quickly with a keyboard. Application switching etc can be done efficiently with a keyboard. At the very least I question the wisdom of moving away from the modifier key paradigm because it is an elegant and easy method of interaction. I also believe that as we engage more and more of the body in a control interface, for it to be natural it is going to need to mimic real-world interaction so that we can rely on our lifetime of experience interacting with physical objects. The better solution in my opinion would contain: -2 mirrored screens so that the one under your hands can be glanced at for reference as well. -A UI approach that allows you to interact with objects and windows in the same ways you would with real objects. (Grabbing the edges moves the window, pushing down on it 'sticks' it in place, etc.)
da5e 17y ago
I wonder if they could with a tap bring up an virtual keyboard on the touchpad were the fingers already are and do away with a keyboard altogether. This sort of harkens back to the chord keyset.
aik 17y ago
Pretty cool. I could see my productivity increase from something like that.
th0ma5 17y ago
Multitouch innovators should take heed. A lot of the NUIGroup community is centered around hardware innovations, which is cool and all, but we need more and more software!
agazso 17y ago
I'm looking forward to use a multitouch touchpad with my desktop keyboard just as I currently do with my laptop. However I will definitely want to keep a mouse also when in need for more precision for tasks like photo editing etc. 10gui seems to be a very cool prototype and I guess that Apple and Microsoft are already working on things like this. I also spotted a weakness of this demo: 10gui uses the little finger to invoke contextual menus. Most of the people who cannot touch type with all ten fingers (and my guess this is about the 99% of all computer users :) will find this uncomfortable.
- NathanKP 17y ago
  I agree. A ten finger system is going to appeal greatly to those of us who already touch type and are used to using multiple fingers. But I suspect that many people aren't going to be too happy with the coordination required to use such a system. But I may be wrong. Perhaps when it comes down to it the interface will feel so intuitive that there will be no problems at all. We'll have to wait and see.
  - timgasper 17y ago
    That was one of the biggest things I worry about - the learning curve of all the gestures. A mouse and keyboard are quite straightforward in terms of how the operate. A 10GUI interface would require a significant amount of training and practice. At which point maybe yes - it will become completely natural.
saurabh 17y ago
It certainly looks cool. I am just not sure about the ergonomics. What would really be awesome is not having to lift my hand at all by switching between touch mode and keyboard mode at a flick of a finger.
mhb 17y ago
tl;dw 30 seconds in and they haven't shown it yet?
- gjm11 17y ago
  It's worse than that. The first time they say anything about their interface is at about 2:00. That's extremely vague (it's a multitouch sensor! where your mouse would be!). No more specifics until about 4:00 (windows are arranged in only one dimension). They don't actually show the damn thing until 4:45. Oh, and all accompanied by the Most Annoying Music In The World.
  - NathanKP 17y ago
    Patience. Any good idea requires a full explanation to build it up. I feel that they did the right thing by contrasting the mouse with their new system because it helped show the benefits of a ten figure multi-touch pad compared with a single point mouse.
    - DanielStraight 17y ago
      I'm not entirely convinced they showed such a benefit. I can already use multiple fingers to move windows by clicking anywhere on them... two of those fingers are just on buttons. Not to mention, I still only have two eyes. I can't be looking at 4 things on the screen at the same time anyway.
  - viraptor 17y ago
    I understand you don't read many typical research papers? (5% abstract, 15% introduction, 15% history and justification, then comes the real meat...) I seriously felt like reading another HCI journal while watching this - almost the same structure :)
    - gjm11 17y ago
      I read plenty of research papers. They don't usually have a gee-isn't-this-cool narration and annoying background music. Also, if an academic paper's background material is familiar and boring then you can easily skim it and see where the new stuff begins. That's not so easy with a video.
bryansum 17y ago
The video mentions the problem of always having one's hands in the way with a direct interface. This could also be alleviated by moving the hands underneath the display ala the capacitive mouse from Microsoft Research: http://arstechnica.com/hardware/news/2009/10/video-microsoft-research-demoes-five-multitouch-mice.ars http://arstechnica.com/hardware/news/2009/10/video-microsoft...
jpwagner 17y ago
Phenomenal video. The ideas are definitely interesting, but I am amazed at the clarity of the presentation.
- deyan 17y ago
  Thanks for posting, it looks quite interesting. The interface is something that I can imagine myself getting adjusted to (for example, I am a big Opera user and use gestures all the time - once you get used to them they are phenomenal). I had a similar idea a while ago and might try to implement it this summer as a side project using Johnny Wii's tricks perhaps + Windows 7 (not sure how much of the interface I can pull off). I would be curious to see in particular how tired I am (i.e. my hands) after using such an interface for a while.
CWuestefeld 17y ago
That's cool as far as it goes. But what about the keyboard? The mouse is just one aspect of the computer interaction. I can't remove my keyboard from my desk just to make room for this mouse replacement. The keyboard seems a big barrier (literally!) for getting this device on the desktop.
- aik 17y ago
  The end of the presentation had an image of a possible solution - basically having the touch pad exist right below the keyboard, just like it is on any laptop built today. Definitely makes the keyboard/touchpad area quite a bit larger though.
  - extension 17y ago
    If you imagine the touch pad being wide but not much taller than the one on a laptop, it starts to seem more practical.
- inffcs00 17y ago
  At the end of the video you can see the multitouch device integrated with a keyboard. :)
  - catch23 17y ago
    it seems like the touch system should be able detect where your main 4 fingers are, and then show a rough virtual keyboard in its space with your 8 fingers on the home row automatically. The only thing lacking would be tactile feedback, but that's solvable. If people are accustomed to seeing the 10 dots representing the fingers, it should be easy to provide small visual cues to indicate if you're about to type a letter that isn't what you want.
    - ismarc 17y ago
      I type 80+ words a minute regularly. It would be insanely difficult to type at that speed without the little nubs on f and j keys, as well as the tactile feedback as fingers slide over keys preparing for what they're going to be typing next.
extension 17y ago
It seems pretty obvious to me that the most efficient and intuitive way to manage windows is just a boring list at the bottom/side of the screen, but I do hold out hope, like everyone else, that it turns out to be some flashy zooming 3D multitouch thing. It is refreshing though, to see research that dispenses with the legacy of GUI convention we've built up over the years, instead of just piling on new features.
bprater 17y ago
They've certainly put lots of thought into the design of this device/software. But the whole time I kept thinking: "what about the keyboard?" (Eventually, at the end, they showed the device in a traditional trackpad position below a keyboard.) I wish they had considered a way to make the keyboard a part of the device. Perhaps an on-screen keyboard, some haptics? The challenge with that part of the input device is that to touch type, you need that spacial reference a keyboard gives you.
tc 17y ago
While we're rethinking the human interface, could we please stop staggering typing keys and put frequently used keys like carriage return and backspace under strong fingers? [1] I was disappointed to see at the end of the 10gui video a concept keyboard with staggered keys and with backspace way over in the corner. If we can't move keyboards past the constraints imposed by mechanical typewriters, what hope do we have at changing the mouse paradigm? [1] Typematrix is the only keyboard maker I know of that currently does this while still building a relatively standard keyboard (and something that could be integrated into a laptop).
- RevRal 17y ago
  Colemak layout turns your caps lock into a backspace. This works surprisingly well, but I really wish the spacebar was split in half. And I've always felt shift+spacebar should = backspace.
  - mjgoins 17y ago
    I use a keyboard with Control where Caps Lock would be on most keyboards (a sun unix keyboard). That combined with the fact that Control-H is backspace in a standard unix environment means I don't have to reach for the backspace key.
- icefox 17y ago
  If you have a Kinesis Advantage keyboard the backspace,delete space,enter are under your thumbs
- lief79 17y ago
  Not sure about the final released product, but the fingerworks prototypes were completely customizable.
- atlei 17y ago
  Hmmm.. I actually have both types of products. The Kinesis keyboard with backspace, ctrl, alt, enter, home etc using my thumb AND the iGesture pad from Fingerworks ( http://www.fingerworks.com/ http://www.fingerworks.com/ ) which is a large "mouse pad" where you can use gestures with 5 fingers (and it works for all applications). The brilliant part is that the iGesture pad fits perfectly on the top of the Kinesis keyboard (the left and right part of the keyboard is separated for ergonomic reasons). I've had the iGesture for a couple of years now - it will be a sad day when it dies as there are no replacements that I know of....maybe Apple will release a similar "mousepad" in the future ?
zain 17y ago
There's a similar device I read about with a hardware prototype that actually works. It doesn't have the software support of 10gui but it's basically a resistive touchscreen mapped to keypress macros (copy/paste, app switching, etc). Here's some pictures: http://www.flickr.com/photos/pauric/sets/72157616093748066/with/3488471659/ http://www.flickr.com/photos/pauric/sets/72157616093748066/w...
- NathanKP 17y ago
  Is that the same device? It doesn't look anything like the prototype design in the video. I also don't see any correlation between Pauric on Flickr and C. Miller of 10/GUI, at least not by exploring Pauric's Flickr profile or Twitter account. Perhaps you can explain why you think that those images are of 10/GUI's device? How did you find them?
  - zain 17y ago
    You're right. It isn't the exact same device. I ran across it while doing more research on 10gui and mistakenly thought it was by the same person. I updated my post. Sorry about that.
    - NathanKP 17y ago
      Okay no problem. I thought that something was funny as I looked through those Flickr photos. ;)
Semiapies 17y ago
Talk about multi-touch reminds me of a friend's perennial complaint, "The future of interfaces seems designed to screw me over." He has a congenital deformity of the arms, but he can type (and is one of those clacky-IBM keyboard fetishists). With his feet, he can use mice and controllers. But multi-touch systems are beyond his ability - while he can pivot and shift to type remarkably quickly, he can't get more than one hand on something at the same time. Worse, each hand is effectively only one digit for him. And then, there are the folks who don't even have his degree of dexterity. I can't help thinking that it might be a good thing for general interface designers to consider the implications of disabilities for users of these designs, instead of shrugging that off onto providers of interfaces specifically for the disabled.
- NathanKP 17y ago
  That is a really good point. The software of any future interface will have to include support for older devices and techniques. But this may perhaps be a good thing because it would motivate the specific design of devices that will be more efficient for the disabled simply because it will no longer be possible for them to use the multi-touch interface.
- gecko 17y ago
  Without disagreeing, I also don't think it makes sense to discount potentially massive improvements in user interfaces merely because a portion of the population cannot use them.
  - joe_the_user 17y ago
    No, but his problem actually points to this being a problem for anyone. It becomes a problem when the machine requires all of my dexterity just to accomplish a given task. I dislike all Apple products specifically because they require mouse operations to accomplish things where Linux and MS interfaces tend to have keyboard shortcuts.
    - deleted 17y ago
      [deleted]
zacechola 17y ago
I'd like to see it with the pad acting as the keyboard, too. though, that's more software implementation, while the video is talking hardware interface. Although, the problem with that kind of implementation is that it doesn't have the tactile feel of a keyboard. however, iPhone users seem to have acclimated after first bitching about it.
- NathanKP 17y ago
  I personally think that a real physical keyboard will always be preferred. 10Gui is doing the right thing by putting the keyboard above the multi-touch pad.
  - lief79 17y ago
    http://www.fingerworks.com/faq.html http://www.fingerworks.com/faq.html Apparently people could get up to 70 wpm with what could be considered the prior version of this. For what it is worth, at least when I was involved with it, most people found a keyboard a lot easier to use for rapid typing.
TimothyFitz 17y ago
This basically already exist(ed): http://www.fingerworks.com/ST_product.html http://www.fingerworks.com/ST_product.html Apple bought them up and stopped making their products (but we got multitouch on the iPhone, an interesting trade-off).
- lief79 17y ago
  I had interned with Fingerworks prior to the formal release of the project. The main problem I heard about was the demand cost/issues. Outside of the proposed UI changes, Fingerworks product did all of this. I still wish I had bought one when I had the chance. Anyway, if they are serious about getting these made, I believe University of Delaware owns a fair amount of Wayne Westerman's patents, as the basic idea was the premise for his doctorate. He was a professor starting in '99, so that should make them easier to locate.
thomasfl 17y ago
A ten finger multitouch pad for desktops makes perfectly sense. The multitouch pad on mac book pro is way more pleasant to use than a mouse, but can't be used with an external monitor.
- jodrellblank 17y ago
  It isn't more pleasant to use than a mouse. If I have to scrape two surfaces together constantly to use a computer, I'd rather one of them wasn't the most sensitive part of my finger.
RevRal 17y ago
As for a keyboard, why not have the virtual buttons relative to where the the fingers are placed? As in, place all eight fingers onto the pad and this automatically becomes home row. Simply strike above and below home row. Depending on the finger you moved, it'll know the key you're striking. I rarely have more than one finger off home row, per hand, when typing. You heard it here first.
- izcs 17y ago
  was thinking the same thing. i think i've something similar before, and something tells me it may not be practical.
mcav 17y ago
Anyone happen to know what software likely produced that video?
bjclark 17y ago
"A single finger manipulates items inside of an application. <insert finger painting>" Ding. Ding. Ding. We have #fail. It's pretty cool and all, but this isn't going to change anyone's life. The best idea here is just putting a big touch pad on the desktop in front of your monitor.
snprbob86 17y ago
What I find most compelling about this design is that it would support existing applications. This is a proposal for a new Window Manager, not a ground-up rewrite. Looking at the local context commands, it seems like traditional top-level menus could be automatically converted to something left-hand friendly.
jrockway 17y ago
Emacs is a 10 finger multi-touch UI. And it works very well.
- stcredzero 17y ago
  Multi-touch is right. A roommate once told me about a guy he knew who rigged up foot-pedal modifier keys to increase his Emacs productivity. And this was in the PS2 heyday! (No flames. I was an avid Emacs user in College & grad school.)
  - lg 17y ago
    I think Bill Clementson uses these: http://bc.tech.coop/blog/060131.html http://bc.tech.coop/blog/060131.html
  - jrockway 17y ago
    Foot pedals are indeed an excellent idea. There don't seem to be any available on the market that are programmable under non-Windows-XP, however, and I have not had the chance to build my own yet.
    - dkersten 17y ago
      I believe the kinesis keyboards have an (optional) foot pedal.
      - jrockway 17y ago
        I am not too excited by their keyboards, unfortunately. They do sell separate USB foot pedals, but they act as the mouse buttons until you reprogram them on a Windows computer. I don't know anyone that uses Windows.
        stcredzero 17y ago
        Get hold of a Win2K disk image and put it in VirtualBox. (No details given, but you should be able to get one, plus install key for, ahem, cheap.) You can connect the virtual image to the USB device. You might need one of the service packs to get USB to work, however.
gnubardt 17y ago
Interesting, would be cool to see worked up with one of thse http://www.technologyreview.com/computing/22358/ http://www.technologyreview.com/computing/22358/ (I saw Ken Perlin of the nyu media lab give a presentation on that earlier in the year and it was incredible, high resolution and cheap to make)
DannoHung 17y ago
Seems interesting, but why not use a fully dynamic window manager instead of the side to side thing?
surfmike 17y ago
Like this example of bold but concrete thinking on a future computing metaphor. Two of the concepts seem very heavily Palm Pre-inspired: The window management scheme (reminiscent of 'cards') and the gesture areas on each side, one of which is used to launch apps just like the Pre
zokier 17y ago
I'm not quite convinced that multi-touch panel that large would be practical, but smaller, one hand version could be nice. Imho it would also be better if the panel was on the side of a keyboard(ie in the place of numpad and arrow keys), not under.
tomstuart 17y ago
Not sure they should've let Patrick Bateman narrate the video. "This is Sussudio, a great, great song, a personal favorite."
alexandros 17y ago
I wonder if they could combine the keyboard with the touch surface. Not by having the surface be the keyboard, but by overlaying a traditional keyboard with touch sensitive key-sized tiles that together make a touch-sensitive area. Keys should still be pressable, but when you treat it like a touch surface it should behave so. If the gaps between keys are snall enough it should work for fingers. (not styluses though)
chrismear 17y ago
"Who would build something like this? Just as in the early days of the mouse and windowed GUI, a system based upon 10/GUI's principles would likely be best suited to a systems builder capable of full vertical integration." I wonder who they're thinking of.
mitko 17y ago
Scrolling alone already had decreased a lot of tension in my arms. 10/GUI would probably have great effect on RSI reduction. It is amazing how intuitive touch can be. I just love the multi-touch on the touchpad that Ubuntu for example supports. I never knew my old Thinkpad T43 would be able to do it but it does.
username 17y ago
I'd rather have more modifier keys and mouse buttons or modifier mouse buttons. Instead of pinching with fingers, you could move the mouse while holding another mouse button. For three or more finger gestures, you could hold down another mouse button.

171 points by zain 17y ago

Having a non-focused, non-active window arbitrarily take up 1/3 of my viewable monitor at all times would drive me nuts. It also makes no sense to me to go to the trouble of allowing a user to use all ten fingers, and then restricting application interaction to only two or three fingers. I work in rich, deep applications with tons of controls and buttons, if I'm going to go to the trouble of learning to use ten fingers to navigate a computer, you better allow me to make use of all of them withing the individual programs that I'm using.

stcredzero 17y ago
I think you're onto something. OS interaction/application switching should be simple, and the individual app interaction should be more complex. I think 4 app selection modes are almost too much. 3 is perfect. Prediction: I suspect that Apple is taking Exposé in exactly this direction.
- buro9 17y ago
  Re: the prediction. Doesn't it help to have a documented piece of prior art like the Linux demo in this video when it comes to patent defence and implementation on other platforms? Of course, R. Clayton Miller may have patented too. Or he could see it in the same vein as the safety match and just record it to give it away.
- TrevorJ 17y ago
  The real logical gap in the thinking here is that they are focusing on replacing the mouse but in reality, they are in fact replacing functions that can be executed simply and quickly with a keyboard. Application switching etc can be done efficiently with a keyboard. At the very least I question the wisdom of moving away from the modifier key paradigm because it is an elegant and easy method of interaction. I also believe that as we engage more and more of the body in a control interface, for it to be natural it is going to need to mimic real-world interaction so that we can rely on our lifetime of experience interacting with physical objects. The better solution in my opinion would contain: -2 mirrored screens so that the one under your hands can be glanced at for reference as well. -A UI approach that allows you to interact with objects and windows in the same ways you would with real objects. (Grabbing the edges moves the window, pushing down on it 'sticks' it in place, etc.)

da5e 17y ago

I wonder if they could with a tap bring up an virtual keyboard on the touchpad were the fingers already are and do away with a keyboard altogether. This sort of harkens back to the chord keyset.

aik 17y ago

Pretty cool. I could see my productivity increase from something like that.

th0ma5 17y ago

Multitouch innovators should take heed. A lot of the NUIGroup community is centered around hardware innovations, which is cool and all, but we need more and more software!

agazso 17y ago

I'm looking forward to use a multitouch touchpad with my desktop keyboard just as I currently do with my laptop. However I will definitely want to keep a mouse also when in need for more precision for tasks like photo editing etc. 10gui seems to be a very cool prototype and I guess that Apple and Microsoft are already working on things like this. I also spotted a weakness of this demo: 10gui uses the little finger to invoke contextual menus. Most of the people who cannot touch type with all ten fingers (and my guess this is about the 99% of all computer users :) will find this uncomfortable.

NathanKP 17y ago
I agree. A ten finger system is going to appeal greatly to those of us who already touch type and are used to using multiple fingers. But I suspect that many people aren't going to be too happy with the coordination required to use such a system. But I may be wrong. Perhaps when it comes down to it the interface will feel so intuitive that there will be no problems at all. We'll have to wait and see.
- timgasper 17y ago
  That was one of the biggest things I worry about - the learning curve of all the gestures. A mouse and keyboard are quite straightforward in terms of how the operate. A 10GUI interface would require a significant amount of training and practice. At which point maybe yes - it will become completely natural.

saurabh 17y ago

It certainly looks cool. I am just not sure about the ergonomics. What would really be awesome is not having to lift my hand at all by switching between touch mode and keyboard mode at a flick of a finger.

mhb 17y ago

tl;dw 30 seconds in and they haven't shown it yet?

gjm11 17y ago
It's worse than that. The first time they say anything about their interface is at about 2:00. That's extremely vague (it's a multitouch sensor! where your mouse would be!). No more specifics until about 4:00 (windows are arranged in only one dimension). They don't actually show the damn thing until 4:45. Oh, and all accompanied by the Most Annoying Music In The World.
- NathanKP 17y ago
  Patience. Any good idea requires a full explanation to build it up. I feel that they did the right thing by contrasting the mouse with their new system because it helped show the benefits of a ten figure multi-touch pad compared with a single point mouse.
  - DanielStraight 17y ago
    I'm not entirely convinced they showed such a benefit. I can already use multiple fingers to move windows by clicking anywhere on them... two of those fingers are just on buttons. Not to mention, I still only have two eyes. I can't be looking at 4 things on the screen at the same time anyway.
- viraptor 17y ago
  I understand you don't read many typical research papers? (5% abstract, 15% introduction, 15% history and justification, then comes the real meat...) I seriously felt like reading another HCI journal while watching this - almost the same structure :)
  - gjm11 17y ago
    I read plenty of research papers. They don't usually have a gee-isn't-this-cool narration and annoying background music. Also, if an academic paper's background material is familiar and boring then you can easily skim it and see where the new stuff begins. That's not so easy with a video.

bryansum 17y ago

The video mentions the problem of always having one's hands in the way with a direct interface. This could also be alleviated by moving the hands underneath the display ala the capacitive mouse from Microsoft Research: http://arstechnica.com/hardware/news/2009/10/video-microsoft-research-demoes-five-multitouch-mice.ars http://arstechnica.com/hardware/news/2009/10/video-microsoft...

jpwagner 17y ago

Phenomenal video. The ideas are definitely interesting, but I am amazed at the clarity of the presentation.

deyan 17y ago
Thanks for posting, it looks quite interesting. The interface is something that I can imagine myself getting adjusted to (for example, I am a big Opera user and use gestures all the time - once you get used to them they are phenomenal). I had a similar idea a while ago and might try to implement it this summer as a side project using Johnny Wii's tricks perhaps + Windows 7 (not sure how much of the interface I can pull off). I would be curious to see in particular how tired I am (i.e. my hands) after using such an interface for a while.

CWuestefeld 17y ago

That's cool as far as it goes. But what about the keyboard? The mouse is just one aspect of the computer interaction. I can't remove my keyboard from my desk just to make room for this mouse replacement. The keyboard seems a big barrier (literally!) for getting this device on the desktop.

aik 17y ago
The end of the presentation had an image of a possible solution - basically having the touch pad exist right below the keyboard, just like it is on any laptop built today. Definitely makes the keyboard/touchpad area quite a bit larger though.
- extension 17y ago
  If you imagine the touch pad being wide but not much taller than the one on a laptop, it starts to seem more practical.
inffcs00 17y ago
At the end of the video you can see the multitouch device integrated with a keyboard. :)
- catch23 17y ago
  it seems like the touch system should be able detect where your main 4 fingers are, and then show a rough virtual keyboard in its space with your 8 fingers on the home row automatically. The only thing lacking would be tactile feedback, but that's solvable. If people are accustomed to seeing the 10 dots representing the fingers, it should be easy to provide small visual cues to indicate if you're about to type a letter that isn't what you want.
  - ismarc 17y ago
    I type 80+ words a minute regularly. It would be insanely difficult to type at that speed without the little nubs on f and j keys, as well as the tactile feedback as fingers slide over keys preparing for what they're going to be typing next.

extension 17y ago

It seems pretty obvious to me that the most efficient and intuitive way to manage windows is just a boring list at the bottom/side of the screen, but I do hold out hope, like everyone else, that it turns out to be some flashy zooming 3D multitouch thing. It is refreshing though, to see research that dispenses with the legacy of GUI convention we've built up over the years, instead of just piling on new features.

bprater 17y ago

They've certainly put lots of thought into the design of this device/software. But the whole time I kept thinking: "what about the keyboard?" (Eventually, at the end, they showed the device in a traditional trackpad position below a keyboard.) I wish they had considered a way to make the keyboard a part of the device. Perhaps an on-screen keyboard, some haptics? The challenge with that part of the input device is that to touch type, you need that spacial reference a keyboard gives you.

tc 17y ago

While we're rethinking the human interface, could we please stop staggering typing keys and put frequently used keys like carriage return and backspace under strong fingers? [1] I was disappointed to see at the end of the 10gui video a concept keyboard with staggered keys and with backspace way over in the corner. If we can't move keyboards past the constraints imposed by mechanical typewriters, what hope do we have at changing the mouse paradigm? [1] Typematrix is the only keyboard maker I know of that currently does this while still building a relatively standard keyboard (and something that could be integrated into a laptop).

RevRal 17y ago
Colemak layout turns your caps lock into a backspace. This works surprisingly well, but I really wish the spacebar was split in half. And I've always felt shift+spacebar should = backspace.
- mjgoins 17y ago
  I use a keyboard with Control where Caps Lock would be on most keyboards (a sun unix keyboard). That combined with the fact that Control-H is backspace in a standard unix environment means I don't have to reach for the backspace key.
icefox 17y ago
If you have a Kinesis Advantage keyboard the backspace,delete space,enter are under your thumbs
lief79 17y ago
Not sure about the final released product, but the fingerworks prototypes were completely customizable.
atlei 17y ago
Hmmm.. I actually have both types of products. The Kinesis keyboard with backspace, ctrl, alt, enter, home etc using my thumb AND the iGesture pad from Fingerworks ( http://www.fingerworks.com/ http://www.fingerworks.com/ ) which is a large "mouse pad" where you can use gestures with 5 fingers (and it works for all applications). The brilliant part is that the iGesture pad fits perfectly on the top of the Kinesis keyboard (the left and right part of the keyboard is separated for ergonomic reasons). I've had the iGesture for a couple of years now - it will be a sad day when it dies as there are no replacements that I know of....maybe Apple will release a similar "mousepad" in the future ?

zain 17y ago

There's a similar device I read about with a hardware prototype that actually works. It doesn't have the software support of 10gui but it's basically a resistive touchscreen mapped to keypress macros (copy/paste, app switching, etc). Here's some pictures: http://www.flickr.com/photos/pauric/sets/72157616093748066/with/3488471659/ http://www.flickr.com/photos/pauric/sets/72157616093748066/w...

NathanKP 17y ago
Is that the same device? It doesn't look anything like the prototype design in the video. I also don't see any correlation between Pauric on Flickr and C. Miller of 10/GUI, at least not by exploring Pauric's Flickr profile or Twitter account. Perhaps you can explain why you think that those images are of 10/GUI's device? How did you find them?
- zain 17y ago
  You're right. It isn't the exact same device. I ran across it while doing more research on 10gui and mistakenly thought it was by the same person. I updated my post. Sorry about that.
  - NathanKP 17y ago
    Okay no problem. I thought that something was funny as I looked through those Flickr photos. ;)

Semiapies 17y ago

Talk about multi-touch reminds me of a friend's perennial complaint, "The future of interfaces seems designed to screw me over." He has a congenital deformity of the arms, but he can type (and is one of those clacky-IBM keyboard fetishists). With his feet, he can use mice and controllers. But multi-touch systems are beyond his ability - while he can pivot and shift to type remarkably quickly, he can't get more than one hand on something at the same time. Worse, each hand is effectively only one digit for him. And then, there are the folks who don't even have his degree of dexterity. I can't help thinking that it might be a good thing for general interface designers to consider the implications of disabilities for users of these designs, instead of shrugging that off onto providers of interfaces specifically for the disabled.

NathanKP 17y ago
That is a really good point. The software of any future interface will have to include support for older devices and techniques. But this may perhaps be a good thing because it would motivate the specific design of devices that will be more efficient for the disabled simply because it will no longer be possible for them to use the multi-touch interface.
gecko 17y ago
Without disagreeing, I also don't think it makes sense to discount potentially massive improvements in user interfaces merely because a portion of the population cannot use them.
- joe_the_user 17y ago
  No, but his problem actually points to this being a problem for anyone. It becomes a problem when the machine requires all of my dexterity just to accomplish a given task. I dislike all Apple products specifically because they require mouse operations to accomplish things where Linux and MS interfaces tend to have keyboard shortcuts.
  - deleted 17y ago
    [deleted]

zacechola 17y ago

I'd like to see it with the pad acting as the keyboard, too. though, that's more software implementation, while the video is talking hardware interface. Although, the problem with that kind of implementation is that it doesn't have the tactile feel of a keyboard. however, iPhone users seem to have acclimated after first bitching about it.

NathanKP 17y ago
I personally think that a real physical keyboard will always be preferred. 10Gui is doing the right thing by putting the keyboard above the multi-touch pad.
- lief79 17y ago
  http://www.fingerworks.com/faq.html http://www.fingerworks.com/faq.html Apparently people could get up to 70 wpm with what could be considered the prior version of this. For what it is worth, at least when I was involved with it, most people found a keyboard a lot easier to use for rapid typing.

TimothyFitz 17y ago

This basically already exist(ed): http://www.fingerworks.com/ST_product.html http://www.fingerworks.com/ST_product.html Apple bought them up and stopped making their products (but we got multitouch on the iPhone, an interesting trade-off).

lief79 17y ago
I had interned with Fingerworks prior to the formal release of the project. The main problem I heard about was the demand cost/issues. Outside of the proposed UI changes, Fingerworks product did all of this. I still wish I had bought one when I had the chance. Anyway, if they are serious about getting these made, I believe University of Delaware owns a fair amount of Wayne Westerman's patents, as the basic idea was the premise for his doctorate. He was a professor starting in '99, so that should make them easier to locate.

thomasfl 17y ago

A ten finger multitouch pad for desktops makes perfectly sense. The multitouch pad on mac book pro is way more pleasant to use than a mouse, but can't be used with an external monitor.

jodrellblank 17y ago
It isn't more pleasant to use than a mouse. If I have to scrape two surfaces together constantly to use a computer, I'd rather one of them wasn't the most sensitive part of my finger.

RevRal 17y ago

As for a keyboard, why not have the virtual buttons relative to where the the fingers are placed? As in, place all eight fingers onto the pad and this automatically becomes home row. Simply strike above and below home row. Depending on the finger you moved, it'll know the key you're striking. I rarely have more than one finger off home row, per hand, when typing. You heard it here first.

izcs 17y ago
was thinking the same thing. i think i've something similar before, and something tells me it may not be practical.

mcav 17y ago

Anyone happen to know what software likely produced that video?

bjclark 17y ago

"A single finger manipulates items inside of an application. <insert finger painting>" Ding. Ding. Ding. We have #fail. It's pretty cool and all, but this isn't going to change anyone's life. The best idea here is just putting a big touch pad on the desktop in front of your monitor.

snprbob86 17y ago

What I find most compelling about this design is that it would support existing applications. This is a proposal for a new Window Manager, not a ground-up rewrite. Looking at the local context commands, it seems like traditional top-level menus could be automatically converted to something left-hand friendly.

jrockway 17y ago

Emacs is a 10 finger multi-touch UI. And it works very well.

stcredzero 17y ago
Multi-touch is right. A roommate once told me about a guy he knew who rigged up foot-pedal modifier keys to increase his Emacs productivity. And this was in the PS2 heyday! (No flames. I was an avid Emacs user in College & grad school.)
- lg 17y ago
  I think Bill Clementson uses these: http://bc.tech.coop/blog/060131.html http://bc.tech.coop/blog/060131.html
- jrockway 17y ago
  Foot pedals are indeed an excellent idea. There don't seem to be any available on the market that are programmable under non-Windows-XP, however, and I have not had the chance to build my own yet.
  - dkersten 17y ago
    I believe the kinesis keyboards have an (optional) foot pedal.
    - jrockway 17y ago
      I am not too excited by their keyboards, unfortunately. They do sell separate USB foot pedals, but they act as the mouse buttons until you reprogram them on a Windows computer. I don't know anyone that uses Windows.
      - stcredzero 17y ago
        Get hold of a Win2K disk image and put it in VirtualBox. (No details given, but you should be able to get one, plus install key for, ahem, cheap.) You can connect the virtual image to the USB device. You might need one of the service packs to get USB to work, however.

gnubardt 17y ago

Interesting, would be cool to see worked up with one of thse http://www.technologyreview.com/computing/22358/ http://www.technologyreview.com/computing/22358/ (I saw Ken Perlin of the nyu media lab give a presentation on that earlier in the year and it was incredible, high resolution and cheap to make)

DannoHung 17y ago

Seems interesting, but why not use a fully dynamic window manager instead of the side to side thing?

surfmike 17y ago

Like this example of bold but concrete thinking on a future computing metaphor. Two of the concepts seem very heavily Palm Pre-inspired: The window management scheme (reminiscent of 'cards') and the gesture areas on each side, one of which is used to launch apps just like the Pre

zokier 17y ago

I'm not quite convinced that multi-touch panel that large would be practical, but smaller, one hand version could be nice. Imho it would also be better if the panel was on the side of a keyboard(ie in the place of numpad and arrow keys), not under.

tomstuart 17y ago

Not sure they should've let Patrick Bateman narrate the video. "This is Sussudio, a great, great song, a personal favorite."

alexandros 17y ago

I wonder if they could combine the keyboard with the touch surface. Not by having the surface be the keyboard, but by overlaying a traditional keyboard with touch sensitive key-sized tiles that together make a touch-sensitive area. Keys should still be pressable, but when you treat it like a touch surface it should behave so. If the gaps between keys are snall enough it should work for fingers. (not styluses though)

chrismear 17y ago

"Who would build something like this? Just as in the early days of the mouse and windowed GUI, a system based upon 10/GUI's principles would likely be best suited to a systems builder capable of full vertical integration." I wonder who they're thinking of.

mitko 17y ago

Scrolling alone already had decreased a lot of tension in my arms. 10/GUI would probably have great effect on RSI reduction. It is amazing how intuitive touch can be. I just love the multi-touch on the touchpad that Ubuntu for example supports. I never knew my old Thinkpad T43 would be able to do it but it does.

username 17y ago

I'd rather have more modifier keys and mouse buttons or modifier mouse buttons. Instead of pinching with fingers, you could move the mouse while holding another mouse button. For three or more finger gestures, you could hold down another mouse button.