5 ms·

Bill Buxton: Multi-Touch Systems that I Have Known (1982)

115 points by johnx123-up 14y ago

ot 14y ago
Why the (1982) in the title? The original article is from 2007, and this is a revision from 2012.
- johnx123-up 14y ago
  For the context: Furthermore, during the development of the iPhone, Apple was very much aware of the history of multi-touch, dating at least back to 1982, and the use of the pinch gesture, dating back to 1983.
creamyhorror 14y ago
Iluminating, exhaustive article and illustrated history of multitouch. I wonder how many of these papers, prototypes and devices the patent examiners came across when they were deciding on the prospective multitouch patent now owned by Apple. Money quote: ---- Multi-touch technologies have a long history. To put it in perspective, my group at the University of Toronto was working on multi-touchin 1984 (Lee, Buxton & Smith, 1985), the same year that the first Macintosh computer was released, and we were not the first. Furthermore, during the development of the iPhone, Apple was very much aware of the history of multi-touch, dating at least back to 1982, and the use of the pinch gesture, dating back to 1983. This is clearly demonstrated by the bibliography of the PhD thesis of Wayne Westerman, co-founder of FingerWorks, a company that Apple acquired early in 2005, and now an Apple employee Westerman, Wayne (1999). Hand Tracking,Finger Identification, and Chordic Manipulation on a Multi-Touch Surface. U of Delaware PhD Dissertation: http://www.ee.udel.edu/~westerma/main.pdf In making this statement about their awareness of past work, I am not criticizing Westerman, the iPhone, or Apple. It is simply good practice and good scholarship to know the literature and do one's homework when embarking on a new product. What I am pointing out, however, is that "new" technologies - like multi-touch - do not grow out of a vacuum. While marketing tends to like the "great invention" story, real innovation rarely works that way. In short, the evolution of multi-touch is a text-book example of what I call "the long-nose of innovation." ---- Interesting examples relating to pinch-to-zoom: [o] 1983: Video Place / Video Desk (Myron Krueger) His use of many of the hand gestures that are now starting to emerge can be clearly seen in the following 1988 video, including using the pinch gesture to scale and translate objects: http://youtube.com/watch?v=dmmxVA5xhuo http://youtube.com/watch?v=dmmxVA5xhuo [o] 1991: Digital Desk(Pierre Wellner, Rank Xerox EuroPARC, Cambridge) Clearly demonstrated multi-touch concepts such as two finger scaling and translation of graphical objects, using either a pinching gesture or a finger from each hand, among other things. This page makes it quite clear how improvements in this subfield of tech, like in many other fields, are evolutionary and build upon existing ideas in the literature. Many potential implementations have been proposed and experimented with. What role do patents play in this picture? What is the breadth of Apple's pinch-to-zoom patent - how far does it extend beyond smartphones? Are any of these devices close enough to the patent to legally count as prior art? Thanks for this great submission. Buxton has been referenced on HN a few times before, but it's my first time reading his page.
- taligent 14y ago
  In NONE of those cases did anybody use a pinch gesture to zoom anything. So no. None of those legally count as prior art.
  - jrmg 14y ago
    Look in the 1998 video (http://youtube.com/watch?v=dmmxVA5xhuo http://youtube.com/watch?v=dmmxVA5xhuo) around 4:30. You could perhaps argue that's 'scaling' not 'zooming', but that's a very pedantic position, and not one I personally would entertain.
    - deleted 14y ago
      [deleted]
    - recoiledsnake 14y ago
      Direct link to the pinch to zoom part. http://www.youtube.com/watch?feature=player_detailpage&v=dmmxVA5xhuo#t=267s http://www.youtube.com/watch?feature=player_detailpage&v... So, tacking on a "BUT ON A CAPACITATIVE SCREEN" got Apple the patent?
      - deltaqueue 14y ago
        Hardly. The screen on Buxton's multi-touch tablet in '85 was capacitive! The patent system is so broken.
    - Someone 14y ago
      Devil's advocate: It is neigh impossible to judge from the video, but to me, the video shows less of a direct link between hand movement and what happens on the display. It is not as if the hand really grabs the screen. I also got the impression that he had to issue a command, chosen from the top of the screen (outside the video image) to switch from scaling to rotation. Another argument could be that this user does not actually seem to touch the screen (at least, that is my impression). It is likely that all these are due to technological limitations, but if I had to defend this as "not prior art" in court, I would try and work from those observations. There is little else to argue about, as a lot of the interactions are there, in a primitive (technologically) way.
xiaoma 14y ago
Slightly off-topic: Does anybody know where the videos of Bill's excellent talk titled What if Leopold didn't have a piano can be found? I saw part of the lecture online several years ago and it was one of the most insightful things I've seen, The story of the sculpting instructor who did an A/B test on his students, grading half of them entirely on the cumulative weight of their assignments was particularly eye-opening. I've googled many times since and never found the full multi-hour video again.
Terretta 14y ago
Thank goodness for companies like Apple that, when needing to rapidly introduce new technologies for users, acquire small businesses already invested in the space like Fingerworks for multi-touch, or Pixo for iPod's interface. Seems much better for us entrepreneurs than Samsung's approach. In the build versus buy equation, I want big companies to calculate they're better off to license or acquire.
- Terretta 14y ago
  It's curious that Hacker News disagrees with the idea it's better for big businesses to buy small companies than to copy them.
  - ktizo 14y ago
    I don't think that was the disagreement, I think it was more about the false comparison. I mean, do you honestly think that Samsung never acquires small businesses and Apple never copies?
  - mcantelon 14y ago
    I think HN understands that the expense and time involved in having to license every simple idea is a barrier to innovation. Licensing should be reserved for protecting the kind of things that require substantial investment -- implementation details, for example -- rather than simple ideas.
- creamyhorror 14y ago
  Let's not mix arguments here: - It's a good thing that Apple acquires smaller companies for their technology/patents instead of building around them, sure. This is an economic calculation that Apple made that turned out well for those companies. - It's not a good thing that any companies possess patents sufficiently broad to cover most of an idea, instead of a specific implementation. - It's not a good thing that companies possess patents for which apparent prior art exists, or which are obvious, as seems to be shown by Buxton's page. (The patent office's definition of "obvious" is that the implementation can be created by combining two or more existing patents or pieces of prior art - which are demonstrated on the page.) - It's not a good thing that companies possessing such patents sue other companies for infringing them, in the sense that it sets off a patent war and makes everyone more concerned about protecting themselves by trying to acquire patents on anything possible. I think this can lead to a chilling effect on competition and innovation in the software market. I don't know what Samsung would have done if the technology it wanted was under patent by small acquirable companies, but that's a hypothetical, because the point of this page is that the patent in question should perhaps not have been granted in the first place. If researchers were making multi-touch interfaces on touchscreens a decade or two earlier, and even displaying scaling functionality in their demos, then clearly an idea was 'plundered' from the public domain and privatised with the issuing of this patent.
dj2stein9 14y ago
Why isn't Bill Buxton suing Apple for a billion dollars?
- ktizo 14y ago
  Perhaps he has class.
  - malandrew 14y ago
    True, but the schadenfreude in me secretly hopes he sues the bejeezus out of Apple and then donates all the money to people who need to defend themselves against Apple's patent trolling. Apple, more than any other company, not only exhibits but flaunts the "long-nose of innovation" in everything they say or do. They need a healthy dose of reality. I think the best illustrative art they could have used in the trial is to hand everyone a rubber band and asked them to make it larger using one hand. I bet you 99% of them would place the band between their index finger and thumb or thumb and all four fingers. They are the two most obvious solutions for any creature with an opposable thumb.
jeremyarussell 14y ago
It strikes me as odd that this didn't last long on the front page or get upvoted nearly as much as all the samsung vs apple submissions that had a large amount of people defending apple in the comments. Yet this, this extremely valid and informational piece gets pushed down the list like nothing. This is why I generally don't post on these submissions anyways, it feels like they are all one sided in defense of apple all the time, nothing but samsung is a thief that copies apple, with no mention of what apple did to xerox and Steve Jobs openly stating that he did indeed take their idea because they weren't going to do anything with it. It's a bit disheartening is all.

115 points by johnx123-up 14y ago

ot 14y ago

Why the (1982) in the title? The original article is from 2007, and this is a revision from 2012.

johnx123-up 14y ago
For the context: Furthermore, during the development of the iPhone, Apple was very much aware of the history of multi-touch, dating at least back to 1982, and the use of the pinch gesture, dating back to 1983.

creamyhorror 14y ago

Iluminating, exhaustive article and illustrated history of multitouch. I wonder how many of these papers, prototypes and devices the patent examiners came across when they were deciding on the prospective multitouch patent now owned by Apple. Money quote: ---- Multi-touch technologies have a long history. To put it in perspective, my group at the University of Toronto was working on multi-touchin 1984 (Lee, Buxton & Smith, 1985), the same year that the first Macintosh computer was released, and we were not the first. Furthermore, during the development of the iPhone, Apple was very much aware of the history of multi-touch, dating at least back to 1982, and the use of the pinch gesture, dating back to 1983. This is clearly demonstrated by the bibliography of the PhD thesis of Wayne Westerman, co-founder of FingerWorks, a company that Apple acquired early in 2005, and now an Apple employee Westerman, Wayne (1999). Hand Tracking,Finger Identification, and Chordic Manipulation on a Multi-Touch Surface. U of Delaware PhD Dissertation: http://www.ee.udel.edu/~westerma/main.pdf In making this statement about their awareness of past work, I am not criticizing Westerman, the iPhone, or Apple. It is simply good practice and good scholarship to know the literature and do one's homework when embarking on a new product. What I am pointing out, however, is that "new" technologies - like multi-touch - do not grow out of a vacuum. While marketing tends to like the "great invention" story, real innovation rarely works that way. In short, the evolution of multi-touch is a text-book example of what I call "the long-nose of innovation." ---- Interesting examples relating to pinch-to-zoom: [o] 1983: Video Place / Video Desk (Myron Krueger) His use of many of the hand gestures that are now starting to emerge can be clearly seen in the following 1988 video, including using the pinch gesture to scale and translate objects: http://youtube.com/watch?v=dmmxVA5xhuo http://youtube.com/watch?v=dmmxVA5xhuo [o] 1991: Digital Desk(Pierre Wellner, Rank Xerox EuroPARC, Cambridge) Clearly demonstrated multi-touch concepts such as two finger scaling and translation of graphical objects, using either a pinching gesture or a finger from each hand, among other things. This page makes it quite clear how improvements in this subfield of tech, like in many other fields, are evolutionary and build upon existing ideas in the literature. Many potential implementations have been proposed and experimented with. What role do patents play in this picture? What is the breadth of Apple's pinch-to-zoom patent - how far does it extend beyond smartphones? Are any of these devices close enough to the patent to legally count as prior art? Thanks for this great submission. Buxton has been referenced on HN a few times before, but it's my first time reading his page.

taligent 14y ago
In NONE of those cases did anybody use a pinch gesture to zoom anything. So no. None of those legally count as prior art.
- jrmg 14y ago
  Look in the 1998 video (http://youtube.com/watch?v=dmmxVA5xhuo http://youtube.com/watch?v=dmmxVA5xhuo) around 4:30. You could perhaps argue that's 'scaling' not 'zooming', but that's a very pedantic position, and not one I personally would entertain.
  - deleted 14y ago
    [deleted]
  - recoiledsnake 14y ago
    Direct link to the pinch to zoom part. http://www.youtube.com/watch?feature=player_detailpage&v=dmmxVA5xhuo#t=267s http://www.youtube.com/watch?feature=player_detailpage&v... So, tacking on a "BUT ON A CAPACITATIVE SCREEN" got Apple the patent?
    - deltaqueue 14y ago
      Hardly. The screen on Buxton's multi-touch tablet in '85 was capacitive! The patent system is so broken.
  - Someone 14y ago
    Devil's advocate: It is neigh impossible to judge from the video, but to me, the video shows less of a direct link between hand movement and what happens on the display. It is not as if the hand really grabs the screen. I also got the impression that he had to issue a command, chosen from the top of the screen (outside the video image) to switch from scaling to rotation. Another argument could be that this user does not actually seem to touch the screen (at least, that is my impression). It is likely that all these are due to technological limitations, but if I had to defend this as "not prior art" in court, I would try and work from those observations. There is little else to argue about, as a lot of the interactions are there, in a primitive (technologically) way.

xiaoma 14y ago

Slightly off-topic: Does anybody know where the videos of Bill's excellent talk titled What if Leopold didn't have a piano can be found? I saw part of the lecture online several years ago and it was one of the most insightful things I've seen, The story of the sculpting instructor who did an A/B test on his students, grading half of them entirely on the cumulative weight of their assignments was particularly eye-opening. I've googled many times since and never found the full multi-hour video again.

Terretta 14y ago

Thank goodness for companies like Apple that, when needing to rapidly introduce new technologies for users, acquire small businesses already invested in the space like Fingerworks for multi-touch, or Pixo for iPod's interface. Seems much better for us entrepreneurs than Samsung's approach. In the build versus buy equation, I want big companies to calculate they're better off to license or acquire.

Terretta 14y ago
It's curious that Hacker News disagrees with the idea it's better for big businesses to buy small companies than to copy them.
- ktizo 14y ago
  I don't think that was the disagreement, I think it was more about the false comparison. I mean, do you honestly think that Samsung never acquires small businesses and Apple never copies?
- mcantelon 14y ago
  I think HN understands that the expense and time involved in having to license every simple idea is a barrier to innovation. Licensing should be reserved for protecting the kind of things that require substantial investment -- implementation details, for example -- rather than simple ideas.
creamyhorror 14y ago
Let's not mix arguments here: - It's a good thing that Apple acquires smaller companies for their technology/patents instead of building around them, sure. This is an economic calculation that Apple made that turned out well for those companies. - It's not a good thing that any companies possess patents sufficiently broad to cover most of an idea, instead of a specific implementation. - It's not a good thing that companies possess patents for which apparent prior art exists, or which are obvious, as seems to be shown by Buxton's page. (The patent office's definition of "obvious" is that the implementation can be created by combining two or more existing patents or pieces of prior art - which are demonstrated on the page.) - It's not a good thing that companies possessing such patents sue other companies for infringing them, in the sense that it sets off a patent war and makes everyone more concerned about protecting themselves by trying to acquire patents on anything possible. I think this can lead to a chilling effect on competition and innovation in the software market. I don't know what Samsung would have done if the technology it wanted was under patent by small acquirable companies, but that's a hypothetical, because the point of this page is that the patent in question should perhaps not have been granted in the first place. If researchers were making multi-touch interfaces on touchscreens a decade or two earlier, and even displaying scaling functionality in their demos, then clearly an idea was 'plundered' from the public domain and privatised with the issuing of this patent.

dj2stein9 14y ago

Why isn't Bill Buxton suing Apple for a billion dollars?

ktizo 14y ago
Perhaps he has class.
- malandrew 14y ago
  True, but the schadenfreude in me secretly hopes he sues the bejeezus out of Apple and then donates all the money to people who need to defend themselves against Apple's patent trolling. Apple, more than any other company, not only exhibits but flaunts the "long-nose of innovation" in everything they say or do. They need a healthy dose of reality. I think the best illustrative art they could have used in the trial is to hand everyone a rubber band and asked them to make it larger using one hand. I bet you 99% of them would place the band between their index finger and thumb or thumb and all four fingers. They are the two most obvious solutions for any creature with an opposable thumb.

jeremyarussell 14y ago

It strikes me as odd that this didn't last long on the front page or get upvoted nearly as much as all the samsung vs apple submissions that had a large amount of people defending apple in the comments. Yet this, this extremely valid and informational piece gets pushed down the list like nothing. This is why I generally don't post on these submissions anyways, it feels like they are all one sided in defense of apple all the time, nothing but samsung is a thief that copies apple, with no mention of what apple did to xerox and Steve Jobs openly stating that he did indeed take their idea because they weren't going to do anything with it. It's a bit disheartening is all.