6 ms·

Giving Stephen Hawking a voice

216 points by Semetric 12y ago

k-mcgrady 12y ago
Is it open source or based on open source? The article seems to switch between both at random. If they've made the entire system open source that would be incredible. I think something like this, which can improve millions of lives, is the perfect project for an open source community to work on. Sick people shouldn't have to pay for this and I'm sure lots of people would be very happy to dedicate time to improving it. It's also good to know that if Intel decided to stop work on it the users who rely on it so heavily aren't screwed - the software can continue to be improved and will always be available.
- devnonymous 12y ago
  From http://iq.intel.co.uk/how-intel-keeps-stephen-hawking-talking/ http://iq.intel.co.uk/how-intel-keeps-stephen-hawking-talkin... > Professor Hawking has been using his new software for several months while > Lama and her team have been debugging and fine-tuning it. It’s almost > finished, and when it is, Intel plans to make the system available to the > open source community.
  - k-mcgrady 12y ago
    Great! Thanks for finding that.
  - bitwize 12y ago
    Interesting. It says Hawking would rather stick to his old system than switch to a new one, which conflicts with reports I read like a year ago that suggested he was interested in a direct brain interface. But perhaps it's not so surprising. He's old, and past a certain age you just don't have time to relearn everything. Your time is better spent squeezing the juice out of what you have.
    - k-mcgrady 12y ago
      >> "It says Hawking would rather stick to his old system than switch to a new one" Huh? The quote you're replying to says he "has been using his new software for several months". Am I missing something? Is he switching back after testing? I know that he doesn't ever plan on changing the 'voice' as it has become known as his own voice but I don't see why he would test new software and then not use it. To test it you will have to learn it.
      - Someone 12y ago
        Three years ago, he was looking for an assistant. http://www.theverge.com/2011/12/29/2668408/stephen-hawking-technical-assistant-speech-system http://www.theverge.com/2011/12/29/2668408/stephen-hawking-t... states that, at the time, his computer was running Windows XP, so he definitely wasn't on the bleeding edge. However, that has changed a bit; he upgraded to Windows 7 (http://www.hawking.org.uk/the-computer.html http://www.hawking.org.uk/the-computer.html)
      - bitwize 12y ago
        His system still uses the cheek trigger and still works more or less the same way, but improvements have been made to its word prediction and other details of its operation.
kmfrk 12y ago
What software did Ebert use? I remember him talking about paying English researchers to synthesize his own voice, but I don't think anything materialized.
- quarterto 12y ago
  CereProc. They're Scottish. http://www.rogerebert.com/rogers-journal/finding-my-own-voice http://www.rogerebert.com/rogers-journal/finding-my-own-voic... https://www.cereproc.com/ https://www.cereproc.com/
- agildehaus 12y ago
  It materialized, but I think there were limitations and he ended up not using it: https://www.youtube.com/watch?v=_0KUw3xr7cA https://www.youtube.com/watch?v=_0KUw3xr7cA And it wouldn't be possible for Hawking. Ebert's system was put together using hundreds of hours of recordings from Skiskel & Ebert At the Movies.
  - cleverjake 12y ago
    Actually the recordings from At The Movies weren't usable because so much of them had movies playing in the background. They used all of his DVD commentaries (mainly from criterion collection, if memory serves).
btown 12y ago
Yahoo has a much more technically detailed article about the software, including a video showing the software in action. https://www.yahoo.com/tech/how-intel-helped-stephen-hawking-communicate-with-104112393054.html https://www.yahoo.com/tech/how-intel-helped-stephen-hawking-... Direct link to the screen-capture video: https://www.youtube.com/watch?v=mPU6mnM2i-k https://www.youtube.com/watch?v=mPU6mnM2i-k TL;DR Hawking only has a single reliable, low-latency binary signal (facial muscle movements), so his interface has been a constantly-moving cursor that he can "click" when it's over the next symbol/command he wishes to select. The innovations here are in the interpretation of those selections: he now has autosuggest for text (designed specifically for him based on the corpus of his works) and shortcuts for filesystem management. I'm looking forward to seeing when the source code is released, or when a paper is written. Just looking at the data-entry video, for instance, there are interesting parallels between the timing specifications for Hawking's Yes/No dialog and GUI design for dialogs for non-disabled users - in both cases, if there's not enough spacing in between buttons, or orderings are unpredictable, it's much easier for someone to mis-click!
- kagebe 12y ago
  I think it's not only the interactive elements, but also the way information is displayed. I actually think he (and others) might benefit from a really responsive desktop environment, especially compared to the Windows floating window manager. The videos you linked to show this very well in my opinion: - In the longer one with Hawking and the System he's using it to type and read Wikipedia, all the while quite some screen estate is wasted with (for him probably unusable) title bars, partially hidden desktop icons in the background and the browser partially behind his input software. - The "data entry" video shows Notepad being opened and being partially hidden by the input UI. That does not seem useful. I would rather use apps that automatically fit themselves to available space and predefined layouts for multiple apps or a dynamic tiling approach.
  - digi_owl 12y ago
    Sounds a bit like the Metro UI, or any number of tiling window managers on *nix.
- 12y ago
deleted 12y ago
[deleted]
acd 12y ago
Where is the source Luke?
- ofcapl_ 12y ago
  so inspiring article.
kps 12y ago
I wonder how much work went into making it still sound exactly like a DECtalk.
applehammer 12y ago
If they had been more agile and iterated to him releases rather than a big bang release he would have embraced the technology more easily.
- xahrepap 12y ago
  Unless it takes him a long time to adjust to even small changes. Or maybe his old software was faster for him or more stable until recently?
omaranto 12y ago
Arun Mehta wrote an essay called "When a Button Is All That Connects You to the World" for the book Beautiful Code (2007) about speech software designed for Stephen Hawking. I don't think the system described in Mehta's article is this one, though.
omaranto 12y ago
Arun Mehta wrote an essay called "When a Button Is All That Connects You to the World" for the book Beautiful Code (2007) about speech software designed for Stephen Hawking. I don't think the system described in Mehta's article is this one, though.
dang 12y ago
Url changed from http://www.wired.co.uk/news/archive/2014-12/02/stephen-hawking-intel-communication-system http://www.wired.co.uk/news/archive/2014-12/02/stephen-hawki..., which points to this.
mwti 12y ago
I was the lead engineer of this project in 2012-2013. To anyone thinking of getting into Intel or these types of technologies STAY AWAY. This project caused a huge scandal among the Silicon Valley elite on whether or not Hawking should be allowed to use microwave-based mind reading technologies (which are classified and he obviously cannot use). I got ensnared in politics and it has ruined my career. I have become a targetted individual like Myron May and Aaron Alexis. Technology behind closed doors is 1000x what your typical HN reader can even imagine. The elite are good at making you think your C++ compiler, web browser, and iPad are "advanced". They aren't. I live in constant fear now, far away from California. Don't mess with these people!! I love Hawking and hope he gets the best, but he is being used like a pawn. http://i.imgur.com/t2bolYC.jpg http://i.imgur.com/t2bolYC.jpg
- NhanH 12y ago
  Why was this comment downvoted? Was it blatantly wrong and too conspiracy-ish, or did I miss anything?
  - cypher543 12y ago
    I think a look at his comment history will give you the answer.
  - chc 12y ago
    Well, the obvious answer is that it sounds like the product of mental illness, which is a) not something that we want to encourage people to shoot back at and b) can't really lead to good conversation.
    - mwti 12y ago
      The only mental illness I suffer from is a propensity for C# and .NET... even though I know better. ;) https://www.youtube.com/watch?v=hi8jMRMsEJo#t=1114 https://www.youtube.com/watch?v=hi8jMRMsEJo#t=1114 http://i.imgur.com/1ry8whI.jpg http://i.imgur.com/1ry8whI.jpg
      - 12y ago
byuu 12y ago
Interesting article ... > "I'm trying to make a software version of Stephen's voice so that we don't have to rely on these old hardware cards," says Wood. So back in 2010 we had someone help us extract the program ROM code from the SNES DSP-n coprocessors (used in games like Pilotwings and Mario Kart.) It turns out these chips were NEC uPD7725 DSPs. There was basically only one very terse document on how the chip worked, and no emulators for it, so I had to write one. Had a bit of help in fixing the overflow flag calculations from Cydrak. A while later, I spoke briefly through a liaison with Sam Blackburn (who was then Stephen Hawking's assistant) back in 2011'ish. They were looking for permission to use my uPD7725 emulation code (which I said yes to, obviously.) Apparently the Speech Plus text synthesizer uses NEC uPD7720s. This is basically the same chip and ISA, but with less ROM/RAM. It's a neat little fact, but not too surprising. These DSPs are really versatile, and different programs can make them do very different things. Reading this article, it sounds like the effort was as yet unsuccessful, though :( (It's also important to note that the uPD7720 is probably an infinitesimal part of the overall system, so I suppose they ran into additional problems.)

216 points by Semetric 12y ago

Is it open source or based on open source? The article seems to switch between both at random. If they've made the entire system open source that would be incredible. I think something like this, which can improve millions of lives, is the perfect project for an open source community to work on. Sick people shouldn't have to pay for this and I'm sure lots of people would be very happy to dedicate time to improving it. It's also good to know that if Intel decided to stop work on it the users who rely on it so heavily aren't screwed - the software can continue to be improved and will always be available.

devnonymous 12y ago
From http://iq.intel.co.uk/how-intel-keeps-stephen-hawking-talking/ http://iq.intel.co.uk/how-intel-keeps-stephen-hawking-talkin... > Professor Hawking has been using his new software for several months while > Lama and her team have been debugging and fine-tuning it. It’s almost > finished, and when it is, Intel plans to make the system available to the > open source community.
- k-mcgrady 12y ago
  Great! Thanks for finding that.
- bitwize 12y ago
  Interesting. It says Hawking would rather stick to his old system than switch to a new one, which conflicts with reports I read like a year ago that suggested he was interested in a direct brain interface. But perhaps it's not so surprising. He's old, and past a certain age you just don't have time to relearn everything. Your time is better spent squeezing the juice out of what you have.
  - k-mcgrady 12y ago
    >> "It says Hawking would rather stick to his old system than switch to a new one" Huh? The quote you're replying to says he "has been using his new software for several months". Am I missing something? Is he switching back after testing? I know that he doesn't ever plan on changing the 'voice' as it has become known as his own voice but I don't see why he would test new software and then not use it. To test it you will have to learn it.
    - Someone 12y ago
      Three years ago, he was looking for an assistant. http://www.theverge.com/2011/12/29/2668408/stephen-hawking-technical-assistant-speech-system http://www.theverge.com/2011/12/29/2668408/stephen-hawking-t... states that, at the time, his computer was running Windows XP, so he definitely wasn't on the bleeding edge. However, that has changed a bit; he upgraded to Windows 7 (http://www.hawking.org.uk/the-computer.html http://www.hawking.org.uk/the-computer.html)
    - bitwize 12y ago
      His system still uses the cheek trigger and still works more or less the same way, but improvements have been made to its word prediction and other details of its operation.

kmfrk 12y ago

What software did Ebert use? I remember him talking about paying English researchers to synthesize his own voice, but I don't think anything materialized.

quarterto 12y ago
CereProc. They're Scottish. http://www.rogerebert.com/rogers-journal/finding-my-own-voice http://www.rogerebert.com/rogers-journal/finding-my-own-voic... https://www.cereproc.com/ https://www.cereproc.com/
agildehaus 12y ago
It materialized, but I think there were limitations and he ended up not using it: https://www.youtube.com/watch?v=_0KUw3xr7cA https://www.youtube.com/watch?v=_0KUw3xr7cA And it wouldn't be possible for Hawking. Ebert's system was put together using hundreds of hours of recordings from Skiskel & Ebert At the Movies.
- cleverjake 12y ago
  Actually the recordings from At The Movies weren't usable because so much of them had movies playing in the background. They used all of his DVD commentaries (mainly from criterion collection, if memory serves).

btown 12y ago

Yahoo has a much more technically detailed article about the software, including a video showing the software in action. https://www.yahoo.com/tech/how-intel-helped-stephen-hawking-communicate-with-104112393054.html https://www.yahoo.com/tech/how-intel-helped-stephen-hawking-... Direct link to the screen-capture video: https://www.youtube.com/watch?v=mPU6mnM2i-k https://www.youtube.com/watch?v=mPU6mnM2i-k TL;DR Hawking only has a single reliable, low-latency binary signal (facial muscle movements), so his interface has been a constantly-moving cursor that he can "click" when it's over the next symbol/command he wishes to select. The innovations here are in the interpretation of those selections: he now has autosuggest for text (designed specifically for him based on the corpus of his works) and shortcuts for filesystem management. I'm looking forward to seeing when the source code is released, or when a paper is written. Just looking at the data-entry video, for instance, there are interesting parallels between the timing specifications for Hawking's Yes/No dialog and GUI design for dialogs for non-disabled users - in both cases, if there's not enough spacing in between buttons, or orderings are unpredictable, it's much easier for someone to mis-click!

kagebe 12y ago
I think it's not only the interactive elements, but also the way information is displayed. I actually think he (and others) might benefit from a really responsive desktop environment, especially compared to the Windows floating window manager. The videos you linked to show this very well in my opinion: - In the longer one with Hawking and the System he's using it to type and read Wikipedia, all the while quite some screen estate is wasted with (for him probably unusable) title bars, partially hidden desktop icons in the background and the browser partially behind his input software. - The "data entry" video shows Notepad being opened and being partially hidden by the input UI. That does not seem useful. I would rather use apps that automatically fit themselves to available space and predefined layouts for multiple apps or a dynamic tiling approach.
- digi_owl 12y ago
  Sounds a bit like the Metro UI, or any number of tiling window managers on *nix.
12y ago

deleted 12y ago

[deleted]

acd 12y ago

Where is the source Luke?

ofcapl_ 12y ago
so inspiring article.

kps 12y ago

I wonder how much work went into making it still sound exactly like a DECtalk.

applehammer 12y ago

If they had been more agile and iterated to him releases rather than a big bang release he would have embraced the technology more easily.

xahrepap 12y ago
Unless it takes him a long time to adjust to even small changes. Or maybe his old software was faster for him or more stable until recently?

omaranto 12y ago

Arun Mehta wrote an essay called "When a Button Is All That Connects You to the World" for the book Beautiful Code (2007) about speech software designed for Stephen Hawking. I don't think the system described in Mehta's article is this one, though.

omaranto 12y ago

dang 12y ago

Url changed from http://www.wired.co.uk/news/archive/2014-12/02/stephen-hawking-intel-communication-system http://www.wired.co.uk/news/archive/2014-12/02/stephen-hawki..., which points to this.

mwti 12y ago

I was the lead engineer of this project in 2012-2013. To anyone thinking of getting into Intel or these types of technologies STAY AWAY. This project caused a huge scandal among the Silicon Valley elite on whether or not Hawking should be allowed to use microwave-based mind reading technologies (which are classified and he obviously cannot use). I got ensnared in politics and it has ruined my career. I have become a targetted individual like Myron May and Aaron Alexis. Technology behind closed doors is 1000x what your typical HN reader can even imagine. The elite are good at making you think your C++ compiler, web browser, and iPad are "advanced". They aren't. I live in constant fear now, far away from California. Don't mess with these people!! I love Hawking and hope he gets the best, but he is being used like a pawn. http://i.imgur.com/t2bolYC.jpg http://i.imgur.com/t2bolYC.jpg

NhanH 12y ago
Why was this comment downvoted? Was it blatantly wrong and too conspiracy-ish, or did I miss anything?
- cypher543 12y ago
  I think a look at his comment history will give you the answer.
- chc 12y ago
  Well, the obvious answer is that it sounds like the product of mental illness, which is a) not something that we want to encourage people to shoot back at and b) can't really lead to good conversation.
  - mwti 12y ago
    The only mental illness I suffer from is a propensity for C# and .NET... even though I know better. ;) https://www.youtube.com/watch?v=hi8jMRMsEJo#t=1114 https://www.youtube.com/watch?v=hi8jMRMsEJo#t=1114 http://i.imgur.com/1ry8whI.jpg http://i.imgur.com/1ry8whI.jpg
    - 12y ago

byuu 12y ago

Interesting article ... > "I'm trying to make a software version of Stephen's voice so that we don't have to rely on these old hardware cards," says Wood. So back in 2010 we had someone help us extract the program ROM code from the SNES DSP-n coprocessors (used in games like Pilotwings and Mario Kart.) It turns out these chips were NEC uPD7725 DSPs. There was basically only one very terse document on how the chip worked, and no emulators for it, so I had to write one. Had a bit of help in fixing the overflow flag calculations from Cydrak. A while later, I spoke briefly through a liaison with Sam Blackburn (who was then Stephen Hawking's assistant) back in 2011'ish. They were looking for permission to use my uPD7725 emulation code (which I said yes to, obviously.) Apparently the Speech Plus text synthesizer uses NEC uPD7720s. This is basically the same chip and ISA, but with less ROM/RAM. It's a neat little fact, but not too surprising. These DSPs are really versatile, and different programs can make them do very different things. Reading this article, it sounds like the effort was as yet unsuccessful, though :( (It's also important to note that the uPD7720 is probably an infinitesimal part of the overall system, so I suppose they ran into additional problems.)