5 ms·

Instead, the greatest value will be in second-generation applications that provide total recall and augmented cognition. Imagine being able to call up (and shar

Instead, the greatest value will be in second-generation applications that provide total recall and augmented cognition. Imagine being able to call up (and share) everything you have ever seen, or read the transcripts for every conversation you ever had, alongside the names and faces of everyone you ever met. Imagine having supplemental contextual information relayed to you automatically so you could win any argument or impress your date. This is the common wearable computing utility argument, but in practice, it doesn't seem to pan out. Gordon Bell digitized much of his life, and everything for the past ten or twenty years. Phone calls, emails, a photo every sixty seconds, more when his heart rate increased. He hardly ever went back to it. Revisiting it was so rare as to be a notable event in and of itself. Rather, he found people with whom he had conversations would go to him and use him as a reference library. Bradley Rhodes' Remembrance Agent was an Emacs thing which actively indexed and cross-referenced anything you were typing with things you had written before. He's working on Glass now, afaik: http://www.remem.org/ http://www.remem.org/ It isn't a generally useful solution because it requires that you live inside of Emacs. Today you'd need something that could index across multiple devices, multiple independent cloud storage systems, multiple independent accounts, etc.; or something that was locked into a single ecosystem and you lived entirely there, but then you'd develop "blind spots" for things that occurred elsewhere, like how you stop hanging out with certain friends because the only way they communicate with you is via Facebook but you turned off all of Facebook's notifications. There was an essay a long time ago, a writer had been filing every link and every note into a pre-Evernote piece of notetaking/hyperlinking/PIM software, something with an X or a Z in the name, but I can't recall it or the piece. The essay was about an article he was writing during which the software brought up a saved article and a note he had written, and forgotten about, in a creepily timely and seemingly prescient moment. He wondered at what point the software needs to get credit for providing the research and the associations. "Forgetting" is a key part of human existence that most wearable applications tend to ignore. "You were last here with [your ex]" says Foursquare. "You haven't talked with [your ex] in a while, make her day by leaving a message on her wall" says Facebook. Using someone's precise words against them can be emotionally cruel. Legally, you're expected to forget all of the specific details of things like trade secrets and company processes when you leave a job; how is that to be reconciled with your perfect, digital memory? None of these things are being actively explored. We already hit the "second generation" of applications as described by the author; it's the third generation that interests me.

kentlyons 14y ago
I wouldn't be surprised if something akin to the Remembrance Agent ends up on Glass. For his PhD thesis, Brad generalized the ideas of the Remembrance Agent to the broader notion of JITIR - Just-In-Time Information Retrieval. He built a web sidebar in that work, but that probably isn't the right app either. However the idea of doing a continuous search based on what you're doing would fit it well. Before it was just searching your own stuff, but now they'd hook in more generally to Google's search. Couple that with something Siri like where the speech recognition is feed as input for a proactive search. I'm still not sure this is a compelling application for a wide audience, but for several of the early everyday users of wearable computers (including myself) it had a lot of appeal.
- vitovito 14y ago
  I had forgotten about what ended up in his thesis. It's linked to from the Remembrance Agent site if anyone else is interested: http://www.bradleyrhodes.com/Papers/rhodes-phd-JITIR.pdf http://www.bradleyrhodes.com/Papers/rhodes-phd-JITIR.pdf (PDF) "Implicit query" is another term for it. On the desktop side, one of the most realized solutions was the Dashboard prototype: http://nat.org/dashboard/ http://nat.org/dashboard/ I've long argued that designing multi-modal interfaces is where applications need to go. We're seeing this now with desktop, web and mobile apps all running against the same APIs, but what it really means is you're more likely to be able to also design a UI that's really an audio-only interface, whispering to you continuously via a Bluetooth headset, or a wrist-mounted UI, or... etc. It also means you can design and build something useful for desktop users, and then mobile users, and then scaled even further down for wearable users. Real-time continuous Google searching as a desktop sidebar, or rendered on an iPad or iPod Touch display linked to desktop input as a second screen, would be a start.
sigkill 14y ago
Your post actually fires up quite a lot of my neurons. I'm thinking about a perfect text to speech system which transcribes whatever you say into pure text files. Maybe it'll generate a new file every ten minutes, every half an hour or possibly even every six hours. That'll leave you with tons of text with respect to what you said. Now integrate that with some kind of a keylogger on all your devices (like a computer, tablet, and phone) which can track states and applications. Also, by virtue of having a current generation smart phone you can also tag locations and timestamps, at least to cell tower triangulation accuracy if not GPS accuracy. This would run transparent to the user. At the end of the day, he'll have pure text files that can be searched. Implementing a google-like algorithm modified to work on flat files would be great. It'd be basically like the Remembrance Agent but more advanced. I don't know how I'd feel about using such a system, but I'm reasonably sure that we're on the way there indirectly. Heck, most of us voluntarily not only record our lives, but also post it publicly. What's wrong with recording all our life (right from that websearch on the tablet, to what you're ordering at the restaurant whose name you forgot but you visited it on the day right after your best friend got married) if it's going to make it life easier. As such, I've noticed personally that I have stopped storing information in my head unless I use it regularly, but instead I store pointers to the location of the information, like whether I did a search {if yes, then what were the keywords, and what was the result's position}, or did I look it up on a forum or maybe even a real book. /thinking-out-loud

kentlyons 14y ago

I wouldn't be surprised if something akin to the Remembrance Agent ends up on Glass. For his PhD thesis, Brad generalized the ideas of the Remembrance Agent to the broader notion of JITIR - Just-In-Time Information Retrieval. He built a web sidebar in that work, but that probably isn't the right app either. However the idea of doing a continuous search based on what you're doing would fit it well. Before it was just searching your own stuff, but now they'd hook in more generally to Google's search. Couple that with something Siri like where the speech recognition is feed as input for a proactive search. I'm still not sure this is a compelling application for a wide audience, but for several of the early everyday users of wearable computers (including myself) it had a lot of appeal.

vitovito 14y ago
I had forgotten about what ended up in his thesis. It's linked to from the Remembrance Agent site if anyone else is interested: http://www.bradleyrhodes.com/Papers/rhodes-phd-JITIR.pdf http://www.bradleyrhodes.com/Papers/rhodes-phd-JITIR.pdf (PDF) "Implicit query" is another term for it. On the desktop side, one of the most realized solutions was the Dashboard prototype: http://nat.org/dashboard/ http://nat.org/dashboard/ I've long argued that designing multi-modal interfaces is where applications need to go. We're seeing this now with desktop, web and mobile apps all running against the same APIs, but what it really means is you're more likely to be able to also design a UI that's really an audio-only interface, whispering to you continuously via a Bluetooth headset, or a wrist-mounted UI, or... etc. It also means you can design and build something useful for desktop users, and then mobile users, and then scaled even further down for wearable users. Real-time continuous Google searching as a desktop sidebar, or rendered on an iPad or iPod Touch display linked to desktop input as a second screen, would be a start.

sigkill 14y ago

Your post actually fires up quite a lot of my neurons. I'm thinking about a perfect text to speech system which transcribes whatever you say into pure text files. Maybe it'll generate a new file every ten minutes, every half an hour or possibly even every six hours. That'll leave you with tons of text with respect to what you said. Now integrate that with some kind of a keylogger on all your devices (like a computer, tablet, and phone) which can track states and applications. Also, by virtue of having a current generation smart phone you can also tag locations and timestamps, at least to cell tower triangulation accuracy if not GPS accuracy. This would run transparent to the user. At the end of the day, he'll have pure text files that can be searched. Implementing a google-like algorithm modified to work on flat files would be great. It'd be basically like the Remembrance Agent but more advanced. I don't know how I'd feel about using such a system, but I'm reasonably sure that we're on the way there indirectly. Heck, most of us voluntarily not only record our lives, but also post it publicly. What's wrong with recording all our life (right from that websearch on the tablet, to what you're ordering at the restaurant whose name you forgot but you visited it on the day right after your best friend got married) if it's going to make it life easier. As such, I've noticed personally that I have stopped storing information in my head unless I use it regularly, but instead I store pointers to the location of the information, like whether I did a search {if yes, then what were the keywords, and what was the result's position}, or did I look it up on a forum or maybe even a real book. /thinking-out-loud