27 ms·
Seeing AI for iOS
- LeoNatan25 9y agoWhenever I take a picture with the camera button on the left, it shows a loading indicator and the app crashes. Not a great first impression. Coming from a company the size of Microsoft, such trivial crashes should have been caught.
- cartyboi29 9y agoLol you wanna prize? All you HN niggas have a problem with everything
- booobooo1 9y ago^ Microsoft employee
- epmaybe 9y agoWhich iOS device do you have?
- LeoNatan25 9y ago7 Plus
- irq 9y agoI am also on a 7 Plus and I do not have this problem.
- rosstex 9y ago6S here, I also have this crashing problem. On iOS 11 Beta.
- mikeash 9y agoDon't run OS betas if you don't want crashes.
- rosstex 9y agoI figured that was the issue, which is specifically why I included that information. I am guessing that the commenter I replied to may have also been running a beta OS. And it turns out I was correct! But thank you for your snark anyway.
- mikeash 9y agoNot really snark. It is, as evidenced here, apparently an issue some people are confused about.
- seeingai 9y agoHi Leo, I am guessing you are using iOS 11 Beta. We are currently working on support for iOS 11 beta, and will shortly be updating the app. Stay tuned! Anirudh from Seeing AI team
- mandeepj 9y agoHi Anirudh, It's a really commendable effort from MS. An immense benefit for people who have vision challenges. Are you guys hiring? If yes then what are the skills you are looking for?
- 2pointsomone 9y agoAnirudh and team: Excellent product release. Thanks for doing such important work for marginalized people! We need more tech like this and less like that.
- taylorhalliday 9y agoNice job Anirudh! You and the team did some awesome work!
- dcw303 9y agoUS App Store only at this stage it seems. Pity, I'd like to try this. edit: I'm wrong. it's in other stores as well, but not in the Australian app store, which is the one that I tried.
- DanielAlimohd 9y agoI was able to download from the Canadian App Store
- dcw303 9y agoSorry, I should have finished my coffee before typing. What I should have said was it's not available in the Australian App Store.
- seeingai 9y agoFor the launch, its available in USA, Canada, India, New Zealand, Singapore and Hong Kong. And gradually, should be available in more countries. Anirudh from Seeing AI team
- TheCoreh 9y agoJust curious, what is the reasoning behind such rollouts? Localization?
- Mtinie 9y agoPresumably licensing and legal coverage.
- ChristianGeek 9y agoYes...translating to Australian is a slow and tedious process.
- mamp 9y agoHopefully this can be expedited by translating from the New Zealand version which is released. Just lower the prior probability of sheep and you're done!
- chenster 9y agoI use "not a hotdog" app whenever in need of a hotdog. It's amazingly accurate.
- pramodzion 9y agoI guess you also live stream using Periscope and not any other service.
- chenster 9y agoIt's supposed to be a joke.
- EA 9y agoI am recommending this for my elderly family members with poor eyesight. This could greatly increase their quality of life.
- booleandilemma 9y agoIt correctly identified my refrigerator and bookshelf. Color me impressed. Things like this make articles like this one seem silly: https://www.madebymany.com/stories/what-if-ai-is-a-failed-dream https://www.madebymany.com/stories/what-if-ai-is-a-failed-dr...
- ianai 9y agoIt was very unjudgey about my living room too.
- ClassyJacket 9y agoNot available in the Australian store. Ah, I forgot my entire country doesn't exist.
- robotresearcher 9y agoThey can't recognize things upside down.
- megablast 9y agoFunny that you are only upset that it is not in your country.
- nharada 9y agoYes, YES, this is what I'm talking about Microsoft. I'm surprised how muted the reaction is from HN here. On the technical side, this is a perfect example of how AI can be used effectively, and is a (very obvious in hindsight) application of the cutting edge in scene understanding and HCI. There are quite a few recent and advanced techniques rolled into one product here, and although I haven't tried it out yet it seems fairly polished from the video. A whitepaper of how this works from the technical side would be fascinating, because even though I'm familiar with the relevant papers it's a long jump between the papers and this product. On the social side, I think this is a commendable effort, and a fairly low hanging fruit to demonstrate the positive power of new ML techniques. On a site where every other AI article is full of comments (somewhat rightfully) despairing about the negative social aspects of AI and the associated large scale data collection, we should be excited about a tool that exists to improve lives most of us don't even realize need improving. This is the kind of thing I hope can inspire more developers to take the reins on beneficial AI applications.
- 52-6F-62 9y agoLow hanging fruit or not, this is the kind of effort that will sway public opinion. This is the direction that needs to be undertaken with as much or more fervour than business automation applications. I'm inspired!
- doomlaser 9y agoThese are all definitely applications of ML we've seen around in different places, but this is the first package I've seen to put them together in an integrated way on a smartphone. I'm also impressed with Microsoft's effort here. It looks pretty nice, especially the different 'channels' metaphor for the various ML processing tasks (Scene, barcode, people, short text etc) for the visually impaired.
- mishurov 9y agoI'd rather say that machine learning is an application of mathematical statistics.
- 9y ago
- engulfme 9y agoIf I remember correctly, this came out of a OneWeek project - Microsoft's company-wide weeklong hackathon. Very cool to see a final published version of this!
- seeingai 9y agoYou are right, this project evolved from the week long 2015 company wide Hackathon, with a participation of close to 16,000 employees, where people are allowed to take a week off to build anything of interest. Many accessibility projects have come out, off the top of my head: (1) Eye Controlled Wheelchair for people with ALS (2) Color Binoculars - App for people who are colorblind (3) Hearing AI - App including live speech recognition and sounds recognition for people with profound hearing loss (4) Learning Tools for OneNote - https://www.onenote.com/learningtools https://www.onenote.com/learningtools (5) Dictate - Speech based keyboard control to type emails/documents ( http://dictate.ms http://dictate.ms), built originally for people with hand dexterity issues A few of my peers now work full time on projects originated at the hackathon. - Anirudh from Seeing AI team
- ddlutz 9y agoLooks like this is only for iOS, any plans on coming to Android?
- ve55 9y agoIf I have a friend that is visually impaired and is using this, I have to consent to their phone recording me and analyzing me and sending all of that data off to who knows where. And this is just from my perspective - someone who is not visually impaired. For the person who is, every single thing they look at and read is going to be recorded and used. It's an unfortunate situation for people to put in, and I'm sure everyone will choose using improvements like this over not using them. As much as I would love to see a focus on privacy for projects like this, I don't imagine it happening any time soon, given how powerful the data involved is. I imagine a future where AI assistants like this are commonplace, and there is no escaping them.
- scalablenotions 9y agoAs far as public use goes, it's always been considered legal / ethical to photograph people in public places. One does not know where the "information" in a photograph will end up, but since they are already technically in public view, it's something of a moot point.
- njsubedi 9y agoWhere do you want to escape?
- visarga 9y agoIn the future, most AI will happen locally, not remote, because of privacy, latency and cost. There is a lot of research in adapting models to fit phones (such as reducing the size of the NN by 20x-100x and still having almost all the accuracy).
- bramd 9y agoYes, you are right, but things like face recognition in this app run locally already. I tried it in airplane mode. Also, check this blog from a blind user and some interesting comments: http://chrishofstader.com/seeing-ai-first-impressions/ http://chrishofstader.com/seeing-ai-first-impressions/
- remind_me_again 9y agoWhat's commendable also is that Microsoft has released a free app on Apple store.
- parish 9y agoI'm impressed. Good job MS
- mattchamb 9y agoJust as an extra piece of information for people, the person presenting the videos in that page is Saqib Shaikh, who is a developer at Microsoft. Earlier on HN, there was a really interesting video of him giving a talk about how he can use the accessibility features in visual studio to help him code. https://www.youtube.com/watch?v=iWXebEeGwn0 https://www.youtube.com/watch?v=iWXebEeGwn0
- ian0 9y agoWow. You can imagine a near future where this, a small wearable camera and an earphone could really make a big difference to a persons daily life. Screw Siri, thats a real AI assistant :)
- Joeri 9y agoFor years I've wanted a pair of glasses that overlays the world with captions, putting names and "you know them from" next to people, address numbers next to buildings, etc. Google glass made me hopeful, but it fizzled out.
- zepto 9y agoYou mean in the future Siri might be able to do this stuff?
- jimmcslim 9y agoApparently there is some research and development taking place into a cane that has a sonar or some similar sensor embedded into its tip, and transmits haptic feedback to the user via the handle. These assistive technologies are fantastic, but I wonder whether a vision-impaired person who has adapted to life before they were available would be weary about adopting them, on the basis that if it breaks or becomes unavailable that they would have maybe lost the skills and sharpness of other senses to compensate?
- deleted 9y ago[deleted]
- deleted 9y ago[deleted]
- mechaman 9y agoDoes it do hot dog or not?
- rb666 9y agoWell, it needs some work, but pretty cool nonetheless, can see where it was going with this :) http://imgur.com/jxsWrEq http://imgur.com/jxsWrEq
- vinitagr 9y agoThis is quite an amazing technology. With new products like HoloLens and this, i think Microsoft is finally coming around.
- arized 9y agoLooks amazing, I really need to dive into machine learning more this year... Waiting impatiently for UK release to give it a try!
- coworkerblues 9y agoDoes this mean RIP OrCam (the other startup form mobileye creator which basically does this as a full hardware / software solution) ? http://www.orcam.com/ http://www.orcam.com/
- Finbarr 9y agoI'm pretty blown away by this. I took a picture of myself in the mirror with the scene description feature, and it said "probably a man standing in front of a mirror posing for the camera". I took a picture of the room in front of me and it said "probably a living room". Think I'll be experimenting with this for days.
- extra88 9y ago"probably a man standing in front of a mirror posing for the camera" I would hope it could get this one right, there's a massive amount of training data to recognize it.
- MrJagil 9y agoThe videos explaining it are really nice https://youtu.be/dqE1EWsEyx4 https://youtu.be/dqE1EWsEyx4
- GeekyBear 9y agoOne of my friends from college has limited vision and the feature to read text aloud will be a game changing convenience. He has a magnifier in his home, but it isn't portable and is limited to working only with documents and images that can lie flat. edit: After speaking with my friend, he already uses a popular app called KNFB Reader that works very well on short text and documents, but costs $100. On the plus side, it works on Android or iOS.
- zyanyatech 9y agoThe use of AI and ML for application purposes is starting to get to a point that it can really be used for problem solving, we did a demo app similar use of this technology, https://zyanya.tech/hashtagger-android https://zyanya.tech/hashtagger-android I am going to give Seeing AI a try as well, but I totally understand why a research department would like to have a demo as an Application available for public.
- jtbayly 9y agoUnimpressed. Took a picture of a ceiling fan and it said, "probably a chair sitting in front of a mirror." Took a pic of a dresser, and it said "a bedroom with a wooden floor." Tried the ceiling fan again and got an equally absurd answer. Deleted app.
- cilea 9y agoI think this is also cool for learning English as well. An English learner who'd like to express what s/he sees can verify with AI's response.
- acheilies 9y agoBrilliant app!!
- leipert 9y agoAwesome technology; and todays SMBC [1] seems to be related. [1]: https://www.smbc-comics.com/comic/the-real-me https://www.smbc-comics.com/comic/the-real-me
- haterswillhate 9y agoIos as a platform is under constant controll by apple app store. Any reason, your app can be removed
- scarface74 9y agoThe text recognition from documents is amazingly primitive. It doesn't use any type of spell checking to make a best guess at what a word is. It's straight text recognition. On the other hand the "short text" feature works amazingly well to read text is sees from the camera. It's fast and accurate when reading text even at some non optimal angles. How do you get it to try to recognize items that the camera sees? Edit: Oops. I guess it would help if I swiped right....
- ouid 9y agoWhy on earth would you want spell check on text-to-speech for the blind? Spell check is for people writing messages, not for people reading messages. This is a terrible idea.
- scarface74 9y agoI'm not talking about the feature that reads text aloud. I'm referring to the OCR "document" feature that makes transcription errors that could easily be fixed if it had any level of text correction.
- ouid 9y agoregardless, text correction is a tool for the writer, not a tool for the reader. In general, text-correction does one of two things, it makes an error better, or it makes an error worse, unless you can guarantee that it only does the first thing, you have no idea what the total value of the operation will be. Additionally, presumably its capacity to make an error better is not better than a human's capacity to make that error better. Even over very small timescales, tens of seconds, a computer confronted with a typo will never outperform a human confronted with the same typo.
- scarface74 9y agoA simple text correction could tell the difference between the word "rnethod" and "method" and know one is more likely an error in ocr.
- hackpert 9y agoThis is amazing. I don't know if a lot of people here realize this, but it is really hard to pull off this level of integration of different computer vision components (believe me, I've tried). Microsoft has really outdone themselves this time.
- rasengan0 9y agoI love the low vision pitch as there are a dearth of low vision resources particularly those hit with age related macular degeneration. I wonder if they are any censored items in the backend that may limit functionality -- Seeing AI won't be seeing any sex toys...
- NicoJuicy 9y agoMicrosoft app ' Office Lens ' is the only app I use for screenshots of documents in Android. I see part of the tech is used in this app to take a screenshot also. Love it
- armandososa 9y agoRemember this? http://i.dailymail.co.uk/i/pix/2013/05/13/article-2323625-19C04224000005DC-589_634x510.jpg http://i.dailymail.co.uk/i/pix/2013/05/13/article-2323625-19... when it was a really big deal that Microsoft agreed to help Apple and release some software for their platform?
- deleted 9y ago[deleted]
- blaines 9y agoThere was a demo of this on the series Bill Nye Saves The World