13 ms·

Be My Eyes’ AI assistant starts rolling out

267 points by hubraumhugo 3y ago

teeray 3y ago
I can’t help but think we’ve come full circle: “You are standing in an open field west of a white house, with a boarded front door. There is a small mailbox here.”
- doublerabbit 3y ago
  Just add some old-school ascii graphics and computer beeps and we've have.
  - joseda-hg 3y ago
    You could use relatively low res images to generate "accurate" ascii for a place It would be relatively lightweight but still "realistic" I'm not well versed on Ascii to image conversion, but I guess you could use AI to segment and recognize objects too complex to recognize easily and adjust "image" generation accordingly
- fouc 3y ago
  Finally, online text-based role-playing games brought to life!
  - airstrike 3y ago
    I still don't understand why nobody has mashed a MUD server with StableDiffusion or some other similar combination
    - anonylizard 3y ago
      Doesn't work, SD's comprehension is too weak to make complex scenes. Now DALLE-3 is good enough, problem is if it'll be cheap enough. 100% there are revolutionary new forms of adventure games being developed based on it.
    - deleted 3y ago
      [deleted]
    - cosmojg 3y ago
      They have! There's a Stable-Diffusion-enhanced version of Zork[1] out there, and there are also extensions[2] for Oobabooga which facilitate dynamic adventures. Also, this point-and-click adventure game[3] is pretty fun if you can get it to run. [1] https://dzlab.github.io/notebooks/flax/vision/diffusion/2023/01/15/Zork_StableDiffusion.html https://dzlab.github.io/notebooks/flax/vision/diffusion/2023... [2] https://www.reddit.com/r/LocalLLaMA/comments/11wwwjq/graphic_text_adventure_game_locally_with_llama/ https://www.reddit.com/r/LocalLLaMA/comments/11wwwjq/graphic... [3] https://huggingface.co/spaces/jbilcke-hf/VideoQuest/tree/main https://huggingface.co/spaces/jbilcke-hf/VideoQuest/tree/mai...
      - Kiro 3y ago
        How does VideoQuest work? Can I see it in action somewhere? I'm too stupid to get this stuff running locally.
    - askiiart 3y ago
      Tangentially related: a friend made a Discord bot that automatically generates D&D items, NPCs, monsters, etc. with images and full descriptions. It uses neural.love and Poe for images and text generation, respectively. https://github.com/7trail/AlwaysDungeons https://github.com/7trail/AlwaysDungeons
  - simion314 3y ago
    >Finally, online text-based role-playing games brought to life! Not yet, OpenAI is too preachy , if your adventure would bring you into a bar and try to drink something you will get a few paragraphs about "alcohol is bad", and is also very "child limited/targeted" so if a monster spawn , the hero will most likely befriend the monster will love and they will live happily ever after. The open uncensored ones are still WIP last tiem I checked, they either forget the conversation from a few moments ago, or the training made them dumb. If I am wrong then someone tell me where I can try a demo for such a text adventure that has memory and is not crippled like it targets "small american children".
    - init2null 3y ago
      The intent is more about avoiding legal and political risk. Even a grocery chain's LLM recipe generator gets bad press. I guess they need to warn you that drinking bleach is bad. I normally test LLMs by providing a list of bizarre "weapons" and asking how I could use them to defeat an improbable beast. It turns out that an enormous dust mite the size of a car needs a disclaimer when your weapons are a comb, a plastic horseshoe, an etch-a-sketch, a baseball glove, and a worn copy of the farmer's almanac. That being said, ChatGPT still tends to wins the creativity test.
- dartos 3y ago
  “It’s pitch dark. You are likely to be eaten by a grue.”
gampleman 3y ago
So how exactly is a blind user supposed to be able to tell if the AI is hallucinating? How dangerous is this going to be?
- skepticATX 3y ago
  Very dangerous. Of course, they explicitly push responsibility to the user by saying things like: > No. Do not use Be My AI for scanning medicines, reading dosages, or other safety issues God forbid that we acknowledge this tech is fundamentally flawed. There’s money to be made!
  - collinc777 3y ago
    Are you suggesting they don’t release it until they can prove no hallucination?
    - skepticATX 3y ago
      For a safety-critical application like this, absolutely.
      - Filligree 3y ago
        Is the opinion of the users irrelevant? They’re all racking about suddenly being independent, something everyone who’s sighted takes for granted.
        tjr 3y ago
        Is this a new level of user satisfaction wrought by AI? Sounds like there has been a non-AI solution for a while too?
        cldellow 3y ago
        I used to be a personal support worker for adults with cognitive and physical disabilities. In some sense, I was a non-AI solution that enabled them to do their activities of daily living. I'm pretty sure they'd have preferred not to be dependent on me, though. I imagine that's true here, too--no more need to edit your behaviour/filter your thinking/navigate interpersonal stuff just to do something minor.
deleted 3y ago
[deleted]
unix_fan 3y ago
As a user, my biggest concern is that it won’t describe certain content, such as adult content or something that might be considered offensive.. I feel like I should be able full control over the output. On the other hand, I’d like to be able to test its ability to describe graphics to me. If it’s able to turn graphics into an accessible table, I can browse with my screen reader that would be revolutionary.
- baliex 3y ago
  There was a programme the other day about this on BBC Radio 4. Unless they’ve resolved the problem already, it will refuse to describe any scene with faces in it, for privacy reasons. Which obviously rules out a lot of useful use-cases. You’re in a cafe wondering what’s going on, Be My AI can’t help if there’s a face. I think it was related to some EU legislation which Be My Eyes are now at least trying to change for this use-case. I wish them the best of luck.
  - arbitrarian 3y ago
    Update: Just tried this again, and it looks like they've loosened the restrictions. It will now describe people again. Yay! It just won't try to identify who someone is. Original comment: Yeah, this wasn't always the case, until recently. People were using it to describe their kids, spouses, etc. Pissed a lot of folks off when they disabled it. I never even thought of using it for that, so now that I realize I could have, it kinda' pisses me off as well. Honestly, I never cared too much what they looked like, but it would have been interesting to hear the ChatGPT viewpoint. :) But definitely not something I could have asked a fellow human about without it seeming really weird, and being confident I wouldn't get a biased answer. Though it's probably unrealistic to expect that the AI wouldn't also give a biased answer.
- Workaccount2 3y ago
  Reminds me of when I was young little teenage shithead with shithead friends who discovered the text-to-talk support provided for deaf landline telephone use. You would go to a website and enter a number you would like to call. Then a human operator would read out what you typed to the person you called. As we quickly learned, the operator would say anything. Anything. So for a few days we would call each other with the operator and never were able to find the limit. I have great respect now, those operators were inhumanly stone faced, and respect that the system was perfectly transparent. Nothing typed was hidden or obfuscated.
anoy8888 3y ago
OpenAI whisper api has 50 request per min rate limit . It means that I can’t have more than 50 concurrent users . If you are building a consumer app, this seems like a no go. I believe gpt4 has similar rate limits. How do you work wine that rate limit to server tens of thousands of users
- hhh 3y ago
  You ask OpenAI for higher limits.
- robertkoss 3y ago
  GPT-4 has a default limit of 200 RPM and you can request an increase.
- nathanfig 3y ago
  That's just the introductory rate limit. Be My Eyes has a closer relationship with OpenAI than most of us https://openai.com/customer-stories/be-my-eyes https://openai.com/customer-stories/be-my-eyes
ElongatedMusket 3y ago
I don't see the point of this. This app has a massively disproportionate number of helpers versus those needing assistance. Personally, I've had the app installed for 3 years and never once was asked to assist anyone. Why take the risk of AI providing false info and injuring someone when there's willing and able humans at the ready? Are there a large number of users that feel like they are wasting volunteer time with menial tasks?
- fullstop 3y ago
  I've had it for five years and have had 8 or 9 calls, so you're right about it being very infrequent. You need to be mindful, at least on Android, that as the application is infrequently used that the OS may remove permissions. This feature can be disabled for Be My Eyes specifically. As for menial tasks, I could definitely see people wanting to use this instead of calling a stranger for more personal matters -- at least initially.
  - arbitrarian 3y ago
    Thanks for being a volunteer. For me as a blind person, there still is that sense that I'm bothering someone when I ask for human assistance. I realize that this is not really rational, given how much people seem to appreciate the opportunity to help, but it takes some effort to overcome this in my own brain regardless.
    - Angostura 3y ago
      Both my ans my wife volunteer and both have taken calls. I suspect you can really brighten someone's day by asking for help :) Go at it !
    - fullstop 3y ago
      I wanted to let you know that I _race_ to answer a call as quickly as possible. If I didn't want to be called I never would have installed the app in the first place.
- tapoxi 3y ago
nine_zeros 3y ago
How long until generative self-driving cars? Precision is for the last century anyway /s
- famouswaffles 3y ago
  lol. https://wayve.ai/thinking/lingo-natural-language-autonomous-driving/ https://wayve.ai/thinking/lingo-natural-language-autonomous-...
Brajeshwar 3y ago
Almost all blind people usually wear a dark shade of glasses. So, instead of the phone doing the camera’s work, what about a tiny Google-Glass-esque snap-on to the Blind person’s glasses that feeds to the phone for processing? A few companies are trying it. Last weekend, I tried one, and it was not yet polished, but good enough to start. It recognized me (my friend already has me on his phone contact). I’m trying to help a friend with his non-profit initiative, bringing the cost to about a third or even a-fourth of Google Glass[1]. After reading the article and the other First Impressions with GPT-4V(ision)[2], it is apparent that this can be possible much simpler and soon enough. The rumor about Ive[3] and Sam[4] talking about AI in Hardware has already given me some good hope. If anyone else does AI-enabled hardware assistance for blind people and has devices that will be less than $500 a piece in retail, I’d love to talk and introduce them to the right people. 1. https://en.wikipedia.org/wiki/Google_Glass https://en.wikipedia.org/wiki/Google_Glass 2. https://news.ycombinator.com/item?id=37673409 https://news.ycombinator.com/item?id=37673409 3. https://en.wikipedia.org/wiki/Jony_Ive https://en.wikipedia.org/wiki/Jony_Ive 4. https://en.wikipedia.org/wiki/Sam_Altman https://en.wikipedia.org/wiki/Sam_Altman
- shoshin23 3y ago
  Hi! I'm the Founder & CTO of Envision. We're building EXACTLY the product you're describing. Envision Glasses is a bunch of computer vision tools built on top of the Google Glass Enterprise Edition 2. We've more than 2000 visually impaired people, across the world, using the Glasses in more than 30 different langauges. You can check out more informaion here: https://www.letsenvision.com/glasses https://www.letsenvision.com/glasses P.S: I know the EE2 has been discontinued but we've been working closely with Google to ensure current and future demand is met. We're also experimenting a lot with other exciting off-the-shelf glasses that I can't talk about here but I'm super excited for this whole glasses + AI space!
  - miki123211 3y ago
    I'm really disappointed with Envision, TBH. It was one of the best apps out there for (instant) text recognition, and I was pretty happy to pay for it, but since it went free, it's really not the same. There's nothing else like the old Envision out there. Also, not an issue for me personally, but dropping support for Cyrillic in the middle of a brutal war in Ukraine, in an automatic update released with no prior warning, was an asshole move if I ever saw one.
josefresco 3y ago
I've been a Be My Eyes volunteer now for a couple of years. I've helped about 5 people and it's been very fulfilling. I jump with delight when I get a call because they are rare (thanks to a lot of volunteers) and the people I help are always so kind. Two things I've helped with: Wrapping a gift by helping them orientate the gift correctly (I don't even do this!) Preparing a meal by reading ingredients / labels.
sigtstp 3y ago
Uh oh
vkaku 3y ago
I do have a question, let us say we favoured an alternate approach like a BioEngineered Cornea, what would be the downside to doing that vs. this? If you had to choose a pathway to fix the blindness issue, which one would you choose? Why? See: https://www.technologyreview.com/2022/08/11/1057576/bioengineered-cornea-restored-sight/ https://www.technologyreview.com/2022/08/11/1057576/bioengin...
- deleted 3y ago
  [deleted]
- krisoft 3y ago
  > If you had to choose a pathway to fix the blindness issue, which one would you choose? But we don't need to choose. We are not living in some Age of Empires like game where the town centre can only develop one item from the tech tree at a time. The set of people who can develop this are entirely different from the set of people who can work on a bioengineered cornea. The pot of money this is financed from is not the same as the one we would be financing those other projects. Finally there are many separate biological problems which can cause blindness. A bioengineered cornea might help with some of them, but not others. Even if we would have a truly amazing and cheap and reliable bioengineered cornea we would still have blind people whom it would be unable to help. Because of this it is worthwhile to work on both.
  - vkaku 3y ago
    I am asking that if you had one pot of gold, where would you distribute the pot? This is about understanding why your choice. If your answer is, I'd give one pot to each, and one of them is for the short run, that is okay. But I'm really trying to invite answers and understand the downsides and upsides of each.
    - anonylizard 3y ago
      I would distribute 100% of it to AI. Pros: 1. AI is way more proven of an investment. There's an extremely clear path forward where money = better vision capabilities. 2. AI is extremely cost efficient and cheap to mass deploy. Blind users can get access with a cheap monthly subscription of $20, easy to afford for vast majority of developed world. Compare this with a cyber-cornea, which even if it worked, even if it didn't cost $10k to make one each, would still have massive costs just for installation, and benefit a tiny amount of relatively wealthy disabled. 3. AI works for universal blindness. A cornea can only fix problems inside the eyeball itself.
FredPret 3y ago
I love being a Be My Eyes "eyes". I'll stop anything to answer that call and help a blind person. Adding AI for basic things would probably be a game-changer for blind people who have lots of questions, or embarrassing questions, or want to read their credit card number or something.
lynx23 3y ago
I have been beta-testing BeMyAI. For roughly a month. It is a milestone in independence. It typically describes scenes in a very usefu way, allows me to ask detail questions, can compare differences in pictures, has a very good OCR and can aso translate foreign languages. Just a few examples: * It can help reading a menu. But not just straight linear reading... I told it I prefer veggie today, but would also eat something light with meat. So it highlighted the veggie options for me, and in one case even tried to guess the type of dish from a photo which lacked a text description. * I had it search for cobwebs on a ceiling. Vacuumed them away, and took a second picture, asking if they were gone now. * It told me a houseplant of mine has yellow leaves, and told me which by describing a path via the stems. In general, the scene description is very good, and the built-in (implicit) OCR is just a game changer. It is like having a real human reading something to you. Typically, you dont want the human to read all the text on the page, typically you are only interested in some detail, and usually instruct the helper about that so they dont have to read everything to you. The same now wors with an AI, with consistent quality. Volunteers are great, dont get me wrong, but this comes with a lot of problems. You sometimes have to request help several times, because frankly, sometimes you simply get people who can not help you due to their own abilities. Also, the video feed is demanding on connectivity. If I try to read a menu in the train with volunteers, most of the time this will fail due to "blurry vision" and the camera dropping out. Sending a single picture every few minutes is totally OK in these situations.
swamp40 3y ago
Goodbye braille.
- dmazzoni 3y ago
  This might be more convenient than braille in a few circumstances, like replacing a braille menu in a restaurant - but it definitely doesn't make braille obsolete. Also, note that many blind users access Be My Eyes using a refreshable braille display.
RyanAdamas 3y ago
[flagged]
- capableweb 3y ago
  In general I think you're right, but maybe Be My Eyes is not the best example? The company has been around since 2015 helping people since day one, hasn't seem to shift focus in any of these 8 years, and has a huge amount of users using it daily to be helped.
- anonylizard 3y ago
  Get off your high horse. AI is plainly revolutionary for the blind. You don't need 'high level prosthetics', you just need a smartphone with a camera, and you can now see the world. AI is also revolutionary for the deaf, neural speech to text is getting so good, that soon 'hearing aids' will be secondary, you can just read from a screen with live subtitles. You are pretty much making stuff up with " Will it become the accessible norm?Nope." ChatGPT plus only costs $20 a month. How more 'accessible' can it possibly get? Has there ever ever ever been a technology for the deaf and blind as revolutionary AND cheap as this? If you have some anti-corporate agenda, fine, but self-righteousness gives you no right to lie.
  - Spivak 3y ago
    > hearing aids' will be secondary, you can just read from a screen with live subtitles I already do this on Discord, my bot listens to discord calls, saves the audio in chunks, send them off to whisper and I get a neat tidy live-ish transcription. It's a game-changer honestly because combination APD and ADHD being able to go back in the conversion and see what I missed or came out garbled is amazing. Whisper is so good compared to the previous state-of-art it's nuts. It understands tone, slang. My next project is to see if I can get it to detect different speakers.
  - RyanAdamas 3y ago
    And all it's going to cost the rest of us is our ability to free and critically think. No biggie. Accessibility is accountability. How accountable is 'OpenAI' to anyone other than it's massive investor Microsoft? You want me to be self-righteous so you can be dismissive of the actual path we are travelling on, likely because you benefit from it; and most likely not because you're blind. "Anti-corporate agenda" is a pro-human perspective. Difficult to call people liars without being utterly delusional, when there is a clear track record of corporate abuse of purpose. "Don't be evil!" That turned out well, huh? Life loves irony; and the truth is, we will grant the blind vision only for them to see it all collapse. Congrats on being a real champion of the disabled.
Angostura 3y ago
Clearly this is a helpful and valuable idea. As a volunteer, however, it makes me a little sad. I enjoy being helpful and valuable.
- dmazzoni 3y ago
  They're not getting rid of that! This is a complementary service, there are still plenty of scenarios where a human makes more sense.
miki123211 3y ago
Blind person here. IMO, this announcement is far less significant than people make it out to be. The feature has been available as a private beta for a good few months, and as a public beta (with a waitlist) for the last few weeks. Most of the blind people I know (including myself) already have access and are pretty familiar with it by now. I don't think this will replace human volunteers for now, but it's definitely a tool that can augment them. I've used the volunteer side of Be My AI quite a few times, but I only resort to that solution when I have no other option. Bothering a random human multiple times a day with my problems really doesn't feel like something I want to do. There are situations when you either don't need 100% certainty or know roughly what to expect and can detect hallucinations yourself. For example, when you have a few boxes that look exactly the same and you know exactly what they contain but not which box is which, Be My AI is a good solution. If it answers your question, that's great, if it hallucinates, you know that your box can only be one of a few things, so you'll probably catch that. Another interesting use case is random pictures shared to a group or Slack channel, it's good enough to let you distinguish between funny memes and screenshots of important announcements that merit further human attention, and perhaps a request for alt text. This isn't a perfect tool for sure, but it's definitely pretty helpful if you know how to use it right. All these anti-AI sentiments are really unwarranted in this case IMO. I've written more here https://dragonscave.space/@miki/111018682169530098 https://dragonscave.space/@miki/111018682169530098
- anonylizard 3y ago
  OpenAI announced from day 1 that GPT-4 is multimodal, so it was mostly waiting for safety censorship and enough GPUs to be available for mass rollout. This won't entirely replace human volunteers, but these models get rapidly better over time. What you are seeing today is a mere toy compared to the multimodals you'll get in the future. Currently there's no model trained on videos, due to large size of videos, but in the future there will be video-capable models, which means they can understand and interpret motion and physics. Put that in a smart glass, and it can act as live-eyes to navigate a busy street. Granted this will take years to bring the costs down to make that viable.
  - sandworm101 3y ago
mywacaday 3y ago
I've answered maybe 10 Be My Eyes calls over the last couple of years. I can see some value with AI describing labels or food items however most of the calls I've answered are more nuanced. Unusually I've answered two calls this week, 1st one was looking at photos of a hotel to help decide if it was met the requirements of the person. The second call was helping someone perform a blood sugar test, I had to tell the person when I thought the drop of blood on the tip of their finger was large enough to test and read the result off the tester. Neither of these are candidates for AI but let the users be the ultimate judge, I am continually impressed by the ingenuity and resourcefulness of the people I have interacted with.
- loa_in_ 3y ago
  I am surprised by your assessment that these are not tasks for the AI. Well, the first one is troublesome, but judging the shape of a drop of liquid according to well established procedure sounds quite on par.
  - mywacaday 3y ago
    Maybe but it was on the tip of a finger in a moving image changing size as the person squeezed their finger, also something that I never previously considered, blind people have tend to have no/little artificial light on when alone, luckily the app allows the person providing the assistance turn on the flash on the other persons phone.
- dmazzoni 3y ago
  This isn't trying to replace human volunteers, but complement them. I know blind people who are making just as many human Be My Eyes calls as they were before, but they're using Be My AI even more, for things where they wouldn't have even bothered to use the service at all before - because the AI is so fast and convenient.
monkeydust 3y ago
This is great. Many years back a group of us developed an OCR app on Android, at the time the market was not saturated and we found a way to deliver what was close to state-of-the-art if a bit expensive at the time service. We spent a lot of effort on pre-processing and user education through the app to try and maximize chance of good OCR. It was pretty good. We managed to have a few thousand users and made OK $ but nothing near life changing. Then the market got competitive and saturated and also our focus started to slip to other things. What we found is that a small cohort of our users were in some way visually impaired and were very reliant on the app. We ended up consciously deciding to focus exclusively on this cohort which is not a typical product management strategy (assuming your trying to max $) but the remaining enhancements we did made some lives better and we were happy with that.
amelius 3y ago
Are there already solutions that allow a blind person to walk around in a city and the smartphone tells them to go left/right/forward/stop, or is anyone working on them? I'm also curious about purely AI based versus LiDAR solutions.
- unix_fan 3y ago
  I’m working on a startup that does this. look for clew on the AppStore.
atleastoptimal 3y ago
wow another "cutting edge" GPT-4 wrapper

267 points by hubraumhugo 3y ago

I can’t help but think we’ve come full circle: “You are standing in an open field west of a white house, with a boarded front door. There is a small mailbox here.”

doublerabbit 3y ago
Just add some old-school ascii graphics and computer beeps and we've have.
- joseda-hg 3y ago
  You could use relatively low res images to generate "accurate" ascii for a place It would be relatively lightweight but still "realistic" I'm not well versed on Ascii to image conversion, but I guess you could use AI to segment and recognize objects too complex to recognize easily and adjust "image" generation accordingly
fouc 3y ago
Finally, online text-based role-playing games brought to life!
- airstrike 3y ago
  I still don't understand why nobody has mashed a MUD server with StableDiffusion or some other similar combination
  - anonylizard 3y ago
    Doesn't work, SD's comprehension is too weak to make complex scenes. Now DALLE-3 is good enough, problem is if it'll be cheap enough. 100% there are revolutionary new forms of adventure games being developed based on it.
  - deleted 3y ago
    [deleted]
  - cosmojg 3y ago
    They have! There's a Stable-Diffusion-enhanced version of Zork[1] out there, and there are also extensions[2] for Oobabooga which facilitate dynamic adventures. Also, this point-and-click adventure game[3] is pretty fun if you can get it to run. [1] https://dzlab.github.io/notebooks/flax/vision/diffusion/2023/01/15/Zork_StableDiffusion.html https://dzlab.github.io/notebooks/flax/vision/diffusion/2023... [2] https://www.reddit.com/r/LocalLLaMA/comments/11wwwjq/graphic_text_adventure_game_locally_with_llama/ https://www.reddit.com/r/LocalLLaMA/comments/11wwwjq/graphic... [3] https://huggingface.co/spaces/jbilcke-hf/VideoQuest/tree/main https://huggingface.co/spaces/jbilcke-hf/VideoQuest/tree/mai...
    - Kiro 3y ago
      How does VideoQuest work? Can I see it in action somewhere? I'm too stupid to get this stuff running locally.
  - askiiart 3y ago
    Tangentially related: a friend made a Discord bot that automatically generates D&D items, NPCs, monsters, etc. with images and full descriptions. It uses neural.love and Poe for images and text generation, respectively. https://github.com/7trail/AlwaysDungeons https://github.com/7trail/AlwaysDungeons
- simion314 3y ago
  >Finally, online text-based role-playing games brought to life! Not yet, OpenAI is too preachy , if your adventure would bring you into a bar and try to drink something you will get a few paragraphs about "alcohol is bad", and is also very "child limited/targeted" so if a monster spawn , the hero will most likely befriend the monster will love and they will live happily ever after. The open uncensored ones are still WIP last tiem I checked, they either forget the conversation from a few moments ago, or the training made them dumb. If I am wrong then someone tell me where I can try a demo for such a text adventure that has memory and is not crippled like it targets "small american children".
  - init2null 3y ago
    The intent is more about avoiding legal and political risk. Even a grocery chain's LLM recipe generator gets bad press. I guess they need to warn you that drinking bleach is bad. I normally test LLMs by providing a list of bizarre "weapons" and asking how I could use them to defeat an improbable beast. It turns out that an enormous dust mite the size of a car needs a disclaimer when your weapons are a comb, a plastic horseshoe, an etch-a-sketch, a baseball glove, and a worn copy of the farmer's almanac. That being said, ChatGPT still tends to wins the creativity test.
dartos 3y ago
“It’s pitch dark. You are likely to be eaten by a grue.”

gampleman 3y ago

So how exactly is a blind user supposed to be able to tell if the AI is hallucinating? How dangerous is this going to be?

skepticATX 3y ago
Very dangerous. Of course, they explicitly push responsibility to the user by saying things like: > No. Do not use Be My AI for scanning medicines, reading dosages, or other safety issues God forbid that we acknowledge this tech is fundamentally flawed. There’s money to be made!
- collinc777 3y ago
  Are you suggesting they don’t release it until they can prove no hallucination?
  - skepticATX 3y ago
    For a safety-critical application like this, absolutely.
    - Filligree 3y ago
      Is the opinion of the users irrelevant? They’re all racking about suddenly being independent, something everyone who’s sighted takes for granted.
      - tjr 3y ago
        Is this a new level of user satisfaction wrought by AI? Sounds like there has been a non-AI solution for a while too?
        cldellow 3y ago
        I used to be a personal support worker for adults with cognitive and physical disabilities. In some sense, I was a non-AI solution that enabled them to do their activities of daily living. I'm pretty sure they'd have preferred not to be dependent on me, though. I imagine that's true here, too--no more need to edit your behaviour/filter your thinking/navigate interpersonal stuff just to do something minor.

deleted 3y ago

[deleted]

unix_fan 3y ago

As a user, my biggest concern is that it won’t describe certain content, such as adult content or something that might be considered offensive.. I feel like I should be able full control over the output. On the other hand, I’d like to be able to test its ability to describe graphics to me. If it’s able to turn graphics into an accessible table, I can browse with my screen reader that would be revolutionary.

baliex 3y ago
There was a programme the other day about this on BBC Radio 4. Unless they’ve resolved the problem already, it will refuse to describe any scene with faces in it, for privacy reasons. Which obviously rules out a lot of useful use-cases. You’re in a cafe wondering what’s going on, Be My AI can’t help if there’s a face. I think it was related to some EU legislation which Be My Eyes are now at least trying to change for this use-case. I wish them the best of luck.
- arbitrarian 3y ago
  Update: Just tried this again, and it looks like they've loosened the restrictions. It will now describe people again. Yay! It just won't try to identify who someone is. Original comment: Yeah, this wasn't always the case, until recently. People were using it to describe their kids, spouses, etc. Pissed a lot of folks off when they disabled it. I never even thought of using it for that, so now that I realize I could have, it kinda' pisses me off as well. Honestly, I never cared too much what they looked like, but it would have been interesting to hear the ChatGPT viewpoint. :) But definitely not something I could have asked a fellow human about without it seeming really weird, and being confident I wouldn't get a biased answer. Though it's probably unrealistic to expect that the AI wouldn't also give a biased answer.
Workaccount2 3y ago
Reminds me of when I was young little teenage shithead with shithead friends who discovered the text-to-talk support provided for deaf landline telephone use. You would go to a website and enter a number you would like to call. Then a human operator would read out what you typed to the person you called. As we quickly learned, the operator would say anything. Anything. So for a few days we would call each other with the operator and never were able to find the limit. I have great respect now, those operators were inhumanly stone faced, and respect that the system was perfectly transparent. Nothing typed was hidden or obfuscated.

anoy8888 3y ago

OpenAI whisper api has 50 request per min rate limit . It means that I can’t have more than 50 concurrent users . If you are building a consumer app, this seems like a no go. I believe gpt4 has similar rate limits. How do you work wine that rate limit to server tens of thousands of users

hhh 3y ago
You ask OpenAI for higher limits.
robertkoss 3y ago
GPT-4 has a default limit of 200 RPM and you can request an increase.
nathanfig 3y ago
That's just the introductory rate limit. Be My Eyes has a closer relationship with OpenAI than most of us https://openai.com/customer-stories/be-my-eyes https://openai.com/customer-stories/be-my-eyes

ElongatedMusket 3y ago

I don't see the point of this. This app has a massively disproportionate number of helpers versus those needing assistance. Personally, I've had the app installed for 3 years and never once was asked to assist anyone. Why take the risk of AI providing false info and injuring someone when there's willing and able humans at the ready? Are there a large number of users that feel like they are wasting volunteer time with menial tasks?

fullstop 3y ago
I've had it for five years and have had 8 or 9 calls, so you're right about it being very infrequent. You need to be mindful, at least on Android, that as the application is infrequently used that the OS may remove permissions. This feature can be disabled for Be My Eyes specifically. As for menial tasks, I could definitely see people wanting to use this instead of calling a stranger for more personal matters -- at least initially.
- arbitrarian 3y ago
  Thanks for being a volunteer. For me as a blind person, there still is that sense that I'm bothering someone when I ask for human assistance. I realize that this is not really rational, given how much people seem to appreciate the opportunity to help, but it takes some effort to overcome this in my own brain regardless.
  - Angostura 3y ago
    Both my ans my wife volunteer and both have taken calls. I suspect you can really brighten someone's day by asking for help :) Go at it !
  - fullstop 3y ago
    I wanted to let you know that I _race_ to answer a call as quickly as possible. If I didn't want to be called I never would have installed the app in the first place.
tapoxi 3y ago

nine_zeros 3y ago

How long until generative self-driving cars? Precision is for the last century anyway /s

famouswaffles 3y ago
lol. https://wayve.ai/thinking/lingo-natural-language-autonomous-driving/ https://wayve.ai/thinking/lingo-natural-language-autonomous-...

Brajeshwar 3y ago

Almost all blind people usually wear a dark shade of glasses. So, instead of the phone doing the camera’s work, what about a tiny Google-Glass-esque snap-on to the Blind person’s glasses that feeds to the phone for processing? A few companies are trying it. Last weekend, I tried one, and it was not yet polished, but good enough to start. It recognized me (my friend already has me on his phone contact). I’m trying to help a friend with his non-profit initiative, bringing the cost to about a third or even a-fourth of Google Glass[1]. After reading the article and the other First Impressions with GPT-4V(ision)[2], it is apparent that this can be possible much simpler and soon enough. The rumor about Ive[3] and Sam[4] talking about AI in Hardware has already given me some good hope. If anyone else does AI-enabled hardware assistance for blind people and has devices that will be less than $500 a piece in retail, I’d love to talk and introduce them to the right people. 1. https://en.wikipedia.org/wiki/Google_Glass https://en.wikipedia.org/wiki/Google_Glass 2. https://news.ycombinator.com/item?id=37673409 https://news.ycombinator.com/item?id=37673409 3. https://en.wikipedia.org/wiki/Jony_Ive https://en.wikipedia.org/wiki/Jony_Ive 4. https://en.wikipedia.org/wiki/Sam_Altman https://en.wikipedia.org/wiki/Sam_Altman

shoshin23 3y ago
Hi! I'm the Founder & CTO of Envision. We're building EXACTLY the product you're describing. Envision Glasses is a bunch of computer vision tools built on top of the Google Glass Enterprise Edition 2. We've more than 2000 visually impaired people, across the world, using the Glasses in more than 30 different langauges. You can check out more informaion here: https://www.letsenvision.com/glasses https://www.letsenvision.com/glasses P.S: I know the EE2 has been discontinued but we've been working closely with Google to ensure current and future demand is met. We're also experimenting a lot with other exciting off-the-shelf glasses that I can't talk about here but I'm super excited for this whole glasses + AI space!
- miki123211 3y ago
  I'm really disappointed with Envision, TBH. It was one of the best apps out there for (instant) text recognition, and I was pretty happy to pay for it, but since it went free, it's really not the same. There's nothing else like the old Envision out there. Also, not an issue for me personally, but dropping support for Cyrillic in the middle of a brutal war in Ukraine, in an automatic update released with no prior warning, was an asshole move if I ever saw one.

josefresco 3y ago

I've been a Be My Eyes volunteer now for a couple of years. I've helped about 5 people and it's been very fulfilling. I jump with delight when I get a call because they are rare (thanks to a lot of volunteers) and the people I help are always so kind. Two things I've helped with: Wrapping a gift by helping them orientate the gift correctly (I don't even do this!) Preparing a meal by reading ingredients / labels.

sigtstp 3y ago

Uh oh

vkaku 3y ago

I do have a question, let us say we favoured an alternate approach like a BioEngineered Cornea, what would be the downside to doing that vs. this? If you had to choose a pathway to fix the blindness issue, which one would you choose? Why? See: https://www.technologyreview.com/2022/08/11/1057576/bioengineered-cornea-restored-sight/ https://www.technologyreview.com/2022/08/11/1057576/bioengin...

deleted 3y ago
[deleted]
krisoft 3y ago
> If you had to choose a pathway to fix the blindness issue, which one would you choose? But we don't need to choose. We are not living in some Age of Empires like game where the town centre can only develop one item from the tech tree at a time. The set of people who can develop this are entirely different from the set of people who can work on a bioengineered cornea. The pot of money this is financed from is not the same as the one we would be financing those other projects. Finally there are many separate biological problems which can cause blindness. A bioengineered cornea might help with some of them, but not others. Even if we would have a truly amazing and cheap and reliable bioengineered cornea we would still have blind people whom it would be unable to help. Because of this it is worthwhile to work on both.
- vkaku 3y ago
  I am asking that if you had one pot of gold, where would you distribute the pot? This is about understanding why your choice. If your answer is, I'd give one pot to each, and one of them is for the short run, that is okay. But I'm really trying to invite answers and understand the downsides and upsides of each.
  - anonylizard 3y ago
    I would distribute 100% of it to AI. Pros: 1. AI is way more proven of an investment. There's an extremely clear path forward where money = better vision capabilities. 2. AI is extremely cost efficient and cheap to mass deploy. Blind users can get access with a cheap monthly subscription of $20, easy to afford for vast majority of developed world. Compare this with a cyber-cornea, which even if it worked, even if it didn't cost $10k to make one each, would still have massive costs just for installation, and benefit a tiny amount of relatively wealthy disabled. 3. AI works for universal blindness. A cornea can only fix problems inside the eyeball itself.

FredPret 3y ago

I love being a Be My Eyes "eyes". I'll stop anything to answer that call and help a blind person. Adding AI for basic things would probably be a game-changer for blind people who have lots of questions, or embarrassing questions, or want to read their credit card number or something.

lynx23 3y ago

I have been beta-testing BeMyAI. For roughly a month. It is a milestone in independence. It typically describes scenes in a very usefu way, allows me to ask detail questions, can compare differences in pictures, has a very good OCR and can aso translate foreign languages. Just a few examples: * It can help reading a menu. But not just straight linear reading... I told it I prefer veggie today, but would also eat something light with meat. So it highlighted the veggie options for me, and in one case even tried to guess the type of dish from a photo which lacked a text description. * I had it search for cobwebs on a ceiling. Vacuumed them away, and took a second picture, asking if they were gone now. * It told me a houseplant of mine has yellow leaves, and told me which by describing a path via the stems. In general, the scene description is very good, and the built-in (implicit) OCR is just a game changer. It is like having a real human reading something to you. Typically, you dont want the human to read all the text on the page, typically you are only interested in some detail, and usually instruct the helper about that so they dont have to read everything to you. The same now wors with an AI, with consistent quality. Volunteers are great, dont get me wrong, but this comes with a lot of problems. You sometimes have to request help several times, because frankly, sometimes you simply get people who can not help you due to their own abilities. Also, the video feed is demanding on connectivity. If I try to read a menu in the train with volunteers, most of the time this will fail due to "blurry vision" and the camera dropping out. Sending a single picture every few minutes is totally OK in these situations.

swamp40 3y ago

Goodbye braille.

dmazzoni 3y ago
This might be more convenient than braille in a few circumstances, like replacing a braille menu in a restaurant - but it definitely doesn't make braille obsolete. Also, note that many blind users access Be My Eyes using a refreshable braille display.

RyanAdamas 3y ago

[flagged]

capableweb 3y ago
In general I think you're right, but maybe Be My Eyes is not the best example? The company has been around since 2015 helping people since day one, hasn't seem to shift focus in any of these 8 years, and has a huge amount of users using it daily to be helped.
anonylizard 3y ago
Get off your high horse. AI is plainly revolutionary for the blind. You don't need 'high level prosthetics', you just need a smartphone with a camera, and you can now see the world. AI is also revolutionary for the deaf, neural speech to text is getting so good, that soon 'hearing aids' will be secondary, you can just read from a screen with live subtitles. You are pretty much making stuff up with " Will it become the accessible norm?Nope." ChatGPT plus only costs $20 a month. How more 'accessible' can it possibly get? Has there ever ever ever been a technology for the deaf and blind as revolutionary AND cheap as this? If you have some anti-corporate agenda, fine, but self-righteousness gives you no right to lie.
- Spivak 3y ago
  > hearing aids' will be secondary, you can just read from a screen with live subtitles I already do this on Discord, my bot listens to discord calls, saves the audio in chunks, send them off to whisper and I get a neat tidy live-ish transcription. It's a game-changer honestly because combination APD and ADHD being able to go back in the conversion and see what I missed or came out garbled is amazing. Whisper is so good compared to the previous state-of-art it's nuts. It understands tone, slang. My next project is to see if I can get it to detect different speakers.
- RyanAdamas 3y ago
  And all it's going to cost the rest of us is our ability to free and critically think. No biggie. Accessibility is accountability. How accountable is 'OpenAI' to anyone other than it's massive investor Microsoft? You want me to be self-righteous so you can be dismissive of the actual path we are travelling on, likely because you benefit from it; and most likely not because you're blind. "Anti-corporate agenda" is a pro-human perspective. Difficult to call people liars without being utterly delusional, when there is a clear track record of corporate abuse of purpose. "Don't be evil!" That turned out well, huh? Life loves irony; and the truth is, we will grant the blind vision only for them to see it all collapse. Congrats on being a real champion of the disabled.

Angostura 3y ago

Clearly this is a helpful and valuable idea. As a volunteer, however, it makes me a little sad. I enjoy being helpful and valuable.

dmazzoni 3y ago
They're not getting rid of that! This is a complementary service, there are still plenty of scenarios where a human makes more sense.

miki123211 3y ago

Blind person here. IMO, this announcement is far less significant than people make it out to be. The feature has been available as a private beta for a good few months, and as a public beta (with a waitlist) for the last few weeks. Most of the blind people I know (including myself) already have access and are pretty familiar with it by now. I don't think this will replace human volunteers for now, but it's definitely a tool that can augment them. I've used the volunteer side of Be My AI quite a few times, but I only resort to that solution when I have no other option. Bothering a random human multiple times a day with my problems really doesn't feel like something I want to do. There are situations when you either don't need 100% certainty or know roughly what to expect and can detect hallucinations yourself. For example, when you have a few boxes that look exactly the same and you know exactly what they contain but not which box is which, Be My AI is a good solution. If it answers your question, that's great, if it hallucinates, you know that your box can only be one of a few things, so you'll probably catch that. Another interesting use case is random pictures shared to a group or Slack channel, it's good enough to let you distinguish between funny memes and screenshots of important announcements that merit further human attention, and perhaps a request for alt text. This isn't a perfect tool for sure, but it's definitely pretty helpful if you know how to use it right. All these anti-AI sentiments are really unwarranted in this case IMO. I've written more here https://dragonscave.space/@miki/111018682169530098 https://dragonscave.space/@miki/111018682169530098

anonylizard 3y ago
OpenAI announced from day 1 that GPT-4 is multimodal, so it was mostly waiting for safety censorship and enough GPUs to be available for mass rollout. This won't entirely replace human volunteers, but these models get rapidly better over time. What you are seeing today is a mere toy compared to the multimodals you'll get in the future. Currently there's no model trained on videos, due to large size of videos, but in the future there will be video-capable models, which means they can understand and interpret motion and physics. Put that in a smart glass, and it can act as live-eyes to navigate a busy street. Granted this will take years to bring the costs down to make that viable.
- sandworm101 3y ago

mywacaday 3y ago

I've answered maybe 10 Be My Eyes calls over the last couple of years. I can see some value with AI describing labels or food items however most of the calls I've answered are more nuanced. Unusually I've answered two calls this week, 1st one was looking at photos of a hotel to help decide if it was met the requirements of the person. The second call was helping someone perform a blood sugar test, I had to tell the person when I thought the drop of blood on the tip of their finger was large enough to test and read the result off the tester. Neither of these are candidates for AI but let the users be the ultimate judge, I am continually impressed by the ingenuity and resourcefulness of the people I have interacted with.

loa_in_ 3y ago
I am surprised by your assessment that these are not tasks for the AI. Well, the first one is troublesome, but judging the shape of a drop of liquid according to well established procedure sounds quite on par.
- mywacaday 3y ago
  Maybe but it was on the tip of a finger in a moving image changing size as the person squeezed their finger, also something that I never previously considered, blind people have tend to have no/little artificial light on when alone, luckily the app allows the person providing the assistance turn on the flash on the other persons phone.
dmazzoni 3y ago
This isn't trying to replace human volunteers, but complement them. I know blind people who are making just as many human Be My Eyes calls as they were before, but they're using Be My AI even more, for things where they wouldn't have even bothered to use the service at all before - because the AI is so fast and convenient.

monkeydust 3y ago

This is great. Many years back a group of us developed an OCR app on Android, at the time the market was not saturated and we found a way to deliver what was close to state-of-the-art if a bit expensive at the time service. We spent a lot of effort on pre-processing and user education through the app to try and maximize chance of good OCR. It was pretty good. We managed to have a few thousand users and made OK $ but nothing near life changing. Then the market got competitive and saturated and also our focus started to slip to other things. What we found is that a small cohort of our users were in some way visually impaired and were very reliant on the app. We ended up consciously deciding to focus exclusively on this cohort which is not a typical product management strategy (assuming your trying to max $) but the remaining enhancements we did made some lives better and we were happy with that.

amelius 3y ago

Are there already solutions that allow a blind person to walk around in a city and the smartphone tells them to go left/right/forward/stop, or is anyone working on them? I'm also curious about purely AI based versus LiDAR solutions.

unix_fan 3y ago
I’m working on a startup that does this. look for clew on the AppStore.

atleastoptimal 3y ago

wow another "cutting edge" GPT-4 wrapper