4 ms·

In case you haven't come across it already I recommend the app https://www.bemyeyes.com/ https://www.bemyeyes.com/. It allows the user to call a volunteer for a

In case you haven't come across it already I recommend the app https://www.bemyeyes.com/ https://www.bemyeyes.com/. It allows the user to call a volunteer for assistance, I've volunteered on this for several years and have helped with everything from booking a holiday online, doing a needle prick for a diabetes to help wrap Christmas presents. The app looks to have gotten ChatGPT like AI help as well recently.

wizzwizz4 3y ago
I think it is actually ChatGPT – and it's one of the few genuinely beneficial uses of the technology. Sure, you have to watch out for it making up an "explanation" of what's in the image, but if you treat it like a whimsical, self-absorbed prankster, it's so much better than not having it. I'm looking forward to a system that only describes what it sees (and what can be drawn from it), rather than doing stuff like trying to explain non-visual jokes (that, inevitably, it doesn't get). As I understand, that would require training new models from scratch, on training data that might not yet exist – but unlike the majority of the LLM GenAI hypey nonsense, this is actually a real, useful tool.
- ska 3y ago
  Fwiw, I've been on a bunch of these calls and often the first problem is the phone isn't pointed quite right at the thing they are asking about. This would be difficult for a chatbot I think.
- TheEaterOfSouls 3y ago
  I'm blind and I use a 13B LLaVA model locally. I haven't checked with a sighted person about how accurate the image descriptions are, but it seems to generally do okay (described some recent vacation photos pretty well, except sometimes it would list objects that I'm pretty sure aren't actually there in the images). Haven't tried GPT via Be My Eyes because I prefer using my laptop over my phone, but I imagine it'd be a lot better. For now I make regular use of the local model with a shell alias when I want something described, even solved a captcha that I couldn't OCR with it the other day. So yeah, this is one application of ML I'm really excited about, the other being Whisper (speech transcription), because I have profound hearing loss and can use it to transcribe things I can't hear in the audio.
ska 3y ago
It's been around for ages, I think the ChatGPT stuff is pretty recent. One really smart thing they do is localize the volunteers as much as possible, so often if the question is something like "Am I at the corner of X & Y" there is a good change you (the volunteer) know the area, etc. Things I have learned on the "volunteer" side: keeping track of canned food is often a problem, people worry about their outfits a fair bit, and parks are sometime difficult because there are lots of alternate paths/routes.

wizzwizz4 3y ago

I think it is actually ChatGPT – and it's one of the few genuinely beneficial uses of the technology. Sure, you have to watch out for it making up an "explanation" of what's in the image, but if you treat it like a whimsical, self-absorbed prankster, it's so much better than not having it. I'm looking forward to a system that only describes what it sees (and what can be drawn from it), rather than doing stuff like trying to explain non-visual jokes (that, inevitably, it doesn't get). As I understand, that would require training new models from scratch, on training data that might not yet exist – but unlike the majority of the LLM GenAI hypey nonsense, this is actually a real, useful tool.

ska 3y ago
Fwiw, I've been on a bunch of these calls and often the first problem is the phone isn't pointed quite right at the thing they are asking about. This would be difficult for a chatbot I think.
TheEaterOfSouls 3y ago
I'm blind and I use a 13B LLaVA model locally. I haven't checked with a sighted person about how accurate the image descriptions are, but it seems to generally do okay (described some recent vacation photos pretty well, except sometimes it would list objects that I'm pretty sure aren't actually there in the images). Haven't tried GPT via Be My Eyes because I prefer using my laptop over my phone, but I imagine it'd be a lot better. For now I make regular use of the local model with a shell alias when I want something described, even solved a captcha that I couldn't OCR with it the other day. So yeah, this is one application of ML I'm really excited about, the other being Whisper (speech transcription), because I have profound hearing loss and can use it to transcribe things I can't hear in the audio.

ska 3y ago

It's been around for ages, I think the ChatGPT stuff is pretty recent. One really smart thing they do is localize the volunteers as much as possible, so often if the question is something like "Am I at the corner of X & Y" there is a good change you (the volunteer) know the area, etc. Things I have learned on the "volunteer" side: keeping track of canned food is often a problem, people worry about their outfits a fair bit, and parks are sometime difficult because there are lots of alternate paths/routes.