3 ms·

I think it is actually ChatGPT – and it's one of the few genuinely beneficial uses of the technology. Sure, you have to watch out for it making up an "explanati

I think it is actually ChatGPT – and it's one of the few genuinely beneficial uses of the technology. Sure, you have to watch out for it making up an "explanation" of what's in the image, but if you treat it like a whimsical, self-absorbed prankster, it's so much better than not having it. I'm looking forward to a system that only describes what it sees (and what can be drawn from it), rather than doing stuff like trying to explain non-visual jokes (that, inevitably, it doesn't get). As I understand, that would require training new models from scratch, on training data that might not yet exist – but unlike the majority of the LLM GenAI hypey nonsense, this is actually a real, useful tool.

ska 3y ago
Fwiw, I've been on a bunch of these calls and often the first problem is the phone isn't pointed quite right at the thing they are asking about. This would be difficult for a chatbot I think.
TheEaterOfSouls 3y ago
I'm blind and I use a 13B LLaVA model locally. I haven't checked with a sighted person about how accurate the image descriptions are, but it seems to generally do okay (described some recent vacation photos pretty well, except sometimes it would list objects that I'm pretty sure aren't actually there in the images). Haven't tried GPT via Be My Eyes because I prefer using my laptop over my phone, but I imagine it'd be a lot better. For now I make regular use of the local model with a shell alias when I want something described, even solved a captcha that I couldn't OCR with it the other day. So yeah, this is one application of ML I'm really excited about, the other being Whisper (speech transcription), because I have profound hearing loss and can use it to transcribe things I can't hear in the audio.