3 ms·

They're already going multi-modal? Holy crap, if google can't deliver in the accessibility space for this (image descriptions better than "the logo for the comp

They're already going multi-modal? Holy crap, if google can't deliver in the accessibility space for this (image descriptions better than "the logo for the company"), then I'll definitely go back to Apple. I mean I do hope Apple cleans out bugs and makes VoiceOver feel like it won't fall over if I breathed hard, but their image descriptions, even without an LLM, are already clean and clear. More like "A green logo on a black background", where Google is, like I said, more like "The logo for the company." I guess it's kinda what we get when AI is crowdsourced rather than given good, high quality data to work with.

zitterbewegung 3y ago
Honestly if they are coming out with a paper now Apple has probably been working on it for a year or two at minimum . Next year releases of macOS / iOS are rumored to have LLMs as a feature .
- beoberha 3y ago
  I don’t mean to discount this work, but this particular model is the product of a few months of work tops. It’s effectively LLava with different training data, targeted at a specific use case. While I’m sure there is a significant effort at multimodal LLMs within Apple, this is just a tiny corner of it.
- refulgentis 3y ago
  > Honestly if they are coming out with a paper now Apple has probably been working on it for a year or two at minimum Why do you say that?
  - zitterbewegung 3y ago
    Academic papers can take that long …
- ex3ndr 3y ago
  They literally mention that they built on top of llava that was released half year ago.
  - smolder 3y ago
    What's the "literally" here? Try: They mention that they built on top of llava, which was released half a year ago.
    - superb_dev 3y ago
      In this context “literally” is being used to draw attention to the perceived obviousness of the information. Essentially saying “how could you not see this?”
      - illiac786 3y ago
        In this context, "what's the literally here?" means that the author think the post would be better without it: it's too agressive, sarcastic, demeaning, etc. for example. Hence it's a false question. It's actually funny because you answered a false question (potentially on purpose). Like: "What the hell are you doing?!?" "Well Im stealing your car, obviously."
        Kerb_ 3y ago
        In this context, the verbose explanation to a rhetorical question seems to be the point of the comment, to reemphasize said aggression or sarcasm. It's actually funny because we both are doing the same thing, also potentially on purpose.
sagz 3y ago
Google's Lookout app (accessibility for the blind and visually impaired) was updated ~6 months ago with a multimodal LLM already. It uses the Flamingo model family: https://deepmind.google/discover/blog/tackling-multiple-tasks-with-a-single-visual-language-model/ https://deepmind.google/discover/blog/tackling-multiple-task...

zitterbewegung 3y ago

Honestly if they are coming out with a paper now Apple has probably been working on it for a year or two at minimum . Next year releases of macOS / iOS are rumored to have LLMs as a feature .

beoberha 3y ago
I don’t mean to discount this work, but this particular model is the product of a few months of work tops. It’s effectively LLava with different training data, targeted at a specific use case. While I’m sure there is a significant effort at multimodal LLMs within Apple, this is just a tiny corner of it.
refulgentis 3y ago
> Honestly if they are coming out with a paper now Apple has probably been working on it for a year or two at minimum Why do you say that?
- zitterbewegung 3y ago
  Academic papers can take that long …
ex3ndr 3y ago
They literally mention that they built on top of llava that was released half year ago.
- smolder 3y ago
  What's the "literally" here? Try: They mention that they built on top of llava, which was released half a year ago.
  - superb_dev 3y ago
    In this context “literally” is being used to draw attention to the perceived obviousness of the information. Essentially saying “how could you not see this?”
    - illiac786 3y ago
      In this context, "what's the literally here?" means that the author think the post would be better without it: it's too agressive, sarcastic, demeaning, etc. for example. Hence it's a false question. It's actually funny because you answered a false question (potentially on purpose). Like: "What the hell are you doing?!?" "Well Im stealing your car, obviously."
      - Kerb_ 3y ago
        In this context, the verbose explanation to a rhetorical question seems to be the point of the comment, to reemphasize said aggression or sarcasm. It's actually funny because we both are doing the same thing, also potentially on purpose.

sagz 3y ago

Google's Lookout app (accessibility for the blind and visually impaired) was updated ~6 months ago with a multimodal LLM already. It uses the Flamingo model family: https://deepmind.google/discover/blog/tackling-multiple-tasks-with-a-single-visual-language-model/ https://deepmind.google/discover/blog/tackling-multiple-task...