3 ms·

I got so hopefuly by your comment I showed it my current bug that I'm working on, I even prepared everything first with my github issue, the relevant code, the

I got so hopefuly by your comment I showed it my current bug that I'm working on, I even prepared everything first with my github issue, the relevant code, the terminal with the failing tests, and I pasted it the full contents of the file and explained carefully what I wanted to achieve. As I was doing this, it repeated back to me everything I said, saying things like "if I understand correctly you're showing me a file called foo dot see see pee" and "I see you have a github issue open called extraneous spaces in frobnicator issue number sixty six" and "I see you have shared some extensive code" and after some more of this "validation"-speak it started repeating the full contents of the file like "backquote backquote backquote forward slash star .. import ess tee dee colon colon .." and so on. Not quite up to excited junior-level programmer standards yet. But maybe good for other things who knows.

nolist_policy 2y ago
You just rediscovered that LLMs become much stupider when using images as input. I think that has been already shown for gpt-4 as well.
- dtquad 2y ago
  Even when using the web search tool GPT-4 becomes stupider. Do they use a dumber model for tool/vision?
  - dwaltrip 2y ago
    I’m guessing that it’s just a much harder problem. Images often contain more information but it is far less structured and refined than language. The transformation process that occurs when people speak and write is incredibly rich and complex. Compared to images which are essentially just the outputs of cameras or screen captures — there isn’t an “intelligent” transformation process occurring.
    - justsid 2y ago
      I think images also have much higher information density than words, or at least they can. There is a reason a picture is worth 1000 words.
- tbruckner 2y ago
  This is news to me. Any good examples of this outside of the above?
  - nolist_policy 2y ago
    Vision language models are blind (192 comments) https://news.ycombinator.com/item?id=40926734 https://news.ycombinator.com/item?id=40926734
    - fnordpiglet 2y ago
      On the other hand if I take pictures of circuits, boards, electronic components, etc GPT4o is pretty reliably able to explain the me the pinouts, the board layouts, reference material in the data sheets, and provide pretty reasonable advice about how to use it (I.e., where to put resistors and why, what pins to use for the component on an esp32, etc). As a beginner in electronics this is fabulously helpful. Its ability to pass vision tests seems like a pretty dumb utility metric when most people judge utility by how useful things are.
jug 2y ago
Not sure this is an AI limitation. I think you'd be better off here with the Gemini Code Assist plugin in VS Code rather than that. Sounds like the AI is provided with unstructured information compared to an actual code base.
zamadatix 2y ago
> foo dot see see pee Well there's your problem!
- nuancebydefault 2y ago
  The ccp extension is just another c++ flavour /i
- lovasoa 2y ago
  ccp is the Chinese version of c++. Or maybe they meant ссср, the Soviet version.

nolist_policy 2y ago

You just rediscovered that LLMs become much stupider when using images as input. I think that has been already shown for gpt-4 as well.

dtquad 2y ago
Even when using the web search tool GPT-4 becomes stupider. Do they use a dumber model for tool/vision?
- dwaltrip 2y ago
  I’m guessing that it’s just a much harder problem. Images often contain more information but it is far less structured and refined than language. The transformation process that occurs when people speak and write is incredibly rich and complex. Compared to images which are essentially just the outputs of cameras or screen captures — there isn’t an “intelligent” transformation process occurring.
  - justsid 2y ago
    I think images also have much higher information density than words, or at least they can. There is a reason a picture is worth 1000 words.
tbruckner 2y ago
This is news to me. Any good examples of this outside of the above?
- nolist_policy 2y ago
  Vision language models are blind (192 comments) https://news.ycombinator.com/item?id=40926734 https://news.ycombinator.com/item?id=40926734
  - fnordpiglet 2y ago
    On the other hand if I take pictures of circuits, boards, electronic components, etc GPT4o is pretty reliably able to explain the me the pinouts, the board layouts, reference material in the data sheets, and provide pretty reasonable advice about how to use it (I.e., where to put resistors and why, what pins to use for the component on an esp32, etc). As a beginner in electronics this is fabulously helpful. Its ability to pass vision tests seems like a pretty dumb utility metric when most people judge utility by how useful things are.

jug 2y ago

Not sure this is an AI limitation. I think you'd be better off here with the Gemini Code Assist plugin in VS Code rather than that. Sounds like the AI is provided with unstructured information compared to an actual code base.

zamadatix 2y ago

> foo dot see see pee Well there's your problem!

nuancebydefault 2y ago
The ccp extension is just another c++ flavour /i
lovasoa 2y ago
ccp is the Chinese version of c++. Or maybe they meant ссср, the Soviet version.