3 ms·

You gave very different prompts to gemini and GPT 4("how much money" vs "how many coins", different coordinates systems etc).

zerojames 3y ago
Thank you! We took screenshots of our tests, which you can download here: https://media.roboflow.com/lmms-tests.zip https://media.roboflow.com/lmms-tests.zip (Gemini ones are not in there yet.)
- uxp8u61q 3y ago
  That's not really answering anything.
- YetAnotherNick 3y ago
  The prompt for the coordinates is still different in GPT 4. Coins one look to be the same but the question is little bit ambiguous as GPT might be answering the total value of coins and did 1+1+1+2. For me, the tax one is working in GPT 4.
  - mvdtnz 3y ago
    > Coins one look to be the same but the question is little bit ambiguous as GPT might be answering the total value of coin Well that wouldn't be a very intelligent "AI" would it? The prompt was completely unambiguous.
mewpmewp2 3y ago
Trying myself with GPT 4 Vision (directly using the API), I get response: "The image shows a total of four coins." For the exact same image with a prompt "how many coins do I have?" I keep getting the same thing every time. If I ask "what is the value shown on the image?" I get the following: "The image shows four coins, and from what I can discern, three of the coins have the number "1" on them, which likely indicates that they are one-unit coins in their respective currency. The fourth coin, which is a different color and appears to be bi-metallic, has the number "2" on it, suggesting it is a two-unit coin. The total value shown by the coins, therefore, appears to be 5 units of their currency. However, without knowing the specific currency, I cannot determine the exact monetary value in any other terms." So to me it's obvious gpt4-vision model can handle this question. Edit: I also get for regular ChatGPT "You have a total of four coins in the image." Now I tried to ask GPT4 Vision to find the Dog, and it said: "The dog is located approximately at 0.29, 0.36, 0.70, 0.90." - Just eyeing it, it may seem decently accurate, but not 100%. My difference is that I took a screenshot from your pictures rather than providing the original though, so that could also probably affect answers. Prompt for the dog was: "Find the dog. Return its location in the format x_min, y_min, x_max, y_max. Respond with 0-1 as percentages."

zerojames 3y ago

Thank you! We took screenshots of our tests, which you can download here: https://media.roboflow.com/lmms-tests.zip https://media.roboflow.com/lmms-tests.zip (Gemini ones are not in there yet.)

uxp8u61q 3y ago
That's not really answering anything.
YetAnotherNick 3y ago
The prompt for the coordinates is still different in GPT 4. Coins one look to be the same but the question is little bit ambiguous as GPT might be answering the total value of coins and did 1+1+1+2. For me, the tax one is working in GPT 4.
- mvdtnz 3y ago
  > Coins one look to be the same but the question is little bit ambiguous as GPT might be answering the total value of coin Well that wouldn't be a very intelligent "AI" would it? The prompt was completely unambiguous.

mewpmewp2 3y ago

Trying myself with GPT 4 Vision (directly using the API), I get response: "The image shows a total of four coins." For the exact same image with a prompt "how many coins do I have?" I keep getting the same thing every time. If I ask "what is the value shown on the image?" I get the following: "The image shows four coins, and from what I can discern, three of the coins have the number "1" on them, which likely indicates that they are one-unit coins in their respective currency. The fourth coin, which is a different color and appears to be bi-metallic, has the number "2" on it, suggesting it is a two-unit coin. The total value shown by the coins, therefore, appears to be 5 units of their currency. However, without knowing the specific currency, I cannot determine the exact monetary value in any other terms." So to me it's obvious gpt4-vision model can handle this question. Edit: I also get for regular ChatGPT "You have a total of four coins in the image." Now I tried to ask GPT4 Vision to find the Dog, and it said: "The dog is located approximately at 0.29, 0.36, 0.70, 0.90." - Just eyeing it, it may seem decently accurate, but not 100%. My difference is that I took a screenshot from your pictures rather than providing the original though, so that could also probably affect answers. Prompt for the dog was: "Find the dog. Return its location in the format x_min, y_min, x_max, y_max. Respond with 0-1 as percentages."