3 ms·

I've been trying Gemini Pro using their $20-ish Goole One subscription for a couple of months, and I also find it consistently does fewer web searches to verify

I've been trying Gemini Pro using their $20-ish Goole One subscription for a couple of months, and I also find it consistently does fewer web searches to verify information than say ChatGPT 5.4 Pro which I have through work. I was planning on comparing them on coding but I didn't get the Gemini VSCode add-in to work so yeah, no dice. The Android and web app is also riddled with bugs, including ones that makes you lose your chat history from the threads if you switch between them, not cool. I'll be cancelling my Google One subscription this month.

WarmWash 6mo ago
I don't sweat sources and almost never check them. I usually prefer to manually check information after it's provided, to prevent the model from borking it's context trying to find sources that justify it's already computed output. Almost all the knowledge is already baked into the latent space of the model, so citing sources generally is a backwards process. I see it like going to the doctor and asking them to cite sources for everything they tell me. It would be ridiculous and totally make a mess of the visit. I much prefer just taking what the doctor said on the whole, and then verifying it myself afterwards. Obviously there is a lot of nuance here, areas with sparse information and certainly things that exist post knowledge cut-off. But if I am researching cell structure, I'm not going to muck up my context making it dig for sources for things that are certainly already optimal in the latent space.
- magicalhippo 6mo ago
  Well, I prefer it actually check datasheets so it doesn't go on a wild rabbit hunt to nowhere, since the capabilities it hallucinated for the chip in question doesn't exist.
  - RealityVoid 6mo ago
    In my experience, they all do this with dathasheets. Even if they read the actual datasheet, they misunderstand them gravely. I can't relie on them to do unusual setups or chaining stuff properly. It's true I did these attempts a couple of months ago, maybe they're better now.
  - deleted 6mo ago
    [deleted]
- svnt 6mo ago
  Good luck all-in. But seriously what are you doing that this works? I guess if you are writing pop culture articles this might work. For anything where the output has consequences I can’t imagine finding success like this.
  - WarmWash 6mo ago
    The latent space knowledge that the models have is stronger than the inference agent going out and trying to find information to integrate into context. If you ask why the sky is blue, the model already has the answer. It's corrosive to your conversation to pull a bunch of unknown sources into context so the model can appease your "feels right" request. If you don't trust the answer, your brain is still way way better at quickly scanning sources to verify the answer. But the fact of the matter is that these models went from stumbling over "9 + 7 =" three years ago to solving erdos problems today. And benchmarks (that are so saturated we don't even both with them anymore) reveal that the models basically all have total encyclopedic knowledge of every major career field. Which also makes sense because the labs have been purposely drilling hard on building pristine datasets of all this knowledge. I would challenge you to find one firmly established general academic question that a SOTA model gets wrong. Good luck.
    - svnt 6mo ago
      I use it claude and gemini all the time and they get more advanced theory, motivation, and history wrong all the time. If you aren’t seeing the errors it is because you are in some really mainstream conversations or because you don’t know what they are saying that is wrong. This is trivial to demonstrate to yourself for any nontrivial project. A single academic question is easy to get the right answer for. That is not the dominant AI use case for most product people or engineers.
hypercube33 6mo ago
You're supposed to download the antigravity VSCode fork and use that and it's rough at best I think. Hey free opus tokens though.

WarmWash 6mo ago

I don't sweat sources and almost never check them. I usually prefer to manually check information after it's provided, to prevent the model from borking it's context trying to find sources that justify it's already computed output. Almost all the knowledge is already baked into the latent space of the model, so citing sources generally is a backwards process. I see it like going to the doctor and asking them to cite sources for everything they tell me. It would be ridiculous and totally make a mess of the visit. I much prefer just taking what the doctor said on the whole, and then verifying it myself afterwards. Obviously there is a lot of nuance here, areas with sparse information and certainly things that exist post knowledge cut-off. But if I am researching cell structure, I'm not going to muck up my context making it dig for sources for things that are certainly already optimal in the latent space.

magicalhippo 6mo ago
Well, I prefer it actually check datasheets so it doesn't go on a wild rabbit hunt to nowhere, since the capabilities it hallucinated for the chip in question doesn't exist.
- RealityVoid 6mo ago
  In my experience, they all do this with dathasheets. Even if they read the actual datasheet, they misunderstand them gravely. I can't relie on them to do unusual setups or chaining stuff properly. It's true I did these attempts a couple of months ago, maybe they're better now.
- deleted 6mo ago
  [deleted]
svnt 6mo ago
Good luck all-in. But seriously what are you doing that this works? I guess if you are writing pop culture articles this might work. For anything where the output has consequences I can’t imagine finding success like this.
- WarmWash 6mo ago
  The latent space knowledge that the models have is stronger than the inference agent going out and trying to find information to integrate into context. If you ask why the sky is blue, the model already has the answer. It's corrosive to your conversation to pull a bunch of unknown sources into context so the model can appease your "feels right" request. If you don't trust the answer, your brain is still way way better at quickly scanning sources to verify the answer. But the fact of the matter is that these models went from stumbling over "9 + 7 =" three years ago to solving erdos problems today. And benchmarks (that are so saturated we don't even both with them anymore) reveal that the models basically all have total encyclopedic knowledge of every major career field. Which also makes sense because the labs have been purposely drilling hard on building pristine datasets of all this knowledge. I would challenge you to find one firmly established general academic question that a SOTA model gets wrong. Good luck.
  - svnt 6mo ago
    I use it claude and gemini all the time and they get more advanced theory, motivation, and history wrong all the time. If you aren’t seeing the errors it is because you are in some really mainstream conversations or because you don’t know what they are saying that is wrong. This is trivial to demonstrate to yourself for any nontrivial project. A single academic question is easy to get the right answer for. That is not the dominant AI use case for most product people or engineers.

hypercube33 6mo ago

You're supposed to download the antigravity VSCode fork and use that and it's rough at best I think. Hey free opus tokens though.