3 ms·

LOL, it did work :-) Regarding LLMs, they are pushed too hard and too abusively by business people. Employees are being laid off and replaced with chatbots tha

by oblio 4mo ago

LOL, it did work :-) Regarding LLMs, they are pushed too hard and too abusively by business people. Employees are being laid off and replaced with chatbots that don't do the job. Frustrating if support for McDonald's, risky if health insurance support. Also the financials don't make sense. AI companies are money pits. Money is ultimately production. We make X amount of stuff yearly, globally. We can't afford to through away 5% of X yearly on technologies that will probably have a proper return in 5 or 10 years. When we mis-allocate resources on scales like these, people die. Look at Communist centralized planning. For $3-4 trillion we could have solved a LOT of actual global problems. LLMs are fine but they should have matured in the software dev domain for 2-3 more years and then non tech products would have followed.

TeMPOraL 4mo ago
FYI: I just had three SOTA LLMs + NotebookLM all fail at the simple task of explaining to me where to put a powder detergent in my particular newly bought washing machine, despite having photos of the machine and ability to find the manual (in case of NotebookLM, it literally had the manual as its only source). After first failure (Gemini 3.5 Flash + NotebookLM), I run the other two (Opus 4.8 on Extra; GPT 5.5 on High) in parallel, and looking at their thought streams, I gave up and dug up the manual and read half of it, before the LLMs finished coming up with - wait for it - wrong answers. Super frustrating. Doubly so, given that I use them for comparable tasks pretty often and they usually sail through them flawlessly. But this experience happens every now and then. It's only fair to report it, if only so you don't think I'm just AI boosting all the time.
- sebastiennight 4mo ago
  I'm curious now. Was the correct answer not "use the compartment marked with two parallel vertical lines"?
  - TeMPOraL 4mo ago
    My machine has compartments arranged like this: [ 2 | 3 ] ---------- [ 1 ] 1 is for powder detergent, 2 is for liquid detergent, 3 is for softeners and such. All three LLMs (Gemini twice, since NotebookLM) insisted I should put the powder detergent into leftmost compartment (2 on the ASCII diagram above). They referred to it by different numbers, but all gave some convincing justification why to put the detergent there. That's despite me posting photos of the compartment drawer, with symbols clearly visible. That's despite demanding they find the manual and cross-ref. I even asked two (Gemini and Claude) to label the actual compartment on the photos I took[0], and both produced some nonsense, with labels in all the wrong places. And they all insisted they're right and issued plenty of warnings about making sure I get this right or else bad things will happen. BTW. I ended up posting a screenshot of the diagram in the manual to Claude with a passive aggressive comment. Looking at its "thinking summary" and tool calls now, I think at least Claude didn't process the image correctly and only saw: [ 2 | 3 ] ------------ as those parts are blue, while the bottom is just in the same color as the entire body/frame of the machine. Maybe the contrast was too low for the models. But it was okay for humans, so it's not excusing much, especially that they all claimed to have found the manual, which had a high-contrast diagram. (Current experience tells me they probably didn't really check the diagram. I noticed recently that all major models seem to have gotten lazy when it comes to reading sources, and are also more than happy to lie about it.) -- [0] - A method I often use with Claude when I'm not sure if it's dealing with spatial tasks correctly - I have it produce intermediate artifacts that involve modifying "ground truth" inputs - e.g. placing two map pictures on top to verify it solved the coordinate transform, and/or (like here) drawing labels and boxes on top of original photos. I found such requests to be helpful enough I set it as general rule for Claude now. [1] - Which normally they'd spot, but for some reasons, they didn't.
    - sebastiennight 4mo ago
      Thanks for the walkthrough! > I have it produce intermediate artifacts that involve modifying "ground truth" inputs - e.g. placing two map pictures on top to verify it solved the coordinate transform, and/or (like here) drawing labels and boxes on top of original photos. Like you I use intermediate artifacts all the time, but have never tried with visual elements. But how do you get Claude to modify images? Do you get it to output things to a canvas? html? use an external library?

by oblio 4mo ago

TeMPOraL 4mo ago

FYI: I just had three SOTA LLMs + NotebookLM all fail at the simple task of explaining to me where to put a powder detergent in my particular newly bought washing machine, despite having photos of the machine and ability to find the manual (in case of NotebookLM, it literally had the manual as its only source). After first failure (Gemini 3.5 Flash + NotebookLM), I run the other two (Opus 4.8 on Extra; GPT 5.5 on High) in parallel, and looking at their thought streams, I gave up and dug up the manual and read half of it, before the LLMs finished coming up with - wait for it - wrong answers. Super frustrating. Doubly so, given that I use them for comparable tasks pretty often and they usually sail through them flawlessly. But this experience happens every now and then. It's only fair to report it, if only so you don't think I'm just AI boosting all the time.

sebastiennight 4mo ago
I'm curious now. Was the correct answer not "use the compartment marked with two parallel vertical lines"?
- TeMPOraL 4mo ago
  My machine has compartments arranged like this: [ 2 | 3 ] ---------- [ 1 ] 1 is for powder detergent, 2 is for liquid detergent, 3 is for softeners and such. All three LLMs (Gemini twice, since NotebookLM) insisted I should put the powder detergent into leftmost compartment (2 on the ASCII diagram above). They referred to it by different numbers, but all gave some convincing justification why to put the detergent there. That's despite me posting photos of the compartment drawer, with symbols clearly visible. That's despite demanding they find the manual and cross-ref. I even asked two (Gemini and Claude) to label the actual compartment on the photos I took[0], and both produced some nonsense, with labels in all the wrong places. And they all insisted they're right and issued plenty of warnings about making sure I get this right or else bad things will happen. BTW. I ended up posting a screenshot of the diagram in the manual to Claude with a passive aggressive comment. Looking at its "thinking summary" and tool calls now, I think at least Claude didn't process the image correctly and only saw: [ 2 | 3 ] ------------ as those parts are blue, while the bottom is just in the same color as the entire body/frame of the machine. Maybe the contrast was too low for the models. But it was okay for humans, so it's not excusing much, especially that they all claimed to have found the manual, which had a high-contrast diagram. (Current experience tells me they probably didn't really check the diagram. I noticed recently that all major models seem to have gotten lazy when it comes to reading sources, and are also more than happy to lie about it.) -- [0] - A method I often use with Claude when I'm not sure if it's dealing with spatial tasks correctly - I have it produce intermediate artifacts that involve modifying "ground truth" inputs - e.g. placing two map pictures on top to verify it solved the coordinate transform, and/or (like here) drawing labels and boxes on top of original photos. I found such requests to be helpful enough I set it as general rule for Claude now. [1] - Which normally they'd spot, but for some reasons, they didn't.
  - sebastiennight 4mo ago
    Thanks for the walkthrough! > I have it produce intermediate artifacts that involve modifying "ground truth" inputs - e.g. placing two map pictures on top to verify it solved the coordinate transform, and/or (like here) drawing labels and boxes on top of original photos. Like you I use intermediate artifacts all the time, but have never tried with visual elements. But how do you get Claude to modify images? Do you get it to output things to a canvas? html? use an external library?