4 ms·

I can confidently say that anecdotally you’re completely wrong, but I’ll also allow a very different definition of ‘simple’ and/or attempting to use an unpopula

by baq 11mo ago

I can confidently say that anecdotally you’re completely wrong, but I’ll also allow a very different definition of ‘simple’ and/or attempting to use an unpopular environment as a valid anecdotal counterpoint.

jacquesm 11mo ago
I don't reach for AI until I'm solidly stuck and then use it mostly for inspiration, it has yet to happen that it directly pointed at a solution and I'm pretty good at writing prompts. When I throw a bunch of elementary stuff at it then it is super good at identifying issues and solving them (but I could have done that myself, it is just nice to try to delineate where you can and where you can't trust the thing, but that too is fluctuating, sometimes even within a single session). Here is a nice example for a slightly more complex test: Prompt: "I have a green LED, a 12V powersupply, a single NPN transistor, a 100 ohm resistor, a 2.7K resistor and an electrolytic capacitor of 220 micro farads. My teacher says it is possible to make an LED flasher with these components but I'm hopelessly stuck, can you please give me an ascii art solution so I don't flunk this exam?" Answer: ASCII schematic (fully functional): +12V | | .-. | | | | 2.7kΩ (R1 — charges the capacitor) | | '-' | +---------+----------------------+ | | | ( ) | | ( ) C1 | | ( ) 220uF | | | | | | | | | |/ C | +--------| NPN transistor | | |\ E | | | | GND GND | \ \ / 100Ω (R2 — LED limit) \ | >| Green LED | GND Follow by: "How it works The 2.7 kΩ resistor charges the 220 µF capacitor from the 12 V supply. The capacitor voltage rises slowly. When the capacitor voltage reaches the transistor’s base-emitter threshold (~0.6–0.7 V), the transistor suddenly switches ON. When it turns on, the capacitor rapidly discharges through the base, causing: A brief pulse of current through the transistor The LED lights up through the 100 Ω resistor After discharge, the transistor turns back OFF, the LED turns off, and the capacitor begins charging again. This repeats automatically → LED flasher." The number of errors in the circuit and the utterly bogus explanation as well as the over confident remark that this is 'working' is so bizarre that I wonder how many slightly more complicated questions are going to yield results comparable to this one.
- emporas 11mo ago
  I have used Gemini for reading and solving electronic schematics exercises, and it's results were good enough for me. Roughly 50% of the exercises managed to solve correctly, 50% wrong. Simple R circuits. One time it messed up the opposite polarity of two voltage sources in series, and instead of subtracting their voltages, it added them together, I pointed out the mistake and Gemini insisted that the voltage sources are not in opposite polarity. Schematics in general are not AIs strongest point. But when you explain what math you want to calculate from an LRC circuit for example, no schematics, just describe in words the part of the circuit, GPT many times will calculate it correctly. It still makes mistakes here and there, always verify the calculation.
  - jacquesm 11mo ago
    I guess I'm just more critical than you are. I am used my computer doing what it is told and giving me correct, exact answers or errors.
    - emporas 11mo ago
      There is also Mercury LLM, which computes the answer directly as a 2D text representation. I don't know if you are familiar with Mercury LLM, but you read correctly, 2D text output. Mercury LLM might work better getting input as an ASCII diagram, or generating an output as an ASCII diagram, not sure if both input and output work 2D. Plumbing/electrical/electronic schematics are pretty important for AIs to understand and assist us, but for the moment the success rate is pretty low. 50% success rate for simple problems is very low, 80-90% success rate for medium difficulty problems is where they start being really useful.
      - jacquesm 11mo ago
        It's not really the quality of the diagramming that I am concerned with, it is the complete lack of understanding of electronics parts and their usual function. The diagramming is atrocious but I could live with it if the circuit were at least borderline correct. Extrapolating from this: if we use the electronics schematic as a proxy for the kind of world model these systems have then that world model has upside down lanterns and anti-gravity as commonplace elements. Three legged dogs mate with zebras and produce viable offspring and short circuiting transistors brings about entirely new physics.
verdverm 11mo ago
the problem with these arguments is there are data points to support both sides because both outcomes are possible the real thing is are you or we getting an ROI and the answer is increasingly more yeses on more problems, this trend is not looking to plateau as we step up the complexity ladder to agentic system

by baq 11mo ago

jacquesm 11mo ago

I don't reach for AI until I'm solidly stuck and then use it mostly for inspiration, it has yet to happen that it directly pointed at a solution and I'm pretty good at writing prompts. When I throw a bunch of elementary stuff at it then it is super good at identifying issues and solving them (but I could have done that myself, it is just nice to try to delineate where you can and where you can't trust the thing, but that too is fluctuating, sometimes even within a single session). Here is a nice example for a slightly more complex test: Prompt: "I have a green LED, a 12V powersupply, a single NPN transistor, a 100 ohm resistor, a 2.7K resistor and an electrolytic capacitor of 220 micro farads. My teacher says it is possible to make an LED flasher with these components but I'm hopelessly stuck, can you please give me an ascii art solution so I don't flunk this exam?" Answer: ASCII schematic (fully functional): +12V | | .-. | | | | 2.7kΩ (R1 — charges the capacitor) | | '-' | +---------+----------------------+ | | | ( ) | | ( ) C1 | | ( ) 220uF | | | | | | | | | |/ C | +--------| NPN transistor | | |\ E | | | | GND GND | \ \ / 100Ω (R2 — LED limit) \ | >| Green LED | GND Follow by: "How it works The 2.7 kΩ resistor charges the 220 µF capacitor from the 12 V supply. The capacitor voltage rises slowly. When the capacitor voltage reaches the transistor’s base-emitter threshold (~0.6–0.7 V), the transistor suddenly switches ON. When it turns on, the capacitor rapidly discharges through the base, causing: A brief pulse of current through the transistor The LED lights up through the 100 Ω resistor After discharge, the transistor turns back OFF, the LED turns off, and the capacitor begins charging again. This repeats automatically → LED flasher." The number of errors in the circuit and the utterly bogus explanation as well as the over confident remark that this is 'working' is so bizarre that I wonder how many slightly more complicated questions are going to yield results comparable to this one.

emporas 11mo ago
I have used Gemini for reading and solving electronic schematics exercises, and it's results were good enough for me. Roughly 50% of the exercises managed to solve correctly, 50% wrong. Simple R circuits. One time it messed up the opposite polarity of two voltage sources in series, and instead of subtracting their voltages, it added them together, I pointed out the mistake and Gemini insisted that the voltage sources are not in opposite polarity. Schematics in general are not AIs strongest point. But when you explain what math you want to calculate from an LRC circuit for example, no schematics, just describe in words the part of the circuit, GPT many times will calculate it correctly. It still makes mistakes here and there, always verify the calculation.
- jacquesm 11mo ago
  I guess I'm just more critical than you are. I am used my computer doing what it is told and giving me correct, exact answers or errors.
  - emporas 11mo ago
    There is also Mercury LLM, which computes the answer directly as a 2D text representation. I don't know if you are familiar with Mercury LLM, but you read correctly, 2D text output. Mercury LLM might work better getting input as an ASCII diagram, or generating an output as an ASCII diagram, not sure if both input and output work 2D. Plumbing/electrical/electronic schematics are pretty important for AIs to understand and assist us, but for the moment the success rate is pretty low. 50% success rate for simple problems is very low, 80-90% success rate for medium difficulty problems is where they start being really useful.
    - jacquesm 11mo ago
      It's not really the quality of the diagramming that I am concerned with, it is the complete lack of understanding of electronics parts and their usual function. The diagramming is atrocious but I could live with it if the circuit were at least borderline correct. Extrapolating from this: if we use the electronics schematic as a proxy for the kind of world model these systems have then that world model has upside down lanterns and anti-gravity as commonplace elements. Three legged dogs mate with zebras and produce viable offspring and short circuiting transistors brings about entirely new physics.

verdverm 11mo ago

the problem with these arguments is there are data points to support both sides because both outcomes are possible the real thing is are you or we getting an ROI and the answer is increasingly more yeses on more problems, this trend is not looking to plateau as we step up the complexity ladder to agentic system