7 ms·

Circuit Design and Applied Optimization (Part 1)

95 points by stefanpie 5y ago

hyperman1 5y ago
The formula for the duty cycle makes no sense. My rusty brain thinks it should be D=(R1+R2)/(R1+2R2), but it's multiple decades ago I used it so I might be wrong
- stefanpie 5y ago
  You are totally correct, this is only a typo in latex which I have just pushed a fix for. Thank for you the note.
  - hyperman1 5y ago
    Now it says 404. The correct URL has a -1 after it Update: Eight o'clock and all is well ;-). Thanks for the interesting article.
    - stefanpie 5y ago
      Should be back to normal again, apologies for the issues, web dev is not my strong point
      - antoine-levitt 5y ago
        ... And so that constraint forces R1 to be zero? Note that finding a solution to the problem originally stated is equivalent to a linear programming problem, which (for such a simple problem) can easily be solved exactly.
        deleted 5y ago
        [deleted]
        kragen 5y ago
        Dividing by R₁ + 2R₂ is not linear, and neither is multiplying C by R₁ + 2R₂, nor dividing by that product. But you could maybe formulate a linear objective function that solves this problem correctly, and then you could formulate the component selection problem as a MILP problem and solve it with GMPL/MathProg/GLPK or the COIN-OR tools. Glancing at it, though, it isn't obvious to me how to formulate it linearly. How would you do it?
neltnerb 5y ago
This reminds me of an undergraduate course on diffusion (atomic scale movement of atoms in a solid). There are known formal differential equations that let you solve for the exact diffusion profile given geometry, composition, diffusion rate constants, and temperature (basically). So on the homework they asked us to tell them how long it took for the concentration of the dopant to reach 10% at a 10 micron depth. So of course we all reached for this new math we had learned. We all got it wrong with the professor commenting that "you should have just used the approximate formula [which was trivial algebra], we only know the diffusion rate to an order of magnitude anyway". This was far more useful as feedback than the 0.2% of our grades we missed credit for. So I look at this essay and am a bit amused at minimizing calculation error to such an extent in a mathematical model when your resistors have a 5% tolerance and your capacitor is probably at 20% tolerance if you happened to have it in a university EE lab. But I do appreciate the fun of doing it this way too if you're not in a hurry =) It is crucial for any systems designer to realize that no matter how well you do your math and theory you have to also understand the sensitivity to variability so that you know what math is worth doing and what components we have no choice but to control the tolerances on better. You can find some fancy software for generating filter networks (similar in concept to this except with more discrete math because component values are discrete...) and ask it to show you sensitivity and you can see exactly how much you'll screw up your perfectly designed 10th order Chebyshev filter if that last resistor is off by 1%...
- stefanpie 5y ago
  Oh interesting application. I have taken a class in microelectronics physics (doping, electron holes, bandgaps, mosfets, bjts) and it was definitely just focusing on simpler approximations and first and second order effects only. But you are correct in that after my second or third electronics lab I realized that even standard component values vary to much that this approach is a little overkill and non-practical in a lot of cases. However there is a way to bake in discrete component values into the continuous optimization problems by using a continuous/"soft"/differentiable approximation of the min function to make "pockets" of optimal regions close to component values part of a set (for example e series values). I plan to do more writeups exploring this idea as well as looking at more complex applications such as higher order filters as you mentioned (my op-amp filter labs are what motivate me to look into this as I wasted so many hours in open lab trying to get my component values right).
hyperman1 5y ago
In feynman's 'Surely you're joking' there's a chapter where he designs some machinery for the army. He gets as guidance to use cogs from the middle of the list of available options, as the smallest or largest parts have all kinds of downsides. This idea works well in all kinds of situations where you have to select parts. I assume it might do well here, too.
- triactual 5y ago
  This works until Microsoft also picks from the middle to make ten trillion Xbox controllers overnight and you can’t get parts for six months. There are so many competing constraints just in component selection.
  - hyperman1 5y ago
    We're talking about E12 resistors here. I'm not going to worry too much.
xondono 5y ago
You might want to look into whatever you are using to render LaTeX, because it doesn’t work on iOS. I’ve had zero problems with Mathjax so far.
- stefanpie 5y ago
  I'll take a look at this, thank you for the node. I believe I'm using mathjax currently but I may have set set some options correctly. I also need to go in and verify a11ty as well.
- deleted 5y ago
  [deleted]
deleted 5y ago
[deleted]
docfort 5y ago
I believe this is supposed to be the first in a series on moving from continuous to discrete optimization, but the EE in me can’t help but point out what I would do in this scenario. It also connects with other interesting aspects of physics. Looking at the governing equations, you can clearly see that if R1 << R2, then the duty cycle is close to 50%. With that done, I also would have fixed the cap to something that is available. Ignoring R1 for the moment (because I just need to ensure R2 is bigger), I solve for R2 in the frequency equation. It is approximately 72 kOhms. I notice that a nonzero value of R1 is really there to tune the frequency. As long as R1 is much smaller than R2, then the frequency equation is more sensitive to changes in R1 than the duty cycle equation. So I can play with different small values of R1 to tune my frequency to get closer. Finally, since I know that I’m likely using imprecise resistor tolerances, I know that I can pass if I just get close, so I might not need to be picky about R1. In my opinion, this chain of reasoning (effective modeling, function sensitivity wrt parameters, tolerance specs) is what the lab experiment is actually about. Developing circuits that are tolerant to parameter variation is the key to real hardware products. That gives you so much flexibility in price and manufacturer and requires the designer to keep this kind of reasoning frontmost.
- WalterBright 5y ago
  Haha, this is just like designing a spring. You have a catalog of wire sizes, and wire materials, and an envelope. Too many unknowns, so a lot of trial and error before you start to hone in on an exact (but who knows if it is globally optimal) solution.
- kurthr 5y ago
  Exactly, and if you know you're not going to get better than 1% metal film resistors, why search more than the discrete space. Between 10^2 and 10^6 there's only 6-96 steps per decade (12 for the standard 10% E12 series). There's less than 400x400 possible resistor values to check.
  - stefanpie 5y ago
    I actually ended up doing some variation of this for scripts I wore for my labs. The space is actually quite small and it's not hard to brute force this problem. Once you get to more open ended designs and larger parameter spaces (like filters) it becomes a bit more challenging to brute force.
mayapugai 5y ago
I'm just here to say this article was such a pleasure to read. I've always relied on intuition to pick good-enough values based on experience. So this rigorous analytical approach to viewing the objective space and picking a reasonable value is refreshing. It's ironic and rightfully hilarious that you still had to take a gander and pick a "close-enough" value. I look forward to part 2 where you incorporate the resistor choices that you had. Perhaps also include the statistics of the tolerance into the mix to find the optimal values that we should all be picking for our future 555 timer hijinks. Your academic research work is also very interesting. Suffice to say I'll be following you on GitHub :) Cheers and happy (almost) new year!
kragen 5y ago
Brute-force search, as kurthr suggested doing, takes under 250ms to find a solution that's better on both axes by an order of magnitude, without Scipy, though it still took me 15 minutes to set it up. With R1 = 100Ω, R2 = 330kΩ, C = 2200pF, the frequency error is 7 Hz rather than 39 Hz (0.7% instead of 4%) and the duty-cycle error is 0.008% instead of 0.03%. Probably your capacitor is not going to be stable to 0.7% or precise to 1%, but you can certainly do better than 4%. (Use an NP0/C0G capacitor, not an X7R or something. In larger capacitance values you'd use a film cap instead, but 2200pF is fine for NP0/C0G.) You probably ought to measure some capacitors before running the optimization if you're only building one circuit, and if you're really interested in precision you might put a couple of trimpots across the resistors and adjust it while watching the scope (modern digital scopes can continuously display the frequency and duty cycle, so this is quick). But that will only help if most of the resistance and capacitance comes from components that won't drift over time or vary too much with temperature. Multiplying the loss values instead of adding them avoids having to choose weights for them. If you were going to use Newton's method or gradient descent, you might want to square them instead of taking the absolute value in order to get faster convergence, but of course that takes you back to solving a continuous relaxation of the discrete component selection problem you actually have. For branch-and-bound search a continuous relaxation can still be a useful thing to do, though. >>> R1 = R2 = 1000 >>> import math >>> u, n, p = 1e-6, 1e-9, 1e-12 >>> caps = [a * b * c for a in [1.0, 2.2, 4.7] for b in [1, 10, 100] for c in [u, n, p]] >>> e24 = [1.0, 1.1, 1.2, 1.3, 1.5, 1.6, 1.8, 2.0, 2.2, 2.4, 2.7, 3.0, 3.3, 3.6, 3.9, 4.3, 4.7, 5.1, 5.6, 6.2, 6.8, 7.5, 8.2, 9.1] >>> C = 47 * u >>> 1/(math.log(2) * (R1 + 2*R2) * C) # f 10.231879722616762 >>> (R1 + R2) / (R1 + 2*R2) # duty cycle 0.6666666666666666 >>> rs = [a * 10**b for a in e24 for b in range(2, 6)] >>> import time >>> s = time.time(); soln = min(((abs(1000 - f)/f) * (abs(0.5 - D)/D), R1, R2, C, f, D) for R1, R2, C, f, D in ((R1, R2, C, 1/(math.log(2) * (R1 + 2*R2) * C), (R1 + R2) / (R1 + 2*R2)) for R1 in rs for R2 in rs for C in caps)); time.time() - s 0.2474360466003418 >>> soln (1.0000300746386372e-06, 100.0, 330000.0, 2.2000000000000003e-09, 993.4411045771049, 0.5000757460990759) It would be fair to argue that the nested generator expression there is pretty hard to read, but it didn't really take that long to type, the Python REPL is pretty shitty at editing multiline functions, and I didn't feel like firing up Jupyter. But this way of writing it is definitely less unreadable and works just as well: >>> def search(rs1, rs2, cs): ... for R1 in rs1: ... for R2 in rs2: ... for C in cs: ... f = 1/(math.log(2) * (R1 + 2*R2) * C) ... D = (R1 + R2) / (R1 + 2*R2) ... yield (abs(1000 - f)/f) * (abs(0.5 - D)/D), R1, R2, C, f, D

95 points by stefanpie 5y ago

The formula for the duty cycle makes no sense. My rusty brain thinks it should be D=(R1+R2)/(R1+2R2), but it's multiple decades ago I used it so I might be wrong

stefanpie 5y ago
You are totally correct, this is only a typo in latex which I have just pushed a fix for. Thank for you the note.
- hyperman1 5y ago
  Now it says 404. The correct URL has a -1 after it Update: Eight o'clock and all is well ;-). Thanks for the interesting article.
  - stefanpie 5y ago
    Should be back to normal again, apologies for the issues, web dev is not my strong point
    - antoine-levitt 5y ago
      ... And so that constraint forces R1 to be zero? Note that finding a solution to the problem originally stated is equivalent to a linear programming problem, which (for such a simple problem) can easily be solved exactly.
      - deleted 5y ago
        [deleted]
      - kragen 5y ago
        Dividing by R₁ + 2R₂ is not linear, and neither is multiplying C by R₁ + 2R₂, nor dividing by that product. But you could maybe formulate a linear objective function that solves this problem correctly, and then you could formulate the component selection problem as a MILP problem and solve it with GMPL/MathProg/GLPK or the COIN-OR tools. Glancing at it, though, it isn't obvious to me how to formulate it linearly. How would you do it?

neltnerb 5y ago

This reminds me of an undergraduate course on diffusion (atomic scale movement of atoms in a solid). There are known formal differential equations that let you solve for the exact diffusion profile given geometry, composition, diffusion rate constants, and temperature (basically). So on the homework they asked us to tell them how long it took for the concentration of the dopant to reach 10% at a 10 micron depth. So of course we all reached for this new math we had learned. We all got it wrong with the professor commenting that "you should have just used the approximate formula [which was trivial algebra], we only know the diffusion rate to an order of magnitude anyway". This was far more useful as feedback than the 0.2% of our grades we missed credit for. So I look at this essay and am a bit amused at minimizing calculation error to such an extent in a mathematical model when your resistors have a 5% tolerance and your capacitor is probably at 20% tolerance if you happened to have it in a university EE lab. But I do appreciate the fun of doing it this way too if you're not in a hurry =) It is crucial for any systems designer to realize that no matter how well you do your math and theory you have to also understand the sensitivity to variability so that you know what math is worth doing and what components we have no choice but to control the tolerances on better. You can find some fancy software for generating filter networks (similar in concept to this except with more discrete math because component values are discrete...) and ask it to show you sensitivity and you can see exactly how much you'll screw up your perfectly designed 10th order Chebyshev filter if that last resistor is off by 1%...

stefanpie 5y ago
Oh interesting application. I have taken a class in microelectronics physics (doping, electron holes, bandgaps, mosfets, bjts) and it was definitely just focusing on simpler approximations and first and second order effects only. But you are correct in that after my second or third electronics lab I realized that even standard component values vary to much that this approach is a little overkill and non-practical in a lot of cases. However there is a way to bake in discrete component values into the continuous optimization problems by using a continuous/"soft"/differentiable approximation of the min function to make "pockets" of optimal regions close to component values part of a set (for example e series values). I plan to do more writeups exploring this idea as well as looking at more complex applications such as higher order filters as you mentioned (my op-amp filter labs are what motivate me to look into this as I wasted so many hours in open lab trying to get my component values right).

hyperman1 5y ago

In feynman's 'Surely you're joking' there's a chapter where he designs some machinery for the army. He gets as guidance to use cogs from the middle of the list of available options, as the smallest or largest parts have all kinds of downsides. This idea works well in all kinds of situations where you have to select parts. I assume it might do well here, too.

triactual 5y ago
This works until Microsoft also picks from the middle to make ten trillion Xbox controllers overnight and you can’t get parts for six months. There are so many competing constraints just in component selection.
- hyperman1 5y ago
  We're talking about E12 resistors here. I'm not going to worry too much.

xondono 5y ago

You might want to look into whatever you are using to render LaTeX, because it doesn’t work on iOS. I’ve had zero problems with Mathjax so far.

stefanpie 5y ago
I'll take a look at this, thank you for the node. I believe I'm using mathjax currently but I may have set set some options correctly. I also need to go in and verify a11ty as well.
deleted 5y ago
[deleted]

deleted 5y ago

[deleted]

docfort 5y ago

I believe this is supposed to be the first in a series on moving from continuous to discrete optimization, but the EE in me can’t help but point out what I would do in this scenario. It also connects with other interesting aspects of physics. Looking at the governing equations, you can clearly see that if R1 << R2, then the duty cycle is close to 50%. With that done, I also would have fixed the cap to something that is available. Ignoring R1 for the moment (because I just need to ensure R2 is bigger), I solve for R2 in the frequency equation. It is approximately 72 kOhms. I notice that a nonzero value of R1 is really there to tune the frequency. As long as R1 is much smaller than R2, then the frequency equation is more sensitive to changes in R1 than the duty cycle equation. So I can play with different small values of R1 to tune my frequency to get closer. Finally, since I know that I’m likely using imprecise resistor tolerances, I know that I can pass if I just get close, so I might not need to be picky about R1. In my opinion, this chain of reasoning (effective modeling, function sensitivity wrt parameters, tolerance specs) is what the lab experiment is actually about. Developing circuits that are tolerant to parameter variation is the key to real hardware products. That gives you so much flexibility in price and manufacturer and requires the designer to keep this kind of reasoning frontmost.

WalterBright 5y ago
Haha, this is just like designing a spring. You have a catalog of wire sizes, and wire materials, and an envelope. Too many unknowns, so a lot of trial and error before you start to hone in on an exact (but who knows if it is globally optimal) solution.
kurthr 5y ago
Exactly, and if you know you're not going to get better than 1% metal film resistors, why search more than the discrete space. Between 10^2 and 10^6 there's only 6-96 steps per decade (12 for the standard 10% E12 series). There's less than 400x400 possible resistor values to check.
- stefanpie 5y ago
  I actually ended up doing some variation of this for scripts I wore for my labs. The space is actually quite small and it's not hard to brute force this problem. Once you get to more open ended designs and larger parameter spaces (like filters) it becomes a bit more challenging to brute force.

mayapugai 5y ago

I'm just here to say this article was such a pleasure to read. I've always relied on intuition to pick good-enough values based on experience. So this rigorous analytical approach to viewing the objective space and picking a reasonable value is refreshing. It's ironic and rightfully hilarious that you still had to take a gander and pick a "close-enough" value. I look forward to part 2 where you incorporate the resistor choices that you had. Perhaps also include the statistics of the tolerance into the mix to find the optimal values that we should all be picking for our future 555 timer hijinks. Your academic research work is also very interesting. Suffice to say I'll be following you on GitHub :) Cheers and happy (almost) new year!

kragen 5y ago

Brute-force search, as kurthr suggested doing, takes under 250ms to find a solution that's better on both axes by an order of magnitude, without Scipy, though it still took me 15 minutes to set it up. With R1 = 100Ω, R2 = 330kΩ, C = 2200pF, the frequency error is 7 Hz rather than 39 Hz (0.7% instead of 4%) and the duty-cycle error is 0.008% instead of 0.03%. Probably your capacitor is not going to be stable to 0.7% or precise to 1%, but you can certainly do better than 4%. (Use an NP0/C0G capacitor, not an X7R or something. In larger capacitance values you'd use a film cap instead, but 2200pF is fine for NP0/C0G.) You probably ought to measure some capacitors before running the optimization if you're only building one circuit, and if you're really interested in precision you might put a couple of trimpots across the resistors and adjust it while watching the scope (modern digital scopes can continuously display the frequency and duty cycle, so this is quick). But that will only help if most of the resistance and capacitance comes from components that won't drift over time or vary too much with temperature. Multiplying the loss values instead of adding them avoids having to choose weights for them. If you were going to use Newton's method or gradient descent, you might want to square them instead of taking the absolute value in order to get faster convergence, but of course that takes you back to solving a continuous relaxation of the discrete component selection problem you actually have. For branch-and-bound search a continuous relaxation can still be a useful thing to do, though. >>> R1 = R2 = 1000 >>> import math >>> u, n, p = 1e-6, 1e-9, 1e-12 >>> caps = [a * b * c for a in [1.0, 2.2, 4.7] for b in [1, 10, 100] for c in [u, n, p]] >>> e24 = [1.0, 1.1, 1.2, 1.3, 1.5, 1.6, 1.8, 2.0, 2.2, 2.4, 2.7, 3.0, 3.3, 3.6, 3.9, 4.3, 4.7, 5.1, 5.6, 6.2, 6.8, 7.5, 8.2, 9.1] >>> C = 47 * u >>> 1/(math.log(2) * (R1 + 2*R2) * C) # f 10.231879722616762 >>> (R1 + R2) / (R1 + 2*R2) # duty cycle 0.6666666666666666 >>> rs = [a * 10**b for a in e24 for b in range(2, 6)] >>> import time >>> s = time.time(); soln = min(((abs(1000 - f)/f) * (abs(0.5 - D)/D), R1, R2, C, f, D) for R1, R2, C, f, D in ((R1, R2, C, 1/(math.log(2) * (R1 + 2*R2) * C), (R1 + R2) / (R1 + 2*R2)) for R1 in rs for R2 in rs for C in caps)); time.time() - s 0.2474360466003418 >>> soln (1.0000300746386372e-06, 100.0, 330000.0, 2.2000000000000003e-09, 993.4411045771049, 0.5000757460990759) It would be fair to argue that the nested generator expression there is pretty hard to read, but it didn't really take that long to type, the Python REPL is pretty shitty at editing multiline functions, and I didn't feel like firing up Jupyter. But this way of writing it is definitely less unreadable and works just as well: >>> def search(rs1, rs2, cs): ... for R1 in rs1: ... for R2 in rs2: ... for C in cs: ... f = 1/(math.log(2) * (R1 + 2*R2) * C) ... D = (R1 + R2) / (R1 + 2*R2) ... yield (abs(1000 - f)/f) * (abs(0.5 - D)/D), R1, R2, C, f, D