11 ms·

We're building computers wrong [video]

75 points by lawrenceyan 5y ago

phendrenad2 5y ago
With a clickbaity title like that, I knew this would be Veritisium before I clicked it. And now, having verified that it is, I share this information with you.
- kgwxd 5y ago
  And if you're curious as to why it's clickbaity, there's a Veritasium video for that too :) https://www.youtube.com/watch?v=S2xHZPH5Sng https://www.youtube.com/watch?v=S2xHZPH5Sng
  - ModernMech 5y ago
    I didn't watch the whole thing but is the ladder ever explained? I didn't understand why he's doing the video on a ladder. Is that part of the clickbait? You have to have a crazy face and be on a ladder, and that's how you make Youtube's algorithm happy? At this point it seems like a vicious spiral. Someone clicks on a guy on a ladder and all of a sudden you're being recommended guys on ladders, so now everyone needs to get on a ladder to get on the front page.
    - droopyEyelids 5y ago
      Youtube runs the thumbnails through a sentiment analysis AI and promotes videos more when they rate with a high likelihood of the bucket of emotions that have been found to correlate with ad impressions.
- l33t2328 5y ago
  Something about Veritasium absolutely rubs me the wrong way, and I generally like Youtube education creators, but his videos are borderline unwatchable for me. Does anyone have an explanation for what it is about his videos that is so grating?
  - jebr224 5y ago
    same, It comes of as smug or some desire to prove he smarter, rather then genuinely trying to educate. The videos are of educational, high quality, and the content is accurate. But the presentation is not for me.
    - netizen-936824 5y ago
      I feel like its more the assertiveness on how right the information he's sharing is, when not everything he makes a video on is quite so correct but there isn't any of this nuance or doubt in his works. Doesn't feel like any of the professors I've spoken to, there's too much confidence. The only thing we can be that confident in is how wrong humans tend to be
  - Brian_K_White 5y ago
    I don't know how to articulate it either. It's some form of uncanny valley type of thing where every detail or facet or property I might point out, there are other people exhibiting the same nominal qualities without bothering me. But something here is just slightly off in some way, but so subtly you can't say what it is. I honestly can't say he's doing anything actually wrong. I don't think even the most dumbed down ones are really guilty of being wrong. Not enough that I'd complain about it in a casual yt video. I can see some people maybe not liking his basic style of exposition. That slightly affected dreamy pondering thing. One thing I can observe is that another guy that just turns me right off even worse is the Smarter Every Day, and these two appear in each others videos at least a few times. Or maybe it was just one time and that one time just bugged me that much :) I don't know what it is but I frequently see a video from either of these guys where the topic they are presenting looks interesting, but then I do not like their video on that topic. I think it's down to the presentation just seems a bit too affected. And it's not even as bad as seeming fake or insincere. They both actually strike me as perfectly sincere.
    - jalgos_eminator 5y ago
      Its interesting you mention Destin from Smarter Every Day, because he always came off as a very earnest learner in his videos. Yeah, he's still putting on a show, but it does seem like he genuinely enjoys and revels in the things that he does. It seems to me like Veritasium has a more contrarian thread of: "Oh, you thought the world worked like that? Well you were wrong, and here's a 15 minute video all about how you were wrong." There's an Onion video making fun of Vox that touches on this perfectly: https://www.youtube.com/watch?v=RpkQEq75y18 https://www.youtube.com/watch?v=RpkQEq75y18 I guess to summarize, with Destin it feels like you are learning with him and Veritasium feels like you are being lectured.
- tronster 5y ago
  I'm subscribed to his channel, as he's covered some great science topics in the past. I skipped this video based on the title - it doesn't tell me anything about the content and "feels" more like a plea for the video to be watched than some good science broken down. I imagine that, over time, this may shift his target audience. But if a new audience is comprised of a demographic he's aiming to reach, and it gets him the views, I can't fault his decision on how he titles the videos, even if they have me skipping them.
  - tacitusarc 5y ago
    He updated the title. I watched it when he posted it, at that time it ended in “(for AI)” I think he’s experimenting with clickbait.
    - deleted 5y ago
      [deleted]
    - CSSer 5y ago
      Committed to, actually. He made a video about how they were going to start doing it more: https://youtu.be/fHsa9DqmId8?t=1035 https://youtu.be/fHsa9DqmId8?t=1035. Also, more recently: https://www.youtube.com/watch?v=S2xHZPH5Sng https://www.youtube.com/watch?v=S2xHZPH5Sng.
    - lIIIllllIIII 5y ago
      I thought it was actually just native advertising for Mythic. Besides that I think the topic is prescient; there's a company local to me in Australia called Brainchip that is doing something similar I think. Given that NNs are just a bunch of matrix multiplications it's a promising approach. I occasionally used to watch Smarter Every Day then he started doing a lot of this stuff, I think one of them might have been for an oil or defense company which I found utterly gross and I've avoided watching the channel ever since.
- autokad 5y ago
  One thing about Veritasium is that they do get things wrong. They did one on electric circuits where if you had a circuit that was miles long with the energy source sitting right next to you, you would get current as soon as you turned on the electrical circuit because 'electricity doesnt follow the wire' or something along those lines. This has been disproven by people doing experiments
  - etskinner 5y ago
    Can you share the experiments that disprove it? The one video I saw proved it. Someone set up some long wires over a farm field, and measured the time to first current flow, which was faster than if the current had to travel the whole wire length. Granted, the full current flow doesn't appear until later, but the first little bit of current is there, which is exactly what Derek said in the Veritasium video
    - autokad 5y ago
      I believe this is the same video, the author theorized (probably correctly) that it was because there was residual 'power' on the wire, as it acts like a giant capacitor, among other things. this is because the wire that immediately showed up was extremely small and dissipated very quickly. https://www.youtube.com/watch?v=2Vrhk5OjBP8 https://www.youtube.com/watch?v=2Vrhk5OjBP8
  - mannykannot 5y ago
    One could quibble over the details of the experiment, such as what sort of lamp to use, but Maxwell's laws would have to be repealed for the experiment to not work essentially as Veritasium claimed. Check out "electromagnetic induction" and "displacement current".
    - LordDragonfang 5y ago
      The problem is that he frames the whole whole experiment as if the inductive current is a property of the circuit rather than a property of two separate wires being next to each other - he heavily implies the inductance is result of the current "starting" to travel through the loop, and IIRC never once uses the term "inductance" (which seems odd for a video purporting to promote understanding rather than just be clickbait). (The fact that the loop is closed can't be relevant by simple thought experiment, because otherwise it implies you could get ftl signaling by having a switch on the physically distant checking whether the "tiny current" flows when you close the source)
123pie123 5y ago
TL;DW found it bit interesting, goes on about the history of AI/ Neural Networks and says these do not need the precision of digital computers (manly requiring matrix math) and this can be done faster with analogue computers (using variable resistors type of transistors)
dkislyuk 5y ago
For a more realistic view on AI accelerators that doesn't overhype analog computing, I enjoyed the series by Adi Fuchs: https://medium.com/@adi.fu7/ai-accelerators-part-i-intro-822c2cdb4ca4 https://medium.com/@adi.fu7/ai-accelerators-part-i-intro-822... At the end of the day, specialized hardware, particularly on the analog side (neuromorphic, optical, etc.) locks development into the path of highly uniform feedforward networks by optimizing large matrix multiplications, and it is unclear if this tradeoff is worth it as we still have so much to figure out about which methods will make progress in AI.
- GlenTheMachine 5y ago
  Neuromorphics aren't analog FWIW. They're just asynchronous.
  - aurelian15 5y ago
    Depends ;-) First of all, there is no single accepted definition of "neuromorphic" [1]. Still, as a point in favour of the "neuromorphic systems are analogue" crowd: the seminal paper by Carver Mead that (to my knowledge) coined the term "neuromorphics" specifically talks about analogue neuromorphic systems [2]. Right now, there are some research "analogue" (or, more precisely "mixed signal") neuromorphic systems being developed [3, 4]. It is correct however that there are no commercially available analogue systems that I am aware of. Unfortunately, the same can be said for digital neuromorphics as well (Intel Loihi is perhaps the closest to a commercial product, and yes, this is an asynchronous digital neuromorphic system). [1] https://iopscience.iop.org/article/10.1088/1741-2560/13/5/051001/pdf https://iopscience.iop.org/article/10.1088/1741-2560/13/5/05... [2] https://authors.library.caltech.edu/53090/1/00058356.pdf https://authors.library.caltech.edu/53090/1/00058356.pdf [3] https://brainscales.kip.uni-heidelberg.de/ https://brainscales.kip.uni-heidelberg.de/ [4] https://web.stanford.edu/group/brainsinsilicon/documents/ANeckar2019.pdf https://web.stanford.edu/group/brainsinsilicon/documents/ANe...
aidenn0 5y ago
Are analog computers actually more energy efficient than a digital ASIC with similar accuracy (say 6-10 bits)?
- boxed 5y ago
  Yes. By a lot.
Strilanc 5y ago
I'm a bit worried for that analog-matrix-multiplication-for-AI company. I vaguely remember reading somewhere that the past half century is littered with companies whose value proposition was "our specialized thing does 10x better than digital transistors" and then they predictably just got steamrolled by Moore's law. And although Dennard scaling ended two decades ago, the flops-per-watt and flops-per-second of AI-specialized chips like TPUs has been improving substantially recently [1][2]. 1: https://en.wikipedia.org/wiki/Tensor_Processing_Unit#Third_generation_TPU https://en.wikipedia.org/wiki/Tensor_Processing_Unit#Third_g... 2: https://arxiv.org/abs/2005.04305 https://arxiv.org/abs/2005.04305
- uxp100 5y ago
  Fuzzy logic was pretty closely tied to Neural Networks at one point (80s to early 90s?) in terms of which researchers talked about them and even just which books info about them was located in. I think dedicated analog fuzzy logic chips were, as you say, steamrolled by Moore's Law. (My zorojushi rice cooker still mentions fuzzy logic, but they must be just implementing the fuzzy "transfer function" with an mcu at this point, right?)
- snakke 5y ago
  You're correct that specialised analog companies have not done well historically. However, we don't find ourselves in exactly the same position in computer architecture/performance as we've been decades before. There's some (relatively) new ideas that now the performance of computers will be pushed more by dedicated silicon for a dedicated purpose, and tools. See for example there's plenty of room at the top [1], or Hennesy's talk at Google [2]. This of course does not mean that analog computers are suddenly viable, but it does mean that they could potentially fill a niche where they failed previously. Anecdotally, when looking at jobs for hardware design by the likes of Infineon, STM, Cyient etc. there seems to be a relatively high ask for (senior) analog designers, and a new focus on mixed-technology chips. It might turn out to be a dud still, but it isn't the same situation as decades before. 1: https://www.science.org/doi/abs/10.1126/science.aam9744 https://www.science.org/doi/abs/10.1126/science.aam9744 , or the IEEE article https://ieeexplore.ieee.org/abstract/document/7863324 https://ieeexplore.ieee.org/abstract/document/7863324 2: https://www.youtube.com/watch?v=Azt8Nc-mtKM https://www.youtube.com/watch?v=Azt8Nc-mtKM , around the 12:15 mark, but the entire video is relevant.
antattack 5y ago
If you think about it, Mythic analog AI accelerators will also develop neurological 'disease' as it's flash cells wear out.
- jecel 5y ago
  Since the chip is meant for inference but not for training, in theory you only write to the Flash once or twice. In this case data retention is a more limiting factor than wear. For digital storage the data can change quite a bit before a 1 becomes 0 or vice-versa, but for analog even a small change can be a problem. So I don't know if we can extrapolate the typical Flash memory data retention number to the expected aging for this device.
mastax 5y ago
This video discusses the history and principles of classical analog computing and then says that analog computers have energy efficiency benefits over digital computers. This can give the incorrect impression that classical analog computers are more efficient than digital computers, which is absolutely not correct. It may be true that modern analog phase-change memory AI accelerators are more efficient than their digital counterparts, but this assertion should be distanced from the discussion of classical analog computers to not give the wrong impression.
- eternityforest 5y ago
  People always love to say some old lost tech is better than the modern version But it's like, a one in a thousand shot.Almost all modern tech is better with regressions lasting a decade at most. Occasionally there are rumors of some modern medical treatments being worse even though medicine as a whole improves, but mechanical and electrical stuff is much better understood and more controlled. The exception is a few specific appliances that are seemingly designed to fail. Even then there's no lost tech. I'm pretty confident we could make a much better washing machine today for much less money if we tried, than anything from 30 or 50 years ago.
  - autokad 5y ago
    The analog computers on the Missouri class battleships was never upgraded because the kill radius of the shells was larger than the margin of error the analog computers introduced and that the analog computers required less electricity which is good for something such as a ship. So in this way, yes, they are better (these computers developed in the 30s). It depends on the application. > Occasionally there are rumors of some modern medical treatments being worse even though medicine as a whole improves, but mechanical and electrical stuff is much better understood and more controlled. Such as the iron lung probably being better than the invasive machines we have today. > The exception is a few specific appliances that are seemingly designed to fail. Even then there's no lost tech. I'm pretty confident we could make a much better washing machine today for much less money if we tried, than anything from 30 or 50 years ago. look at the space industry, it actually costs us more (adjusting for inflation) to try and go back to the moon than when we did before when we had no idea on how to do it. Or look at tractors, farmers are clamoring for 1980s tractors over these new expensive (break down easy with no way to repair) 'modern' tractors John Deer is pushing.
jodrellblank 5y ago
In the intro to this video he mentions analogue computers predicting tides. His video before this one[1] goes into detail on that, and it was incredible. In the 1800s, Lord Kelvin spent years working on tidal rise and fall patterns, applying Fourier Transforms to break them down into ten individual sine waves, then combining those sine waves back together to predict future tides. And built analog computers to do all the involved integration, multiplying and summing. It's a part of the history of computing I'd never heard of, despite hearing quite a lot about the dawn of electro-mechanical computers in the early 1900s. [1] https://www.youtube.com/watch?v=IgF3OX8nT0w https://www.youtube.com/watch?v=IgF3OX8nT0w
- ggm 5y ago
  The machines (or later derivatives) is in the science museum. One variant uses cones, I imagine they provide for a huge amount of variance of the parameters of the fourier transform, another is pulleys, but changing out a pulley is a lot more work than moving where two cones are co-incident rubbing to transfer motion. Along with a meccano numerical analyser, and bits of babbage's original work
mrvenkman 5y ago
I have to say that the title is definitely “click baity” - I watched the video because I wondered what we were doing wrong. It is focused on how to improve AI when we run out of atom sized hardware. I don’t appreciate the title.
agumonkey 5y ago
i find funny the parallel between the uptake of reactive programming and analog computing (same kind of real time smooth coupling)
Animats 5y ago
That video has one of the best explanations of how and why deep learning works. The argument for analog is weaker. I've tried some stuff with analog computers, and at one time I was into op-amp circuits. The basic problems are noise and inflexibility. However, that may change as people develop ICs that are reconfigurable, like an FPGA. Flash memory cells as multipliers by a changeable but not dynamic value are a new idea. That only leads to analog ICs that do a pre-trained neural net, though. Running neural nets isn't that expensive. It's training deep neural nets that needs entire data centers. If they can figure out a way to do the whole backpropagation thing in analog, that would be impressive.
periheli0n 5y ago
Yes, analog computing can be more efficient and faster. But there are reasons why analog was eclipsed by digital. 1. Flexibility. Reprogramming digital computation is easy and quick. 2. Robustness. One can mass-produce devices that operate in the digital domain and sell them as working as spec'd. But operate them in the analog domain, and you will work with a million snowflakes that all operate slightly differently, giving different computation results. And when temperature changes, the result of your analog computation will inevitably change, too. You can work around that by adding more circuitry, and partly also on the algorithmic end but it will cost efficiency and precision. Matrix multiplication on an analog device is great, unless you want an exact and reproducible result.
- R0b0t1 5y ago
  He touches on this in his first video. The workaround is we are now getting into a regime of computing where exact reproducibility is not necessary. It will take a while though. It is still quite hard to remove noise for opamp circuits.
  - periheli0n 5y ago
    > we are now getting into a regime of computing where exact reproducibility is not necessary. I think that is a myth. Predictable, reproducible and explainable outcomes are the holy grail of computing, in particular in AI. If stochasticity is desired, there are methods to inject it, with precise control of the level of stochasticity and the distribution. This level of control is absent in analog computing. Device mismatch introduces some randomness, but it cannot be controlled in practice. Instead of adapting the randomness to the algorithm, one ends up adapting the algorithm to the randomness. Working with analog computers is a fascinating academic exercise. For practical applications, I doubt we’ll see it compete with digital computation anytime soon.
    - R0b0t1 5y ago
      I said exact reproducibility. Every time you add numbers your neurons likely fire in slightly different ways, but the end result is what is reproducible and useful. He's not totally correct because, like I said, the way we think of analog computers right now is likely not going to get us to where we need to be. Noise is a huge problem. Our own neurons do not seem to use continuous voltage, for example, and use pulse density coding, at least for some of their operation.
mikewarot 5y ago
Let's use the LM358DT as an example of a modern op-amp. Just sitting there, it has current sources sinking 100uA of current. At 5 volts, it consumes about 0.7 mA with no load. So, compare that to a 4 bit ALU (an old one, at that) the 74HC181. At 5 volts it consumes 80uA at idle. Analog circuits require transistors have bias flows through them at all times to ensure linearity. CMOS digital logic has the transistors either fully on or off, except during switching times. Going analog isn't as great an option as this video makes it seem.
rapjr9 5y ago
Check out the work of Kofi Odame on implementing algorithms in analog circuitry on integrated circuits: https://engineering.dartmouth.edu/community/faculty/kofi-odame https://engineering.dartmouth.edu/community/faculty/kofi-oda... He's interested in creating analog circuits that are more efficient than digital ones for signal processing and machine learning. I've programmed an analog computer when I was working in the military-industrial complex long ago, implemented a standard missile guidance algorithm for an IR missile using a dual feedback loop and adjustable nav ratio. It was kind of like programming with wires to implement math. It had noise limitations on accuracy, in part because of the many wires exposed to florescent lights and digital computers in the room. Quite interesting. I think the Analog Thing might make a good EuroRack music making module! Turn math into music.
olliej 5y ago
I liked some of veritasium's older stuff but it seems to be getting more and more clickbait as time goes on (which is reasonable as it's clear that YT is his job, and YT promotes clickbait above everything else). A few people said that they "knew" it would be veritasium, but I had guessed it would be Mill computer folk :D
- deleted 5y ago
  [deleted]
charcircuit 5y ago
I feel like it was a bit misleading to mention large amount of energy to train a model and then show a chip which can only be used for inference and not training. In general people are optimizing for power use at the edge and not for training.
teleforce 5y ago
For a wonderful introduction to digital and analog computers please check the book by Steiglitz, "The Discrete Charm of the Machine" [1]. [1] https://leonardo.info/review/2019/07/review-of-the-discrete-charm-of-the-machine-why-the-world-became-digital https://leonardo.info/review/2019/07/review-of-the-discrete-...

75 points by lawrenceyan 5y ago

With a clickbaity title like that, I knew this would be Veritisium before I clicked it. And now, having verified that it is, I share this information with you.

kgwxd 5y ago
And if you're curious as to why it's clickbaity, there's a Veritasium video for that too :) https://www.youtube.com/watch?v=S2xHZPH5Sng https://www.youtube.com/watch?v=S2xHZPH5Sng
- ModernMech 5y ago
  I didn't watch the whole thing but is the ladder ever explained? I didn't understand why he's doing the video on a ladder. Is that part of the clickbait? You have to have a crazy face and be on a ladder, and that's how you make Youtube's algorithm happy? At this point it seems like a vicious spiral. Someone clicks on a guy on a ladder and all of a sudden you're being recommended guys on ladders, so now everyone needs to get on a ladder to get on the front page.
  - droopyEyelids 5y ago
    Youtube runs the thumbnails through a sentiment analysis AI and promotes videos more when they rate with a high likelihood of the bucket of emotions that have been found to correlate with ad impressions.
l33t2328 5y ago
Something about Veritasium absolutely rubs me the wrong way, and I generally like Youtube education creators, but his videos are borderline unwatchable for me. Does anyone have an explanation for what it is about his videos that is so grating?
- jebr224 5y ago
  same, It comes of as smug or some desire to prove he smarter, rather then genuinely trying to educate. The videos are of educational, high quality, and the content is accurate. But the presentation is not for me.
  - netizen-936824 5y ago
    I feel like its more the assertiveness on how right the information he's sharing is, when not everything he makes a video on is quite so correct but there isn't any of this nuance or doubt in his works. Doesn't feel like any of the professors I've spoken to, there's too much confidence. The only thing we can be that confident in is how wrong humans tend to be
- Brian_K_White 5y ago
  I don't know how to articulate it either. It's some form of uncanny valley type of thing where every detail or facet or property I might point out, there are other people exhibiting the same nominal qualities without bothering me. But something here is just slightly off in some way, but so subtly you can't say what it is. I honestly can't say he's doing anything actually wrong. I don't think even the most dumbed down ones are really guilty of being wrong. Not enough that I'd complain about it in a casual yt video. I can see some people maybe not liking his basic style of exposition. That slightly affected dreamy pondering thing. One thing I can observe is that another guy that just turns me right off even worse is the Smarter Every Day, and these two appear in each others videos at least a few times. Or maybe it was just one time and that one time just bugged me that much :) I don't know what it is but I frequently see a video from either of these guys where the topic they are presenting looks interesting, but then I do not like their video on that topic. I think it's down to the presentation just seems a bit too affected. And it's not even as bad as seeming fake or insincere. They both actually strike me as perfectly sincere.
  - jalgos_eminator 5y ago
    Its interesting you mention Destin from Smarter Every Day, because he always came off as a very earnest learner in his videos. Yeah, he's still putting on a show, but it does seem like he genuinely enjoys and revels in the things that he does. It seems to me like Veritasium has a more contrarian thread of: "Oh, you thought the world worked like that? Well you were wrong, and here's a 15 minute video all about how you were wrong." There's an Onion video making fun of Vox that touches on this perfectly: https://www.youtube.com/watch?v=RpkQEq75y18 https://www.youtube.com/watch?v=RpkQEq75y18 I guess to summarize, with Destin it feels like you are learning with him and Veritasium feels like you are being lectured.
tronster 5y ago
I'm subscribed to his channel, as he's covered some great science topics in the past. I skipped this video based on the title - it doesn't tell me anything about the content and "feels" more like a plea for the video to be watched than some good science broken down. I imagine that, over time, this may shift his target audience. But if a new audience is comprised of a demographic he's aiming to reach, and it gets him the views, I can't fault his decision on how he titles the videos, even if they have me skipping them.
- tacitusarc 5y ago
  He updated the title. I watched it when he posted it, at that time it ended in “(for AI)” I think he’s experimenting with clickbait.
  - deleted 5y ago
    [deleted]
  - CSSer 5y ago
    Committed to, actually. He made a video about how they were going to start doing it more: https://youtu.be/fHsa9DqmId8?t=1035 https://youtu.be/fHsa9DqmId8?t=1035. Also, more recently: https://www.youtube.com/watch?v=S2xHZPH5Sng https://www.youtube.com/watch?v=S2xHZPH5Sng.
  - lIIIllllIIII 5y ago
    I thought it was actually just native advertising for Mythic. Besides that I think the topic is prescient; there's a company local to me in Australia called Brainchip that is doing something similar I think. Given that NNs are just a bunch of matrix multiplications it's a promising approach. I occasionally used to watch Smarter Every Day then he started doing a lot of this stuff, I think one of them might have been for an oil or defense company which I found utterly gross and I've avoided watching the channel ever since.
autokad 5y ago
One thing about Veritasium is that they do get things wrong. They did one on electric circuits where if you had a circuit that was miles long with the energy source sitting right next to you, you would get current as soon as you turned on the electrical circuit because 'electricity doesnt follow the wire' or something along those lines. This has been disproven by people doing experiments
- etskinner 5y ago
  Can you share the experiments that disprove it? The one video I saw proved it. Someone set up some long wires over a farm field, and measured the time to first current flow, which was faster than if the current had to travel the whole wire length. Granted, the full current flow doesn't appear until later, but the first little bit of current is there, which is exactly what Derek said in the Veritasium video
  - autokad 5y ago
    I believe this is the same video, the author theorized (probably correctly) that it was because there was residual 'power' on the wire, as it acts like a giant capacitor, among other things. this is because the wire that immediately showed up was extremely small and dissipated very quickly. https://www.youtube.com/watch?v=2Vrhk5OjBP8 https://www.youtube.com/watch?v=2Vrhk5OjBP8
- mannykannot 5y ago
  One could quibble over the details of the experiment, such as what sort of lamp to use, but Maxwell's laws would have to be repealed for the experiment to not work essentially as Veritasium claimed. Check out "electromagnetic induction" and "displacement current".
  - LordDragonfang 5y ago
    The problem is that he frames the whole whole experiment as if the inductive current is a property of the circuit rather than a property of two separate wires being next to each other - he heavily implies the inductance is result of the current "starting" to travel through the loop, and IIRC never once uses the term "inductance" (which seems odd for a video purporting to promote understanding rather than just be clickbait). (The fact that the loop is closed can't be relevant by simple thought experiment, because otherwise it implies you could get ftl signaling by having a switch on the physically distant checking whether the "tiny current" flows when you close the source)

123pie123 5y ago

TL;DW found it bit interesting, goes on about the history of AI/ Neural Networks and says these do not need the precision of digital computers (manly requiring matrix math) and this can be done faster with analogue computers (using variable resistors type of transistors)

dkislyuk 5y ago

For a more realistic view on AI accelerators that doesn't overhype analog computing, I enjoyed the series by Adi Fuchs: https://medium.com/@adi.fu7/ai-accelerators-part-i-intro-822c2cdb4ca4 https://medium.com/@adi.fu7/ai-accelerators-part-i-intro-822... At the end of the day, specialized hardware, particularly on the analog side (neuromorphic, optical, etc.) locks development into the path of highly uniform feedforward networks by optimizing large matrix multiplications, and it is unclear if this tradeoff is worth it as we still have so much to figure out about which methods will make progress in AI.

GlenTheMachine 5y ago
Neuromorphics aren't analog FWIW. They're just asynchronous.
- aurelian15 5y ago
  Depends ;-) First of all, there is no single accepted definition of "neuromorphic" [1]. Still, as a point in favour of the "neuromorphic systems are analogue" crowd: the seminal paper by Carver Mead that (to my knowledge) coined the term "neuromorphics" specifically talks about analogue neuromorphic systems [2]. Right now, there are some research "analogue" (or, more precisely "mixed signal") neuromorphic systems being developed [3, 4]. It is correct however that there are no commercially available analogue systems that I am aware of. Unfortunately, the same can be said for digital neuromorphics as well (Intel Loihi is perhaps the closest to a commercial product, and yes, this is an asynchronous digital neuromorphic system). [1] https://iopscience.iop.org/article/10.1088/1741-2560/13/5/051001/pdf https://iopscience.iop.org/article/10.1088/1741-2560/13/5/05... [2] https://authors.library.caltech.edu/53090/1/00058356.pdf https://authors.library.caltech.edu/53090/1/00058356.pdf [3] https://brainscales.kip.uni-heidelberg.de/ https://brainscales.kip.uni-heidelberg.de/ [4] https://web.stanford.edu/group/brainsinsilicon/documents/ANeckar2019.pdf https://web.stanford.edu/group/brainsinsilicon/documents/ANe...

aidenn0 5y ago

Are analog computers actually more energy efficient than a digital ASIC with similar accuracy (say 6-10 bits)?

boxed 5y ago
Yes. By a lot.

Strilanc 5y ago

I'm a bit worried for that analog-matrix-multiplication-for-AI company. I vaguely remember reading somewhere that the past half century is littered with companies whose value proposition was "our specialized thing does 10x better than digital transistors" and then they predictably just got steamrolled by Moore's law. And although Dennard scaling ended two decades ago, the flops-per-watt and flops-per-second of AI-specialized chips like TPUs has been improving substantially recently [1][2]. 1: https://en.wikipedia.org/wiki/Tensor_Processing_Unit#Third_generation_TPU https://en.wikipedia.org/wiki/Tensor_Processing_Unit#Third_g... 2: https://arxiv.org/abs/2005.04305 https://arxiv.org/abs/2005.04305

uxp100 5y ago
Fuzzy logic was pretty closely tied to Neural Networks at one point (80s to early 90s?) in terms of which researchers talked about them and even just which books info about them was located in. I think dedicated analog fuzzy logic chips were, as you say, steamrolled by Moore's Law. (My zorojushi rice cooker still mentions fuzzy logic, but they must be just implementing the fuzzy "transfer function" with an mcu at this point, right?)
snakke 5y ago
You're correct that specialised analog companies have not done well historically. However, we don't find ourselves in exactly the same position in computer architecture/performance as we've been decades before. There's some (relatively) new ideas that now the performance of computers will be pushed more by dedicated silicon for a dedicated purpose, and tools. See for example there's plenty of room at the top [1], or Hennesy's talk at Google [2]. This of course does not mean that analog computers are suddenly viable, but it does mean that they could potentially fill a niche where they failed previously. Anecdotally, when looking at jobs for hardware design by the likes of Infineon, STM, Cyient etc. there seems to be a relatively high ask for (senior) analog designers, and a new focus on mixed-technology chips. It might turn out to be a dud still, but it isn't the same situation as decades before. 1: https://www.science.org/doi/abs/10.1126/science.aam9744 https://www.science.org/doi/abs/10.1126/science.aam9744 , or the IEEE article https://ieeexplore.ieee.org/abstract/document/7863324 https://ieeexplore.ieee.org/abstract/document/7863324 2: https://www.youtube.com/watch?v=Azt8Nc-mtKM https://www.youtube.com/watch?v=Azt8Nc-mtKM , around the 12:15 mark, but the entire video is relevant.

antattack 5y ago

If you think about it, Mythic analog AI accelerators will also develop neurological 'disease' as it's flash cells wear out.

jecel 5y ago
Since the chip is meant for inference but not for training, in theory you only write to the Flash once or twice. In this case data retention is a more limiting factor than wear. For digital storage the data can change quite a bit before a 1 becomes 0 or vice-versa, but for analog even a small change can be a problem. So I don't know if we can extrapolate the typical Flash memory data retention number to the expected aging for this device.

mastax 5y ago

This video discusses the history and principles of classical analog computing and then says that analog computers have energy efficiency benefits over digital computers. This can give the incorrect impression that classical analog computers are more efficient than digital computers, which is absolutely not correct. It may be true that modern analog phase-change memory AI accelerators are more efficient than their digital counterparts, but this assertion should be distanced from the discussion of classical analog computers to not give the wrong impression.

eternityforest 5y ago
People always love to say some old lost tech is better than the modern version But it's like, a one in a thousand shot.Almost all modern tech is better with regressions lasting a decade at most. Occasionally there are rumors of some modern medical treatments being worse even though medicine as a whole improves, but mechanical and electrical stuff is much better understood and more controlled. The exception is a few specific appliances that are seemingly designed to fail. Even then there's no lost tech. I'm pretty confident we could make a much better washing machine today for much less money if we tried, than anything from 30 or 50 years ago.
- autokad 5y ago
  The analog computers on the Missouri class battleships was never upgraded because the kill radius of the shells was larger than the margin of error the analog computers introduced and that the analog computers required less electricity which is good for something such as a ship. So in this way, yes, they are better (these computers developed in the 30s). It depends on the application. > Occasionally there are rumors of some modern medical treatments being worse even though medicine as a whole improves, but mechanical and electrical stuff is much better understood and more controlled. Such as the iron lung probably being better than the invasive machines we have today. > The exception is a few specific appliances that are seemingly designed to fail. Even then there's no lost tech. I'm pretty confident we could make a much better washing machine today for much less money if we tried, than anything from 30 or 50 years ago. look at the space industry, it actually costs us more (adjusting for inflation) to try and go back to the moon than when we did before when we had no idea on how to do it. Or look at tractors, farmers are clamoring for 1980s tractors over these new expensive (break down easy with no way to repair) 'modern' tractors John Deer is pushing.

jodrellblank 5y ago

In the intro to this video he mentions analogue computers predicting tides. His video before this one[1] goes into detail on that, and it was incredible. In the 1800s, Lord Kelvin spent years working on tidal rise and fall patterns, applying Fourier Transforms to break them down into ten individual sine waves, then combining those sine waves back together to predict future tides. And built analog computers to do all the involved integration, multiplying and summing. It's a part of the history of computing I'd never heard of, despite hearing quite a lot about the dawn of electro-mechanical computers in the early 1900s. [1] https://www.youtube.com/watch?v=IgF3OX8nT0w https://www.youtube.com/watch?v=IgF3OX8nT0w

ggm 5y ago
The machines (or later derivatives) is in the science museum. One variant uses cones, I imagine they provide for a huge amount of variance of the parameters of the fourier transform, another is pulleys, but changing out a pulley is a lot more work than moving where two cones are co-incident rubbing to transfer motion. Along with a meccano numerical analyser, and bits of babbage's original work

mrvenkman 5y ago

I have to say that the title is definitely “click baity” - I watched the video because I wondered what we were doing wrong. It is focused on how to improve AI when we run out of atom sized hardware. I don’t appreciate the title.

agumonkey 5y ago

i find funny the parallel between the uptake of reactive programming and analog computing (same kind of real time smooth coupling)

Animats 5y ago

That video has one of the best explanations of how and why deep learning works. The argument for analog is weaker. I've tried some stuff with analog computers, and at one time I was into op-amp circuits. The basic problems are noise and inflexibility. However, that may change as people develop ICs that are reconfigurable, like an FPGA. Flash memory cells as multipliers by a changeable but not dynamic value are a new idea. That only leads to analog ICs that do a pre-trained neural net, though. Running neural nets isn't that expensive. It's training deep neural nets that needs entire data centers. If they can figure out a way to do the whole backpropagation thing in analog, that would be impressive.

periheli0n 5y ago

Yes, analog computing can be more efficient and faster. But there are reasons why analog was eclipsed by digital. 1. Flexibility. Reprogramming digital computation is easy and quick. 2. Robustness. One can mass-produce devices that operate in the digital domain and sell them as working as spec'd. But operate them in the analog domain, and you will work with a million snowflakes that all operate slightly differently, giving different computation results. And when temperature changes, the result of your analog computation will inevitably change, too. You can work around that by adding more circuitry, and partly also on the algorithmic end but it will cost efficiency and precision. Matrix multiplication on an analog device is great, unless you want an exact and reproducible result.

R0b0t1 5y ago
He touches on this in his first video. The workaround is we are now getting into a regime of computing where exact reproducibility is not necessary. It will take a while though. It is still quite hard to remove noise for opamp circuits.
- periheli0n 5y ago
  > we are now getting into a regime of computing where exact reproducibility is not necessary. I think that is a myth. Predictable, reproducible and explainable outcomes are the holy grail of computing, in particular in AI. If stochasticity is desired, there are methods to inject it, with precise control of the level of stochasticity and the distribution. This level of control is absent in analog computing. Device mismatch introduces some randomness, but it cannot be controlled in practice. Instead of adapting the randomness to the algorithm, one ends up adapting the algorithm to the randomness. Working with analog computers is a fascinating academic exercise. For practical applications, I doubt we’ll see it compete with digital computation anytime soon.
  - R0b0t1 5y ago
    I said exact reproducibility. Every time you add numbers your neurons likely fire in slightly different ways, but the end result is what is reproducible and useful. He's not totally correct because, like I said, the way we think of analog computers right now is likely not going to get us to where we need to be. Noise is a huge problem. Our own neurons do not seem to use continuous voltage, for example, and use pulse density coding, at least for some of their operation.

mikewarot 5y ago

Let's use the LM358DT as an example of a modern op-amp. Just sitting there, it has current sources sinking 100uA of current. At 5 volts, it consumes about 0.7 mA with no load. So, compare that to a 4 bit ALU (an old one, at that) the 74HC181. At 5 volts it consumes 80uA at idle. Analog circuits require transistors have bias flows through them at all times to ensure linearity. CMOS digital logic has the transistors either fully on or off, except during switching times. Going analog isn't as great an option as this video makes it seem.

rapjr9 5y ago

Check out the work of Kofi Odame on implementing algorithms in analog circuitry on integrated circuits: https://engineering.dartmouth.edu/community/faculty/kofi-odame https://engineering.dartmouth.edu/community/faculty/kofi-oda... He's interested in creating analog circuits that are more efficient than digital ones for signal processing and machine learning. I've programmed an analog computer when I was working in the military-industrial complex long ago, implemented a standard missile guidance algorithm for an IR missile using a dual feedback loop and adjustable nav ratio. It was kind of like programming with wires to implement math. It had noise limitations on accuracy, in part because of the many wires exposed to florescent lights and digital computers in the room. Quite interesting. I think the Analog Thing might make a good EuroRack music making module! Turn math into music.

olliej 5y ago

I liked some of veritasium's older stuff but it seems to be getting more and more clickbait as time goes on (which is reasonable as it's clear that YT is his job, and YT promotes clickbait above everything else). A few people said that they "knew" it would be veritasium, but I had guessed it would be Mill computer folk :D

deleted 5y ago
[deleted]

charcircuit 5y ago

I feel like it was a bit misleading to mention large amount of energy to train a model and then show a chip which can only be used for inference and not training. In general people are optimizing for power use at the edge and not for training.

teleforce 5y ago

For a wonderful introduction to digital and analog computers please check the book by Steiglitz, "The Discrete Charm of the Machine" [1]. [1] https://leonardo.info/review/2019/07/review-of-the-discrete-charm-of-the-machine-why-the-world-became-digital https://leonardo.info/review/2019/07/review-of-the-discrete-...