4 ms·
I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for cl
by theamk 6y ago
I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs.
But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function.
Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices.
- waste_monk 6y ago>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.
- amelius 6y agoTrue, but they're hard to do experiments with, which is useful if you're building such a brain.
- jokethrowaway 6y agoIf the trade-off between energy-cheap human-like processing and our computers is reliability, it's definitely worth doing. If we had unlimited human like processing, we would basically have unlimited slaves without affecting biological organism similar to us. Not to mention, it's worth doing it just for having 100000 times cheaper machine learning.
- birdyrooster 6y agoOh boy I can’t wait for the unlimited slaves
- MichaelZuo 6y agoWell bounded by the expanding co-moving light cone of mass-energy, barring FTL travel.
- ddingus 6y agoDon't worry, we will take short cuts, because greed and ignorance, leaving them with agency we will not have accounted for. Should be exciting.
- JoeAltmaier 6y agoIs the better alternative, to create actual slaves? Intelligent feeling beings that we put governors on, like a Thrall shock-collar from Star Trek, so they don't get to uppity? Because that's a horror show of its own right there.
- ddingus 6y ago[Goes with morbid flow here] Perhaps! If we get it right, and they are in fact beings, there may not be better alternatives. May always end in conflict. I say that because a core aspect of beings, is that they "be", and that boils down to agency of some kind. Negative reinforcement runs in conflict to all that and warrants a response, from the oppressed beings point of view. Us being creators will only go so far. John Varley proposed an interesting solution to this in his Gaea series. A slave was created by connecting all their pleasure centers to an area on their forehead. Any touch other than their own triggered pleasure so intense it is worth pretty much anything to experience.
- JoeAltmaier 6y agoOk that's not a shock collar, but creepily similar to keeping folks drugged into compliance. Also an ethical hole. I'm not sure what the solution is. Intelligent but not self-aware?
- avs733 6y agoNot really... Psychology does this all the time. Outcomes can be described as distributions as much as they can binaries. It just takes a change in mindset.
- MauranKilom 6y agoHuman brains are also ridiculously expensive to copy and distribute. Not to mention that you need an extremely loose definition of "copy".
- danw1979 6y agoMe and the wife made a couple and whilst they aren't cheap, I wouldn't say they're outrageously costly to make/run. Not particularly reliable though... and yes, we seem to be having great difficulty copying the required information into them.
- avs733 6y agoI'm in the first trimester of the manufacturing process, excited but also having some sticker shock at day care options. Never built something so initially unstable that ends up Turing complete before.
- Datenstrom 6y ago> Not to mention that you need an extremely loose definition of "copy". Makes me wonder, with a nearly perfect simulation for a reinforcement learning agent and a human brain or equivalent architecture, would you be able to make exact copies. Assuming the PRNGs and other settings were deterministic. Now that would be an interesting and controversial experiment.
- Veedrac 6y agoHuman brains try really hard not to be analogue.
- ddingus 6y agoMy thoughts exactly. People want what is in our heads. Which is it? Some idealized, and by nature limited sub set or idea, or the real deal?
- tartoran 6y ago> Human brains have the same problems and seem to be fairly popular. Not only that but I'm sure there's a usecase where they are more suitable, we just haven't found it.
- nelsonmandela 6y agoYou might be interested in 'Robust First Computing' - https://www.youtube.com/watch?v=6CNg1Q3RNWI https://www.youtube.com/watch?v=6CNg1Q3RNWI edit: here is a far better explanation by the terms creator:https://www.youtube.com/watch?v=7hwO8Q_TyCA https://www.youtube.com/watch?v=7hwO8Q_TyCA
- rkagerer 6y agoI agree with all your points. Perhaps the opportunity is if your neural networks are tolerant to that kind of variance there may be applications where you can shrink the hardware footprint (and cost) dramatically while improving latencies (and maybe power usage).
- svantana 6y ago> Would people want a chip which is basically unpredictable Judging by the prevalence of (pseudo-) random numbers in machine learning, I'd say yes. Reentrancy is a big plus, but not always a dealbreaker. It's possible we might end up with a sort of left/right-brain setup, with a noisy analog hypothesis generator paired with a robust, logic-based evaluator/planner.
- eru 6y agoThose pseudo-random numbers are still reproducable.
- svantana 6y agoSorry, I meant reproducible, not reentrant.
- tartoran 6y ago> It's possible we might end up with a sort of left/right-brain setup, with a noisy analog hypothesis generator paired with a robust, logic-based evaluator/planner. That sounds very interesting to explore, definitely would teach us something new.