3 ms·

Ask HN:Artificial Neural Networks for electronics engineers

8 points by rick_2047 17y ago

Hi all, I am a First year electronics engineering student.I am aiming at a scholarship which requires me to make a project, anything original and research based. I am still searching for ideas, and one that came into my mind was artificial neural networks in electronics. I thought it was perfect (mind you I have a very limited knowledge of ANN, so feel free to correct me).Neural networks are mathematical models of brain and if I could implement such a thing,even on a very very small scale, into simple circuits it would definitely result in a very impressive research project. The immediate second thought was that this cannot be original. There has to be some work done on it as this is such an obvious thing. I found out about BEAM robotics and Nv Nets, but as this is still in the research phase there is not much material on it to read. I couldn't find books,post,papers anything which will make me understand Neural Networks from the perspective of an electronics engineer. But I think this could be the mistake of my not-so-good searching skills. Can anyone please point me to such a resource?

jacquesm 17y ago
There are chips that contain neural networks: http://www.particle.kth.se/~lindsey/HardwareNNWCourse/ http://www.particle.kth.se/~lindsey/HardwareNNWCourse/ That's fairly dated, lots more to read: http://www.google.com/search?q=neural+networks+in+hardware http://www.google.com/search?q=neural+networks+in+hardware The Japanese have done tons of research in this field and the producers of quite a few of the chips, there are lots of fields of applications for these, one of them is in process control.
joshu 17y ago
It's not model of a brain, it's a model of a bunch of connected neurons. I think it's probably easy to emulate a neural network in an embedded CPU. That seems somewhat boring. It might be more fun to build a bunch of discrete neurons out of analog parts. Building a forward-propagation-only (perceptron) system should be pretty easy. Might be hard to actually train them.
jjguy 17y ago
I spent some time studying Self-Organizing Maps, a neural network algorithm from Teuvo Kohonen, a Finnish professor. The internet has plenty of resources: http://www.google.com/search?q=self+organizing+maps http://www.google.com/search?q=self+organizing+maps I implemented the two canonical 'hello world' projects in processing, but have yet to apply to a 'real' problem. The emergent behavior is fascinating, as is watching maps coalesce in real-time. There are plenty of other algorithms available; a basic "back propagation" algorithm is widely used in OCR algorithms. Try reading some of these: http://www.google.com/search?q=neural+networks+image+recognition http://www.google.com/search?q=neural+networks+image+recogni... Don't get too wrapped around the axle of trying to find immediate electronics engineering applications, but understand the algorithms and the applications will become self-evident.
paraschopra 17y ago
All neural networks do is to aggregate input values (which you can model with current) and multiply it by weights (which you can model by variable resistors) and finally output a squished sigmodial output (for which I am sure a circuit must exist). Make elementary units of such neurons and connect them together. Then train the network by manually changing resistors to emulate backpropagation algorithm. Though why you want to do all this through circuits (instead of a computer) is beyond me.
- rick_2047 17y ago
  Because I want to show that these things can be done without a computational power. So my work can be further expanded to give IC's. Also it would be cool to have just circuits who have emergent behavior.
  - paraschopra 17y ago
    You are doing computation here. Moreover, it is not an emergent behavior, you are training your ANN here.
    - rick_2047 17y ago
      Sorry for not making myself clear, by computation I meant just micro controllers and microprocessors. And I was wrong in stating that in such a way because computation is such a big umbrella. Also the whole purpose of the project will be to make a customized circuit which could be faster than an embedded system.
rick_2047 17y ago
All this is quite good, and I was right in thinking it was just my not-so-good searching skills that did me in. But there are two big questions which I mean to ask 1)Are these projects too big and complicated for a first year student?I can put in as much hardwork as is required but sometimes somethings are beyond a persons comprehensions. 2)What should be the approach to learning the skills required to tackle such problems, considering a student with no previous knowledge of ANNs but some good maths and electronics skills?
transmit101 17y ago
This is slightly more centred on evolutionary electronics, but nevertheless you might find some interesting papers here: http://www.cogs.susx.ac.uk/users/adrianth/ade.html http://www.cogs.susx.ac.uk/users/adrianth/ade.html
JohnIdol 17y ago
If you wanna implement logic operations using neural networks (you'll need 1 neuron for most of them - but XOR and NAND will require a few more and some tricks) and combine them to do some non-trivial computation - which is simple but good enough for a first year project - have a look at the first 3 chapters of this free book for an overview of this topic and have a look at the last chapter focusing on hardware for neural networks implementations (which will come in handy if we're not talking about software implementation) --> http://page.mi.fu-berlin.de/rojas/neural/ http://page.mi.fu-berlin.de/rojas/neural/

8 points by rick_2047 17y ago

jacquesm 17y ago

There are chips that contain neural networks: http://www.particle.kth.se/~lindsey/HardwareNNWCourse/ http://www.particle.kth.se/~lindsey/HardwareNNWCourse/ That's fairly dated, lots more to read: http://www.google.com/search?q=neural+networks+in+hardware http://www.google.com/search?q=neural+networks+in+hardware The Japanese have done tons of research in this field and the producers of quite a few of the chips, there are lots of fields of applications for these, one of them is in process control.

joshu 17y ago

It's not model of a brain, it's a model of a bunch of connected neurons. I think it's probably easy to emulate a neural network in an embedded CPU. That seems somewhat boring. It might be more fun to build a bunch of discrete neurons out of analog parts. Building a forward-propagation-only (perceptron) system should be pretty easy. Might be hard to actually train them.

jjguy 17y ago

I spent some time studying Self-Organizing Maps, a neural network algorithm from Teuvo Kohonen, a Finnish professor. The internet has plenty of resources: http://www.google.com/search?q=self+organizing+maps http://www.google.com/search?q=self+organizing+maps I implemented the two canonical 'hello world' projects in processing, but have yet to apply to a 'real' problem. The emergent behavior is fascinating, as is watching maps coalesce in real-time. There are plenty of other algorithms available; a basic "back propagation" algorithm is widely used in OCR algorithms. Try reading some of these: http://www.google.com/search?q=neural+networks+image+recognition http://www.google.com/search?q=neural+networks+image+recogni... Don't get too wrapped around the axle of trying to find immediate electronics engineering applications, but understand the algorithms and the applications will become self-evident.

paraschopra 17y ago

All neural networks do is to aggregate input values (which you can model with current) and multiply it by weights (which you can model by variable resistors) and finally output a squished sigmodial output (for which I am sure a circuit must exist). Make elementary units of such neurons and connect them together. Then train the network by manually changing resistors to emulate backpropagation algorithm. Though why you want to do all this through circuits (instead of a computer) is beyond me.

rick_2047 17y ago
Because I want to show that these things can be done without a computational power. So my work can be further expanded to give IC's. Also it would be cool to have just circuits who have emergent behavior.
- paraschopra 17y ago
  You are doing computation here. Moreover, it is not an emergent behavior, you are training your ANN here.
  - rick_2047 17y ago
    Sorry for not making myself clear, by computation I meant just micro controllers and microprocessors. And I was wrong in stating that in such a way because computation is such a big umbrella. Also the whole purpose of the project will be to make a customized circuit which could be faster than an embedded system.

rick_2047 17y ago

All this is quite good, and I was right in thinking it was just my not-so-good searching skills that did me in. But there are two big questions which I mean to ask 1)Are these projects too big and complicated for a first year student?I can put in as much hardwork as is required but sometimes somethings are beyond a persons comprehensions. 2)What should be the approach to learning the skills required to tackle such problems, considering a student with no previous knowledge of ANNs but some good maths and electronics skills?

transmit101 17y ago

This is slightly more centred on evolutionary electronics, but nevertheless you might find some interesting papers here: http://www.cogs.susx.ac.uk/users/adrianth/ade.html http://www.cogs.susx.ac.uk/users/adrianth/ade.html

JohnIdol 17y ago

If you wanna implement logic operations using neural networks (you'll need 1 neuron for most of them - but XOR and NAND will require a few more and some tricks) and combine them to do some non-trivial computation - which is simple but good enough for a first year project - have a look at the first 3 chapters of this free book for an overview of this topic and have a look at the last chapter focusing on hardware for neural networks implementations (which will come in handy if we're not talking about software implementation) --> http://page.mi.fu-berlin.de/rojas/neural/ http://page.mi.fu-berlin.de/rojas/neural/