4 ms·
From the first chapter of this book, it is described that each layer adds more abstraction. The first layer (input) works on individual pixels, the next layer o
by _ikke_ 13y ago
From the first chapter of this book, it is described that each layer adds more abstraction. The first layer (input) works on individual pixels, the next layer on a part of the image. Each deeper layer can make more high-level decissions.
- j2kun 13y agoI understand why more layers adds complexity, but that doesn't answer my question: why layers instead of an arbitrary acyclic graph?
- jbooth 13y agoI'm not an expert, but my guess is because it makes it really easy to materialize the weights between layers as a 2d matrix and we have some really good library code out there for dealing with matrices. There's probably room for experimenting with more novel constructions and even some papers out there looking into the matter.
- j2kun 13y agoEfficiency is the expected answer. I'm just wondering if there's a more theoretical reason, such as "every function that can be computed by a non-layered acyclic network can be computed by a complete layered network using only a small number of extra nodes/layers."
- jbooth 13y agoI think that it can. With some weights of 0 and some weights of 1, you can trivially map 'jumps' that skip from a node in one layer to a node a couple layers distant, by means of some incorporate-no-other-inputs intermediate nodes, right? Sigmoid function on 1 is still 1? Once you have those, it's just a matter of how many layers you need for any acyclic structure, I think. Although if you wanted to come up with difficult scenarios, it's not hard to think of structures that would make some of those middle layers really tall, or add a lot of middle layers.
- j2kun 13y agoAs I mentioned in another branch of this thread, selectively choosing edges between nodes isn't an option, because in the standard model you have complete incidence between nodes in adjacent layers.