For some types of system, searching through all possible rules for the behaviour that you want isn't out of the question, and can sometimes lead to finding a so

For some types of system, searching through all possible rules for the behaviour that you want isn't out of the question, and can sometimes lead to finding a solution faster than wandering around in parameter-space (either along a directed path or randomly). A recommendation of this 'search-based' approach (where it can be used) instead of always constructing solutions is one of the 'new' aspects of Wolfram's "New Kind of Science". I believe that this ability to tolerate searching for a solution is a component of what can allow neural-nets to find solutions better than programmers would ever write. A NN will happily explore nonsensical or illogical bits of parameter-space where there are still some good solutions to be found in places where the 'rational' or 'logical' human mind wouldn't (or even couldn't?) go.