Really liked the post except I'm not sure about the hilited bit. A brain or maybe a neural network is not mathematically functional in the same way a set of logic gates are I think. At least I remember reading that the state of and result that a neural network creates based on its inputs is not entirely predictable?
Just a small derail. But interesting point in your post.
You're correct. Neurons don't actually work like logic gates do. So what I was doing was looking for what they do have in common, along with the basic building block units of all other systems that have been demonstrated to be computationally universal (capable of any computation a digital computer, or more technically a Universal Turing Machine, can perform).
That simplest common aspect appears to be nonlinear behavior; a small change in input can cause no change in output, or a large change, depending on the states of other inputs. If instead, all neurons or logic gates could do was output the average of all their inputs, they wouldn't be nearly as useful.
Linear systems can perform a lot of computations. The lens of the eye (or of a camera), for instance, sorts incoming photons, no matter where on the surface of the lens they happen to strike, based on the direction they came from. That's how a focused image is created. But as far as I know, lenses or collections of lenses aren't capable of universal computation. You couldn't make a collection of lenses run Minecraft*, for instance, no matter how many lenses you used in how complex an arrangement. To do that you'd have to include elements that were nonlinear, such as optical switches.
As for the outcome of such a system often being unpredictable, that's true of all universal computing systems, and it's an important clue (though not proof) that a system might be computationally universal. You might look at the extremely simple rules of, for instance, Wolfram's Rule 110, and think that it shouldn't be hard to predict how a given starting state would behave. But because Rule 110 is computationally universal, running it with some particular starting state and answering some specific question about its future behavior (such as whether or not it will eventually start repeating itself) might turn out to be equivalent to computing whether a certain 10,000-digit number is prime. The only known way to learn what the system will do is to run it, or run some other system that simulates its operation step by step.
(Of course, that doesn't mean all possible starting states are unpredictable, even if the system is potentially universal. For instance, if you run Rule 110 with a starting state of all "white" cells, nothing will ever happen. And we try pretty hard to design our computer programs so that their behavior is predictable.)
*I use that example because Minecraft itself is computationally universal. You can build structures within the rules of Minecraft that are equivalent to Turing machines or other computationally universal devices. So, any system that can run Minecraft must itself be computationally universal as well.
My darwiness!
