Neural network

“ Each probe will have gravity instrumentation. Using the microwaves naturally occurring in space, they will be connected by a vast neural network and thus act as a single instrument, with all data being fed back to various information centers. ”

“ Progress in understanding and emulating human thinking shows a progression from a literacy-based model to a model rooted in the new pragmatic framework. Rule- based, pattern-matching systems generalize predicate calculus; neural networking is devoted to mimicking the way minds work, in a synthetic neuron-plex array; fuzzy logic addresses the limitations of Boolean calculus and the nondeterminism of neural networks, and concentrates on modeling imprecision, ambiguity, and undecidability as these are embodied in new human practical experiences. ”

“ Such interactive education centers are simultaneously libraries of knowledge, heuristic environments, laboratories, testing grounds, and research media. The hybrid human-machine machine that constitutes their nucleus alters as the individual involved in the interaction changes.
As we all know, the best way to learn is to teach. Students should be able to teach their neural network partners subjects of interest to their own practical experiences. ”

“ It is possible to go even a step beyond the integrated world of digital processing and to entrust extremely complicated processes to neural networks trained to perform functions of command, control, and evaluation. Unexpected situations can be turned into learning experiences. Where individuals sometimes fail-for instance under emotional stress-neural networks can easily perform as well as humans do, without the risks associated with the unpredictability of human behavior. ”

“ One such component which has evolved from bionics research is the Artron, or artificial neuron. Inside the Artron are logic gates and inhibit 137gates. By means of reward or punishment, the Artron learns to operate a “statistical switch” and send impulses to other Artrons or to a readout. There are two interesting parallels here besides the operation of a simulated neural net. One is the statistical approach to decisions and learning. The late John von Neumann theorized that the brain’s actions might be statistical, or based on probability. ”

“ While one neuron may not be as reliable as a vacuum tube or transistor, the complete brain is millions of times more dependable than any of its single parts. This happy end result is just the reverse of what man has come up with in his complex computer systems. For instance, individual parts in the Minuteman missile must have a reliability factor of 99.9993% so that the system will have a fair chance of working properly. Duplication of the brain’s network may well lead to electronic systems that are many times more reliable than any of their individual parts. ”

“ Here we show how one can go beyond linearizations to take into account nonlinear effects in neural winner-take-all models. In particular, we will show how models of neuronal dynamics in two-choice decision making tasks can be formally reduced to a one-dimensional nonlinear diffusion equation. Instead of focusing on a particular model system we consider a generic model of the neuronal dynamics with two key features: nonlinearity and competition. One obvious advantage of using such a general framework is that the extent of validity of the reduction is potentially very large. ”

“ One of the biggest roadblocks to implementation of a humanlike neural net is economic. Since there are some 10 billion 138neurons in the brain, and early electronic neurons consisted of several components including transistors which are a bargain at $2 each, building such a computer might double our national debt. Bionics workers have been thinking dreamily in terms of something like one cent per artificial neuron. This is a ridiculously low figure, but even at that a one-tenth brainpower computer with only a billion penny neurons would cost $10 million for those components alone! ”

“ It can be demonstrated that if a sufficiently large number of linear threshold elements is used, with the outputs of some being the inputs of others, then a final output can be produced which is any desired logical function of the inputs. The difficulty in such a network lies in the fact that we are no longer provided with a knowledge of the correct output for each element, but only for the final output. If the final output is incorrect there is no obvious way to determine which sets of weights should be altered. ”

“ Each element requires a bias, an error signal, and the desired final output, these things being uniform for all elements in a network. Some external device must manipulate the bias properly, but this is a simple behavior depending only on an error signal and the desired final output—not on the state of individual elements in the network. What one has, then, is a network consisting of elements which are nearly autonomous as regards their decisions to change weights. Such a scheme appears to be the only way to avoid constructing a central weight-change decision apparatus of great complexity. ”

“ Humans generally use physical and structural features in pattern recognition because they can easily be discerned by human eyes and other common means. However, human sensory organs are difficult to imitate with artificial devices so these methods are not always effective for machine recognition of objects. But by using carefully designed algorithms, machines can easily extract mathematical features of patterns which humans may have great difficulty in detecting. The algorithms will often involve a fusion of knowledge across multisensor data. ”

“ Experiments have been performed on simplifications of the model described so far which had the occasional result that a small number of elements came to a state where they received most of the weight increments, much to the detriment of the learning process. In such cases the network behaves as if it were composed of many fewer adjustable elements. In a sense this is asking each element to maintain a record of the data it is being asked to store so that it does not attempt to exceed its own information capacity. ”

“ Indeed, in a series of papers, Wang and co-workers have shown that biophysically motivated cortical network models of a two-choice decision making task can qualitatively replicate some salient aspects of both behavioral and neurophysiological data [24] – [25] . Such models posit two populations of recurrently coupled excitatory neurons each of which receives input proportional to the relative evidence in favor of the choice which it encodes. The populations compete through interneuron-mediated inhibition leading to winner-take-all behavior. ”

“ This can then be the signal for the weights on an element to change—a change of final output from right to wrong accompanied simultaneously by a zero sum in the element itself.
After such a weight change, the final output will be correct once more and the bias can again proceed to fall. Before it reaches zero, this process may occur a number of times throughout the network. When the bias finally stands at zero with the final output correct, the network is ready for the next input. Of course if -1 is desired, the bias will change in the opposite direction. ”

“ Having listed which seem to be the important properties of a neuron, it is possible to synthesize a simple model which has all of them. [Pg 73]
A number of input stimuli are fed to the neuron through a resistive summing network which establishes the threshold and accomplishes spatial integration. The voltage at the summing junction triggers a “one-shot” circuit, which, by its very nature, accomplishes pulse generation and exhibits temporal integration and a refractory period. ”

“ Therefore, once it superimposes a three dimensional matrix over holographic information it wishes to interpret and reduces that information mathmatically to two dimensional form, it can completely process it using its fundamental binary system just as any computer made by the hand of man can process volumes of data and make various comparisons between the data and information stored in its digital memory. ”

“ In putting forth a model for such an adapting or “learning” network, a requirement is laid down that the complexity of the adaption process in terms of interconnections among elements needed for producing appropriate weight changes, should not greatly exceed that already required to produce outputs from inputs with a static set of weights. In fact, it has been found possible to use the output-from-input computing capacity of the network to help choose proper weight changes by observing the effect on the output of a variety of possible weight changes. ”

“ Computational approaches that have been hugely popular and found important applications include computational modeling, Bayesian and graphical models, machine learning, deep-learning, pattern recognition, optimization, spectral and pseudospectral analysis, stochastic modelling, iterative system model adaptation, and multiscale multiphysics analysis to name a few. ”

“ Thus, a major technology driver is the development of an advanced machine intelligence system capable of reorganizing rejected hypotheses, integrating that information with data acquired through sensory apparatus, generating new hypotheses which coordinate all existing information, and, finally, testing these hypotheses in some systematic fashion. ”

“ There are two very compelling reasons why mathematical and physical models of the neuron should be built. Model building, while widely used in the physical sciences, has been largely neglected in biology. However, there can be little doubt that building neuron models will increase our understanding of the function of real neurons, if experience in the physical sciences is any guide. Secondly, neuron models are extremely interesting in their own right as new technological devices. ”

“ However, adequate theories of all three types of inference are a necessary foundation for successful machine learning systems. Both of these barriers to machine learning -the abductive inference barrier and the theory barrier -must be bridged before machine intelligence systems can be given a full learning capability. The abductive inference barrier has already been fully treated, but some additional discussion of the "theory barrier" is useful here. Historically, technology has developed in two distinct patterns -empirical and theoretical. Empirical technology is a "black box" approach. ”

“ We forget sometimes that the world is one giant computer, with thousands and millions of arms, just like the brain. The neurons," he pointed at his head, "are connected to each other with synapses. Just like a computer network."
The Flying Dutchman's explanation was a little less ethereal than the Spook's and Scott found himself anticipating further enlight- enment.
"The neuron is a computer. It can function independently, but because it's capacity is tiny, a neuron is really quite limited in what it can achieve alone. The synapse is like the network wire, or phone company wiring. ”

“ The simplest network would consist of a single linear threshold element with a set of weights, c0, c1, c2, ..., cn. These determine the [Pg 38] output-input relation or function so that y is +1 or -1 according as the quantity, c0 + c1x1 + c2x2 + ... + cnxn, is positive or not, respectively. It is possible for such a single element to exhibit an adaptive behavior as follows. If, for a given set, x1, x2, ..., xn, the output, y, is correct, then make no changes to the weights. ”

“ Since an investigation of abductive inferences seems to hold many keys to solving the problem of machine learning, and since recent developments in AI seem to promise receptivity to such an investigation, the development of a theory of abductive inference for machine intelligence appears to be the preferred research direction for work on machine learning systems. ”

“ John von Neumann, and a large number of other researchers in theoretical computer science following him, have shown that there are numerous alternative strategies by which a machine system can duplicate itself. There is a large repertoire of theoretical computer science results showing how machine systems may simulate, construct, inspect, and repair machine systems including, to some extent, themselves. ”

“ Second, the designers of Artron see a similarity in its operation and Darwin’s theory of natural selection.
Another new component in the bionics approach is the “neuristor.” This semiconductor diode simulates the axon, the nerve fiber that connects with the neuron. Another device is the “memistor,” unique in that it uses electrochemical phenomena to function as a memory unit. A different kind of artificial neuron called MIND is made up of magnetic cores.
There is another plus factor in this duplication of what we think is the system used by the brain. ”

“ In summary, it appears that a rather simple neuron model can be built which can mimic most of the important functions of real neurons. A system built of these could be punished or rewarded by an observer, so that it could be trained to give specified responses to specified stimuli. In some cases, the observer could be simply the environment, so that the system would learn directly from experience, and would be therefore a self-organizing system. ”

“ A machine can send destruction signals into cells to reduce them to the quiescent form. Unfortunately, in some cases one must know the state of the cell ahead of time in order to determine what destructive signal must be sent to destroy it.
Finally, all machines that can be produced in von Neumann's cell space system are essentially information transactional devices. Even construction is, in this context, a form of information processing. ”