Thinking Solutions Detail

It is usually taken for granted that the brain is automatically segmented into specialized processing areas whose operating principles are beyond our understanding. Indeed, even the lowly reptile brain’s processing mechanism is currently too complex to explain. By taking an alternative view aligned with work from neural network (brain) research, we hope to show that the paradigm of storing, matching and using patterns is more useful to describe the operation of the brain. In doing so, a number of observations made by neuroscientists seem to make sense, such as the deficits found with brain-damaged patients and brain activity information seen through medical imaging. Treating the brain as a machine that is built on a simple principle may focus AI progress: it appears that the brain is a modular, bidirectional and hierarchical pattern-matching machine providing the framework needed for learning systems. The brain has no homunculus: each part of the brain operates based on its received inputs. If this is correct, it means that robotic brain builders can possibly build more effective applications, neuroscientists have more chance to treat damage and integrate brains with machines and linguists have another way to address their core doctrine.

^ back to top ^

Stored patterns are preferable to processing

AI and computer science have been closely linked for as long as computers have been available. Since that time countless problems have been solved, but a number of seemingly simple problems have remained unsolved and without effective robotic emulation. Examples of these unsolved problems include the emulation of human language, generic robotic movement and knowledge representation.

According to Ockham’s Razor, scientists ought to seek the simplest explanation that accurately models observations. Can we simplify the human brain into just a few principles? If so, the problems facing AI researchers will reduce significantly. On this basis and assuming that the linkage between current computer knowledge and AI is one of the main inhibiting factors, we pose these questions: What would a robotic brain design look like if we had no knowledge of computers? Are there better models of the brain than those based on processing?

This paper explores the answers to these questions by looking at our only working models of intelligence – those produced by the brain. The term brain will apply to our ancestor’s brains as well as our own because many required AI functions are present in our common ancestors who were similar to mammals, birds, reptiles and fish. Human brains will be specified only when relating to the unique capabilities. Once we replicate the capabilities of earlier brains, we have likely solved all the basic problems that apply to the human brain and possibly the difficult ones as well. The extensions that provide our linguistic capability may be just further applications of existing brain principles. There is no homunculus in our head controlling us: our brain is on its own. Indeed, many calls by researchers for brain modules to plan, process, decide or construct are perhaps just calls for anthropomorphic relief.

Our conclusion is that the use of stored patterns rather than the execution of algorithms aligns more closely with the brain’s model. A number of examples are presented to validate this approach and persuade you to consider alternatives to currently accepted beliefs. While the alternative approach may ultimately prove to be incorrect, it provides an explanation in an area currently without competition.

^ back to top ^

Proposed model overview

I propose that the brain is a machine comprised of a number of simple building blocks, each of which provides pattern storage. Studying stored patterns has a number of advantages over the creation of algorithms because the starting premise – stored patterns – caters to a building block model. Further, learning becomes the task of acquiring new patterns. The struggle by AI researchers to create appropriate algorithms also highlights the difficulty in doing something we believe the brain doesn’t do.

These building blocks are indivisible. That is, the building blocks are the atoms of our brain, being the smallest units that store, match and use patterns. We call these pattern matching atoms, Patoms. Patoms are connected in a hierarchy both forwards and backwards (bidirectionally). In the world of neuroscience, localized areas are referenced by a historical naming scheme. For example, V1 is the primary visual area, the amygdala is an area in the limbic system and M1 is the primary motor area. For our purposes here, the term Patom will reference any localized neural network performing a specific role. Patomic theory, the application of Patoms, proposes that the brain is comprised of a number of Patoms that only store, match and use patterns.

The topology of the brain is shown in Figure 1. The brain starts with sensors connecting to a Patom. These combine into single sense Patoms, storing patterns based on continuity. These in turn connect to multi-sensory Patoms (layer 3). These multi-sensory Patoms connect back to motor Patoms which connect to muscles and other organs. The description of the brain above can be reversed as the brain is bi-directional: each Patom connects back to the one that projects onto it. Architectural options allow for any number of variations to this general hierarchy.

Stored brain patterns are either snapshots or sequences of snapshots. Snapshot patterns are like digital photos, collections of active and inactive neurons with sufficient detail to signal a match or to activate a set of muscles. Layer 1 patterns reflect the sensor patterns presented, while layer 2 and above need only link via a unique signal. The signal needs to uniquely determine upstream and downstream (reverse) patterns, but need not be similar to the layer 1 pattern itself. Sequential patterns are sequences of snapshots or other sequential patterns. The brain learns by linking patterns, starting from the body’s sensors. Linkset patterns are weighted collections of connected patterns. Linkset intersection combines two or more linksets to determine common elements, a method that finds the best fully-consistent fit – a crucial step in decision making.

linkset representation

Figure 1: Sensors and muscles connect to Patoms. Patoms operate in a hierarchical, bi-directional manner.

^ back to top ^

Are brains running patterns, not programs?

The impacts of human brain damage are revealing. The loss of brain tissue leads to consistent deficits. What was the brain tissue doing prior to its loss? Is it the loss of processing expertise or something else? I argue that the simpler explanation is that the lost brain tissue removes either stored patterns or links between stored patterns.

For example, an often discussed Patom, the amygdala, is located in the limbic system (Goleman, 1995). We know that this Patom assists in the creation of the emotion we call fear. Many say that the amygdala is “responsible” for processing the fear response by considering the potential impacts sent as encoded signals from visual and other brain senses. Another explanation is that the amygdala stores a library of snapshots and sequential patterns necessary to produce the fear reaction by signaling to other areas of the brain. When appropriate signals are received by the amygdala, it initiates the response using a stored pattern. The fear response is well documented: one element is the easily recognized contraction of a set of facial muscles. Another releases chemicals into the blood stream to increase our muscle’s readiness and yet another to increase our heart rate. If the right signal is received, the Patom initiates its appropriate stored sequential pattern. The amygdala may just be accepting pre-determined inputs from other areas that trigger the release of one of the stored pattern sequences. There is no intelligence in it! Indeed, as the brain has no controller, we should avoid calling on one.

What is the processing involved between the experiences “see friend, feel love” and “see boss, feel angry”? It makes sense to think that the linkset for friend and the linkset for boss connect to separate emotional linksets, rather than processing alternatives, just as associative neural networks propose. Along these lines, psychologists are being increasingly successful in treating people with memories that need altering. Often associations with experiences that include emotional reactions, like fear responses, are undesirable. The process to change memory varies, but involves a few common factors articulated by supporters of neuro-linguistic programming, for instance. The patient is helped to access a beneficial emotion by moving and acting appropriately, activating the emotional linkset due to the brain’s bidirectional structure. The stored memory is then simultaneously activated resulting in a change to the linkset. The initial memory is altered when simultaneous activation (linkset intersection) swaps the connection to a beneficial emotion from the disempowering one.

^ back to top ^

Knowledge representation

Symbols are a key aspect of a brain theory. Patomic theory’s symbols are linksets: collections of stored sensor patterns linked through experience based primarily on continuity of experience. While a processing-based symbol is usually a single object, a linkset can comprise a large number of separate instances of an individual object. For example, thousands of layer 1 patterns may link to a layer 2 pattern which becomes the symbol. To match a sensor pattern, matched layer 2 patterns link back to layer 1. Layer 1 in response confirms any true fit. This results in the world conforming to what “has been experienced before”. It also allows for the brain to recognize patterns without the need to process idealized, mathematical models. Implementation options are numerous.

How can a wild orangutan sitting in a tree be recognized by the brain, for example? To recognize an object in a complex setting, the layer 1 Patoms identify the best potential matches which link to layer 2. Layer 2 sends back its matches catering to layer 1 confirmation. Patterns are rapidly matched at layer 2 because received signals aren’t arbitrary; they follow similar paths to the initially stored experience.

Figure 2 shows a simplified example of stored language patterns. Each set shows examples of linksets comprising multiple symbol instances. Label c shows a subset of auditory sequential patterns providing equivalence to grammar, built up through experience. Of course if the sequential patterns overlap because a symbol has multiple potential next steps, as is seen by word sequences for “I want” and You want”, the observation of novelty (new sentences that were never experienced) becomes a feature of the design. To overcome this, a pattern must inhibit parts of the existing linksets. Inhibition is a known feature of neurons.

Another feature of pattern sequences is the property of inheritance. By linking a new pattern to an existing one, downstream patterns are automatically connected for the same reason as novelty, inheriting its characteristics and requiring inhibition if aspects are to be excluded.

Diagram of a link set

Figure 2: Visual and auditory patterns are integral to human language. Labels a-e show separate linkset examples in different Patoms.

Philosophically, experiences trace back to sensory input or impressions (Hume, 1739). Sensor knowledge is stored in layer 1 in the Patomic model – the only layer that directly reflects received patterns. This means that the loss of sensory Patoms “loses that sense’s representation” – an observation made with certain types of brain damage. The bidirectional nature of the brain allows for patterns to be stored only once – layer 1 patterns need be the only area to hold sensor patterns (knowledge representation in the form of the initially captured sensor patterns). Higher layer patterns access these through reverse linkage. This applies to other sequential patterns such as language: these sequences appear to be located in Broca’s Patom whether used for sign or spoken language (Hickok et al, 2002).

Brain redundancy relates to pattern storage. On a computer screen, you can still see the picture even if a few pixels are broken. Even with a number of missing neurons the brain still retains the ability to recognize the best-fit pattern and being bidirectional, knowledge stored at the source is compared for consistency. By contrast, computer database models use keys to associate separate records. That method becomes expensive as more associations are needed and it is fragile in that lost or damaged keys can be catastrophic.

^ back to top ^

What is a brain?

Let’s try to ensure a mutual understanding of the brain by reviewing its function. Our knowledge of the brain has grown exponentially in recent years and is worthy of summary because that neural network research may hold solutions for other AI research.

A brain can be said to be an organ required by an “organism that moves from place to place” (Greenfield, 1996). The validity of this observation is reinforced by the sea squirt Ascidian that absorbs its own brain later in life when it attaches itself permanently to a rock. In evolutionary terms, it is reasonable to assume that the brain initiates appropriate movement to increase survival chances. In this context, the brain primarily recognizes patterns from sensor experience and uses stored patterns for motion. It appears to be more a pattern recognition machine than a pattern recalling machine.

^ back to top ^

Brain building blocks are similar

A brain, and I include all the body’s neurons in this definition, is a machine that takes as input a myriad of sensor signals and produces output in the form of muscle contractions and chemical release. All the elements of the nervous system are included in the definition of a brain as each influences the control of an animal. The brain is not just the macroscopic organ located within the skull. Therefore AI replication of the brain must replicate the entire nervous system.

Early in life damage to a Patom normally used for one specialty can be replaced by another area because the localized areas have a degree of plasticity. The brain is significantly changed in the early years of life with a massive erosion of neurons and their connections following an initial over supply. Later in life the brain has far less plasticity. The trigger for a neuron is the receipt of sufficient neurotransmitter chemical and there are different chemicals needed by different neurons, although the principle is consistent.

The topology of Patoms determines their function – they operate the same regardless of their connections. Patoms receive either signals from other Patoms or direct input from sensors. The signals are then used by connecting Patoms to produce motion via the body’s muscles. Muscles must work in conjunction with other muscles to effect smooth motion. For an animal to run, for example, its brain must control sequences of muscle contractions in harmony with sequences of muscle relaxations. While it could be said that parallel processing is taking place, it is possibly more accurate to say that parallel signaling is taking place. The creation of appropriate signaling could be a processing output of algorithms, but another way to look at this signaling control is the use of a sequence of stored patterns that activate and deactivate muscles as needed. Indeed, AI researchers ought to avoid rolling up (compressing) a body’s sensor patterns into single senses (artificial groupings of sensor combinations) to gain the benefits of additional stored patterns.

^ back to top ^

Brains have a localized bidirectional structure

An animal’s brain is comprised of Patoms connecting forwards and backwards in a hierarchical topology as discussed in the following passage by Damasio (1995):

… each collection of early sensory areas must talk first to a variety of interposed regions, which talk to regions farther away, and so forth. The talking is carried out by forward-projecting axons, or feedforward projections, which converge to other regions.

It may seem that these multiple, parallel, converging streams terminate at some apex points, such as the cortex nearest to the hippocampus (the entorhinal cortex), or some sectors of the prefrontal cortex… But this is not quite accurate. For one thing, they never “terminate” as such, because, from the vicinity of each point to which they project forward, there is a reciprocal projection backward. It is appropriate to say that signals in the stream move both forward and backward.

Why do Patoms connect bi-directionally? It allows for the connection of complex patterns back to their constituent sensory patterns and vice versa. It allows for sensor patterns and sequences to be stored only once in the network!

^ back to top ^

Brains are slow

It is well documented that the materials that drive a brain are very slow. A typically human reaction occurs within 200 ms (McCrone, 1999) and a neuron transmits every 4 ms. This suggests that roughly 100 neurons can be involved in sequence to produce the reaction (Waltz, 1999). Either the parallel processing is very clever or another technique is operating. Parallel pattern matching explains the ability to find patterns quickly without the need to postulate the existence of efficient parallel algorithms.

For programmers, the idea of writing 100 lines of code to solve a general problem is problematic. It becomes more challenging when we also consider that the brain compartmentalizes its functions requiring a number of steps to be dedicated to communications between Patoms. Allowing for some reverse pattern hits, Patoms must limit reverse matches to get a result in 200 ms. Known Patoms include the senses like vision, object recognition, auditory memory, and word sequence production. Each increases the number of links (and delays) necessary for a reaction.

The brain’s elements do operate in parallel, but are they processing information, or linking forwards and backwards to identify stored patterns before using them? While it is true that parallel processing caters to the need to rapidly create answers to problems, directly linking to stored answers is even quicker and simpler.

The brain makes up for slowness provided it can rapidly identify an appropriate stored response in around 100 sequential steps/neural signals, sufficient time for roughly 25 Patoms to work in series provided the stored pattern match in each Patom is selected immediately. As this flow of signals is between directly linked stored patterns, results are rapid.

^ back to top ^

Brain areas operate independently

Split-brain research from the 1960s provides further evidence of brain function. Under controlled conditions, Gazzaniga’s report (2002) indicates that a brain takes action based on its available information, even in cases where the language part of the brain is not involved. Received visual sensor signals are sufficient for a person to take action. In the research, a patient saw one object through the part of the brain involved in speech, while the other part of the brain received a different message. In line with the use of linksets, the patient reacted to messages independently of language suggesting that the brain is not operating on a linguistic basis.

^ back to top ^

Brains learn

Learning takes place in phases. We learn aspects of language before others. We walk before we run. We learn how to move our fingers before we learn to play the piano. We learn the individual dance steps before we combine those stored patterns in sequence. Each of these is a consequence of lower layer patterns being stored before they are available to create aspects of higher layer patterns. Until a match is made at layer 1, its connection to layer 2 isn’t possible. This also follows for layer 3 and 4. In the case of motion, the layer 1 sequential motor pattern is stored first. Once stored, a layer 2 motor pattern sequence caters to more complex motion. In other words, until the brain stores the motions in Patoms linking to muscles, it cannot control groups of such patterns and execute them in sequence.

Learning is about the creation and maintenance of linksets. To learn about an orangutan, for example, sensors store patterns in layer 1 and over time progress through higher layers. The features of an orangutan will probably include: (a) single sensor orangutan shape patterns, like those from an eye’s 120 million or so rods. Concurrent visual motion patterns provide additional pattern isolation, limiting the shape patterns to a subset of the visual field. Over time and based on continuity, a collection of sensor patterns will comprise the visual linkset of perhaps thousands of different orangutan images facilitating future recognition. (b) Concurrent experience with the visual linkset connects the layer 2 sounds of the orangutan in layer 3. (c) Layer 4 patterns connect the layer 3 orangutan with other aspects of the environment. There is no intelligence around how patterns are linked although repetition strengthens associations.

The brain’s modular structure results in some peculiar anomalies. Examples of layer 4 patterns are episodic memories, which involve the hippocampus. In severe cases of hippocampus loss, new skills can be learned despite the denial by the patient of prior experience. This is consistent with the brain successfully storing layer 1-3 patterns through experience (learned skills) but without storing layer 4 patterns (memory of event).

^ back to top ^

Why look for new brain models?

Frankly, the processing brain model doesn’t work well and its doctrine has infiltrated many aspects of AI research.

Experts readily agree that while processing engines are rapidly approaching the power of the human brain because computational power is quadrupling every year as described by Moore’s Law (Kurzweil, 2003), there is little to suggest that software will be ready. The hardware appears willing, but the software isn’t.

If we didn’t have the computer to help us decide how to produce actions and large individual motors to produce robotic motion, AI researchers would probably build models based on stored pattern use because it is conceptually simpler to create and manage.

Since the 1950s some of the world’s greatest scientists have successfully implemented machines that provide many benefits, but lack the synergies expected with future robotic requirements. It is hard to accept that human experts cannot solve this problem if they are working on the right path. For this reason, another brain model appears necessary for us to progress.

Even neural network research with associative brain models, such as those considered by emotional brain expert LeDoux (1998) seems to be lacking a design that is sufficient to allow its implementation.

^ back to top ^

AI people often rely on processing approaches

The language used by AI experts indicates the brain paradigm used is often processing based. For example, Pinker (1997) wrote “The mind is a system of organs of computation…” and “The mind is what the brain does; specifically the brain processes information, and thinking is a kind of computation.” And Hoffman (1998) “what happens when you see is … a sophisticated process of construction” And Hickok, Bellugi and Klima (2002), “The brain is a highly modular organ, with each module organized around a particular computational task.”

Another good illustration of that paradigm involves ballistic motion as written by Calvin (1998)

Ballistic arm movements … are so rapid that the brain must plan the sequence of muscle contractions in advance.

Rather than the brain planning the series of muscle contractions, which is another way to say processing, the brain’s input sensors and experience may just select a stored pattern. In this case, throwing a baseball is more than an arm motion; it also includes a whole body motion including torso and legs. Thousands of muscles are synchronized in order to throw a ball, and the brain is simply too slow to do it if the processing of algorithms is needed – a principle that applies to other human actions like speech.

^ back to top ^

Animal brains should align with the human brain

Our brain probably experiences our environment in a similar way to dogs, cats and horses. Indeed, for all we know, our subjective experience of the world may be the same as reptiles and our other ancestors originating from our common brain stem. If so, access to memory could be based on feedback from layer 1 experience only. Humans evolved from simpler animals and our brain inherited many characteristics. As seen in split-brain patients, we appear to have no access to the decision process taking place in higher layer Patoms – only indirect reverse feedback to layer 1.

Animals cannot talk, of course. Therefore the brain shouldn’t rely on linguistic constructs. Human uniqueness from language and episodic memory may result from the additional Patoms located in the human brain’s comparatively large frontal and temporal lobes, mere extensions to an animal’s brain. Because evolution didn’t create humans from scratch, it is a simpler proposition that human brains use stored patterns like other animals rather than proposing the human (and ape) brain spontaneously developed clever algorithms. The same model applies.

^ back to top ^

Thought experiment

Some of the brain’s stored patterns are innate, some are not. Early Patoms like those in our brainstem simply run patterns to keep our heart beating using one of a number of stored patterns. Signals to it change the pattern in use to a faster or slower one. Other early patterns maintain our eye’s pupil dilation and control digestion by initiating stored patterns appropriately. These patterns and many others are run in parallel and in no way impact on each other. They are independent aspects of the body’s machinery utilizing separate Patoms that for evolutionary reasons are located in the one organ, the brain.

^ back to top ^

Evidence from Broca’s and Wernecke’s areas

Specific brain regions control aspects of human language, at least within the confines of the brain’s plastic topology. As discussed by Carter (1998), damage to Broca’s region causes deficits to fluent speech, but leaving comprehension intact. Damage to Wernecke’s area causes deficits in comprehension and spoken meaning, but preserving grammatical speech production.

My conclusions are that: (1) Broca’s area stores the links to sequential speech motion control (in the motor cortex) that we call grammar and pronunciation. Other parts of the brain bias Broca’s area to dictate the meaning of the sentences. (2) Wernecke’s Patom stores the patterns that bias Broca’s Patom (forward links) and the patterns that connect other areas essential for comprehension (backward links). That is, its other function is to provide the snapshot links between other sensory areas. Of course this is a simplified explanation as a patient’s brain damage will not necessarily damage the entire Patom, nor is the Patom the same in any two individuals.

Reverse nature of link set

Figure 3: Recognizing a box may be no more than working backwards from a matched layer 2 pattern.

Visual recognition

How can the brain recognize a box, even when it is not fully visible? The concept of bidirectional pattern matching holds the key. As shown in figure 3, by storing multiple copies of a linked object like a box, a Patom can find that the best match is an object that matches well, even if missing some information. As the best fit, the layer 2 pattern biases the layer 1 pattern resulting in optical illusions. Hoffman (1998) provides a number of good examples of these. The brain rapidly recognizes previously stored objects and by focusing on a visual area for a hundred or so milliseconds, any recognized patterns should be identified in layer 2 providing, again, what we “expect to see” (a key feature of brains).

^ back to top ^

Conclusion

In science, when a model fails to explain observations or cannot predict outcomes, it is replaced. Patomic theory is an alternative approach to the brain’s function that appears to provide plausible solutions to current impasses, aligning with our current observations. It therefore hints at a path for the integration of new functions into future robots and future human implants.

A brain based on stored patterns is a simpler model to one based on algorithmic processing. It aligns an animal’s capabilities with the extensions needed by humans by providing an evolutionary path from animal to human brain capability. And the brain is just too slow to do much else – at least without postulating additional complexities against the teachings of Ockham.

While you may not yet be persuaded of the relative merits of Patomic theory versus the existing approaches, I hope that time will show the merit of applying linkset patterns to the problems of knowledge representation and AI in general. This paper has suggested that there are potentially new areas for AI research that align closely with biological brains.

^ back to top ^

References

Calvin, W. Vol 19, No. 4, 1998. The Emergence of Intelligence. New York,NY: Scientific American.

Carter, R. 1998. Mapping the mind. London, Great Britain: Phoenix.

Damasio, A. 1995. Descartes’ Error. London, Great Britain: Picador.

Goleman, D. 1995. Emotional Intelligence: why it can matter more than IQ. London, Great Britain: Bloomsbury.

Gazziniga, M., Vol 12, No 1, 2002. The Split Brain Revisited. New York, NY: Scientific American.

Greenfield, S., ed. 1996. The human mind explained. Surry Hills, Australia: Reader’s Digest.

Hickok, G., Bellugi, V., and Klima, E., Vol 12, No 1, 2002. Sign Language in the Brain. New York,NY: Scientific American.

Hoffman, D. 1998. Visual intelligence: how we create what we see. New York, NY: W. W. Norton and Co.

Hume, D., 1739 ed Perry, J. and Bratman, M. 1986. Introduction to Psychology: Classical and Contemporary Readings (An Enquiry Concerning Human Understanding). New York, NY: Oxford University Press.

Kurzweil, R., ed Brockman, D. 2003. Science at the Edge. London, United Kingdom: Weidenfeld & Nicolson.

LeDoux, J. 1998. The Emotional Brain. New York, NY: Phoenix.

McCrone, J. 1999. Going Inside: A Tour Round a Single Moment of Consciousness. London, Great Britain: Faber and Faber.

Pinker, S. 1997. How the Mind Works. London, England: Penguin Books.

Waltz, D. 1999. The Importance of Importance. Menlo Park, California: AAAI.

^ back to top ^