Introducing Deep Symbolics
the next step from deep learning towards AGI
Today I’ll explain how Deep Symbolics (DS) represents the next step in the evolution of systems towards Artificial General Intelligence (AGI).
Deep learning (DL) burst into the technology scene in 2012 with an improved performance in ImageNet. The hand-annotated data in ImageNet is a good use-case for statistical models. However, in our quest for AGI we need ‘General’ solutions that also deal with a brain’s symbolic capability. The move from Deep Learning to Deep Symbolics addresses the shortfalls.
In Diagram 1, the typical concept for artificial neural networks is presented. In operation, inputs are received, the hidden layers do their magic with weighted associations, and the output layer suggests the result. The training step presents training data to update the network’s weights based on the error between the predicted output and the actual output, using tools like backpropagation, before use in production.
Of course, there is a vast variety of ANNs, but the consistent part is (a) inputs received, (b) neural networks in hidden layers next in a variety of configurations, and (c) outputs for results. But DL results in the black-box problem where the results aren’t explainable. It just predicts based on the training set.
Deep Learning isn’t brain-like: but Patoms are
DL is inspired by brains. But those movies that are inspired by real-life aren’t as compelling as those that are based on real-life! Isn’t the ML model too simple?
When I developed Patom theory (PT), the goal was to explain everything we knew about brain function — considering capabilities and the effects of damage in particular. Rather than look at what a neuron does, or the brain’s anatomy, it looked at the evolution of brains, the capabilities of those early brains, and the capabilities of perhaps the most evolved brain — the one in humans.
Brain Evolution
I’ve read that brains exist to control an animal’s movement. It makes sense. If a species can move to feed, reproduce, avoid predators and catch prey, their chances of survival improve.
Human brains also enable languages and the sciences involving mathematics. How did those evolve? Do those capabilities work like motion and senses?
How do brains work? (thought experiment)
The thought experiment was set inside a human brain. What would you see when the brain is throwing a ball? There would be a complex sequence of neurons firing (signaling) to control the muscles in the arms, back, legs, hands and such being sent down from the brain. Coming back would be sensory neurons, signaling as touch and proprioception changed, as well as inner ear feedback on balance.
The control of that mass of signaling is one of the tasks our brain does, along with learning on the fly to improve each throw over time.
When you combine this basic ability, performed by fish as they swim, by birds and bats as they fly, by mammals as they run, and by humans as we speak some language; the control would be the same mechanism if the brain was modeled as a pattern matcher and user.
Deep Symbolics - Emulating Brains
Patom theory emerges from the thought experiment
Bidirectional
Patom theory is a high-level theory — what a brain does, rather than how it does it in detail.
The next step was to notice that what we sense, we can repeat with muscles (think learning golf with a coach helping by moving your body). As we move, our brain gets sensory feedback and vice versa; as we are moved, our brain gets sensory information. The principle is that patterns are bidirectional.
Also, in a human cortex, the anatomy of brain regions shows projections both forwards and backwards between regions. The neurons aren’t bidirectional, but the brain regions are. If regions store, match and use patterns, that is as expected.
Hierarchical
In a human brain’s cortex, our motor control strip is physically next to our sensory strip. The neurons in each region project towards neurons that contract muscles and receive sensory neuron inputs, respectively, via similar but opposite anatomy. This is a hierarchy of patterns: control of patterns is easily distributed. You can imagine a single output from the motor area, whose signal works through the brain and nerves to control hundreds of muscle contractions in a complex sequence.
A similar hierarchy is the recognition of your dog. If you see your dog, you recognize the entire dog, not just the vision of it: its look, feel, sounds and smell are also able to recognize the entire dog. Sensory input is consolidated in the brain into multisensory patterns, not just sense by sense.
Patom theory predicts the connection of sensory regions into a single region for this purpose. The entorhinal cortex is one such region that:
“receives and combines input from all the primary sensory areas… and it sends its output back again to all of them” (Freeman 1999 P104-105)
Linkset Patterns
The last element of Patom theory is that the patterns are linked sets, sequences or combinations of both. When our brain moves our body, a set of muscles are involved. A sequence is also involved as it’s important not to contract opposing muscles at the same time!
PT’s basic assumption is that patterns are stored in brain regions. It follows that they are linked in a brain, not stored like in a database. The pattern is stored in the brain region itself using links to and from other regions for hierarchical patterns.
The material in a digital computer is vastly faster than neurons in a brain. Computer techniques like hash algorithms can locate encoded data. By not encoding data in a brain at all, the use of a simple link between associated areas is efficient and adequate.
Store, match and use bidirectional, hierarchical, linkset patterns
This is where the one-liner comes from: in Patom theory, all a brain does is store, match and use hierarchical, bidirectional linkset patterns.
In Diagram 2, as in a brain, patterns are stored where received in a brain. Sense loss results from that material being lost, or connections to it being lost. Matched sensory patterns connect to dedicated sensory areas (vision has an area for motion and one for color, for example).
The original patterns are retained via reverse links, not encoded versions.
You can see Patom 6 takes sequences of sounds to recognize words and then links them to Patom 5 which would be the word’s meaning. Patom 5 has access to the sensory patterns but doesn’t have them itself.
The real brain’s model is more detailed of course as explained here. The diagram illustrates the principle of pattern matching with senses and brain regions.
Why Deep Symbolics is Better
Knowledge is symbolic, not statistical. You learn details about things (referents) in detail. Normal brains don’t confuse what happened to whom once understood.
In today’s DL systems like LLMs, there are known problems attributable to the statistical nature of the memory used (word vectors and transformers are lossy).
In human brain problems of the 1960s, split-brain patients were tested. When patients responded to cues to create motion, they would be asked a question that was beyond the knowledge of their language. But they answered anyway with an educated guess. That guess, based on incomplete context, is called confabulation.
In LLM’s errors, it confabulates an answer. A hallucination would mean that they incorrectly recognized the details in the first place. Nobody claims that LLMs store all context losslessly as a human does.
Deep Symbolics engines such as the one at Pat Inc. rely solely on symbolic elements. An example of a symbolic element is a dictionary. On one side are the signs (words) in a language and on the other side are the interpretants (definitions) that apply. To create knowledge, sentences are converted from their syntactic forms to their semantic representations. The semantics are added to the situation to comprise knowledge.
Ambiguity resolution in Deep Symbolics
While DS systems can retain ambiguity, the human approach can be replicated because it would ask a clarifying question. How would DS interpret the following:
“I saw her duck!”
This is ambiguous because the pronoun ‘her’ can mean possession (the duck of hers) and it can mean a female (she ducked by dropping her head). Questions like: “She ducked?” surface an answer that removes ambiguity completely.
Deep Learning Architecture is Limited
Humans continuously learn throughout life. Deep learning systems (and LLMs) learn in a training run, and then move to an operational mode.
As the goal of AGI is to emulate humans, an LLM that doesn’t know what happened after its training run is never going to be AGI. The ‘G’ in Artificial General Intelligence should make AI general. In humans we learn new skills as we grow up - like learning arithmetic, history, learning to drive, learning another language, and learning to play games, like chess. Learning on the fly is integral to human skills.
A system that cannot learn on the fly is not human-like. In a given day, humans picks up new names for things — such as new products, services, history and people.
Deep Symbolics Learning
Deep symbolics emulates a brain with the goal of resolving new things by learning on the fly. Like children learning arithmetic, and language, and new topics in school and university. Children don’t shut down their brains and reload them, but learn all the time.
To learn in a Deep Symbolics system, the normal approach is to use the existing environment (dictionary of words, syntactic phrases, and knowledge from context) to add new context in the conversation. By itself, this is useful.
But humans do more than just learn new things like facts and opinions. There is another level. We also learn its building blocks such as new words and phrases for our ‘internal dictionary.’ Here’s an example of learning in action:
M: “Do you like glubs?”
Y: “What’s a glub?”
M: “It’s a red and sweet fruit.”
Y: “Oh, then I like them!”
In this example, glub can be resolved with semantics — a hypernym association, fruit, and even attributes, red and sweet. Effectively the dictionary is updated for ongoing use in conversation.
That beats another training run!
Note that dictionaries for any language can be learned once and used forever with that language, unlike DL systems that need retraining to bring them up to date. The future may well see DS machines learning new languages via dialog, but for now, DS promises new and useful interactions not otherwise possible.
GOFAI is not the answer
Good old-fashioned AI (GOFAI) is often called upon to save us from the problems of LLMs. There’s some merit in researching alternative ways to solve problems, such as ‘classical symbolic AI’. I mean, that is what I do every day, but it isn’t GOFAI. It’s an improved model!
GOFAI is based on computation. And there has been decades of work trying to write programs to make computations fit our world (1950-1990 in particular). In academia, just as statistical systems started to dominate, linguistics became ‘computational linguistics’ in many universities. Psychology became more popular as ‘computational psychology’. Today, the IEEE, probably the strongest supporter of computational disciplines, even has a “computational intelligence society”!
I think it was John McCarthy, one of the founding fathers of AI from the Dartmouth Conference as proposed in 1955 (and developer of LISP for ‘LISt Processing’), behind this extract:
(from Dartmouth Proposal) “every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”
It’s a great plan, but given our viewpoint almost 70 years later, that’s asking too much!
I tend to think that the main limitations we have had relate to locking down our science and engineering too early. The lack of iteration is to blame for the lack of AGI progress. There are many examples, but for one, a plan based on computation alone has limitations that are hard to see.
In terms of machine learning, many of the problems of the world require infinite examples to learn all possibilities (the so-called long-tail) that result from processing new situations continuously.
A solution, to emulate every feature of intelligence as proposed at Dartmouth, is to endlessly write programs. Patterns can reduce the effort by putting that workload into a meaning-based hierarchy. The meaning reduces combinations and hierarchy reduces sequences.
Conclusion
Deep Symbolics promises to align with AGI, because it incorporates a brain model. What better way to emulate a human brain than by using a model of a human brain?
Deep Learning and its most recent popular tool, the LLM, have proven to be excellent tools for many use cases, but they aren’t heading towards AGI because human languages in particular require learning on the fly, including at the word level.
The founding fathers of AI set out a roadmap that still influences our world as its 70th anniversary approaches. The mathematical focus in the 1956 meeting reflects the ongoing focus on computation today. In 2024, systems that cannot in theory achieve AGI, such as machine learning systems, should continue with what they are good at, but not at the expense of future AGI.
The cognitive sciences, as integrated into Deep Symbolics, can help us get on the path to AGI, unconstrained by today’s technology issues.