What is needed for AGI by 2029?

a SCIENTIFIC change, such as using lossless knowledge

Predictions of engineering are easy when compared to predictions about science.

Today, we can look at Ray Kurzweil’s prediction that AI will be at human level, called AGI by some, by 2029. He first made that prediction, that a machine will pass the Turing test in 1999, in his second book titled The Age of Spiritual Machines. He clarified with Gary Marcus that the predication is unchanged on June 22, 2024.

Kurzweil’s definition was clarified as: “AI that can perform any cognitive task an educated human can.” So that certainly requires the machine to solve today’s limitations with LLMs like hallucinations, and the inclusion of lossless knowledge in context!

But, if we use the prediction method of one of the great scientists in recent history, Max Plank: “Science progresses one funeral at a time.” In other words, in science:

“What does happen is that its opponents gradually die out, and that the growing generation is familiarized with the ideas from the beginning”

In many cases, however, today’s students aren’t taught the new approaches. They are just taught the old approaches that are not working!

Engineering Predictions

Moore’s law is an engineering prediction. It:

“suggests that computational progress will become significantly faster, smaller, and more efficient over time.” per Investopedia.

Gordon Moore’s 1965 prediction has been, arguably, accurate up to the present. But it doesn’t predict a scientific change, just an improvement of technology that is already well known - an engineering change.

Scientific Predictions

AI today isn’t at human level. I don’t say that because it is too slow, but because the underlying science doesn’t support it.

In my speciality area, Natural Language Understanding (NLU), I believe that the computing power needed to emulate human speech and comprehension was sufficient in the year 2000. Today’s Large Language Models (LLMs) have severe problems in profitability due to their cost and energy consumption, but also in function as I mentioned last time. A system that returns hallucinations has limited use cases in the commercial world.

What is needed to fix the hallucination problem?

A few months ago, Thomas Dietterich presented on LLM problems and what is needed, instead. As “one of the pioneers of the field of machine learning” he is someone I trust to critique approaches like LLMs.

He has a few suggestions, that are on the scientific change roadmap, rather than the engineering roadmap, to solve. In his video at the 39 minute mark, he presents this diagram:

YouTube video screen image from Prof. Dietterich’s presentation

This differs somewhat from Patom theory’s model, but the point is leading to his recommended improvements to LLMs. Note solving ‘common sense’ has been expressed as a problem in AI since its inception in the 1950s. This breakdown leads to this diagram:

From the same presentation, factual world knowledge is shown as a solution.

In this diagram, the addition of factual world knowledge and attribution is shown as a part of the solution on the right side - a solution to deal with hallucinations.

For AGI, this knowledge would need to be learned by the machine, automatically from text on-the-fly, as people do, of course, even if contradictory.

In terms of linguistics, we should think of the ideal solution to follow the model from Role and Reference Grammar (RRG) in which knowledge is independent of any particular language. In this case, attribution of where the knowledge comes from is a part of context, too. And the meaning of the knowledge should be independent to the language itself, since the words in a language are only pointers in the brain to its meaning as we see from some cases of brain damage.

So LLMs do a number of things, BUT they suffer from errors called “hallucinations” that has no current solution. Some solutions can improve the results, but not enough to trust its output 100% of the time. If a system like an LLM generated only 100% accurate results, they would still be lacking in some respects, but developers would probably only focus on their lack of profitability.

The next 5 years will be impressive if such a large jump in capability to be solved.

Conclusion

A prediction from Kurzweil that AGI, or human-level AI will arrive by 2029, was based on Moore’s law - faster computation meeting the human brain’s computational level by 2029. The problem isn’t speed, but function. It isn’t an engineering limitation, but a scientific one.

Looking at Thomas Deitterich’s next steps suggests a new approach entirely to the problems in LLMs. A new scientific approach is needed. As he writes, LLMs are “Statistical models of knowledge bases rather than knowledge bases.”

The next steps would benefit from the cognitive sciences, such as that provided by RRG, but such a project merging LLM technologies that combines linguists and computer scientists may take more than the 5 years available to organize and complete by 2029.

The alternative would be a full pivot away from LLMs using the cognitive sciences as I advocate for. Perhaps that’s the next step?