A theory of intelligence that denies teleological purpose

🌈 Abstract

The article explores the challenges in defining the "human task" or the purpose of human intelligence, which is crucial for developing general artificial intelligence (AGI) that can match human-level intelligence. It examines four popular paradigms for defining the purpose of the human mind - optimization of a metric, problem-solving, world-modeling, and spiritual development - and highlights the limitations of each. The article suggests that a coherent understanding of the mind requires transcending these individual paradigms and questioning the common assumptions about interpreting human intelligence.

🙋 Q&A

[01] The mysteries of the mind and the need to define the "human task"

1. What is the key question the article aims to address?

• The article explores the fundamental question of what the "human task" or purpose of human intelligence is, which is crucial for developing AGI that can match human-level intelligence.

2. Why is it important to define the "human task"?

• If the goal is to create an AI that is comparable to human intelligence, then it is imperative that we discover the task and accompanying metric by which we measure human success or intelligence. Any AGI would be measured against the same overall benchmark by which we measure ourselves.

3. What is the challenge in defining the "human task"?

• There is no consensus on what represents the true purpose of human cognition in general, and the very purpose of the brain remains elusive. Existing theories either over-generalize or are too vaguely defined to be useful for measuring performance in a practical agent.

[02] Removing "purpose" and examining the mind's functions

1. What is the alternative approach proposed in the article?

• Instead of defining a high-level "purpose" for the mind, the article suggests framing the mind as a set of proper functions or procedures, and focusing on understanding "what a mind does" rather than "what is the purpose of the mind".

2. What are the challenges with this alternative approach?

• Without relying on the crutch of finding a high-level "purpose", every mental event - its nature, causes, and alterations - must be elucidated in mechanical detail, which is a significant challenge for current research.

3. What are the four popular candidates for what the mind does, as discussed in the article?

1. The mind tries to optimize some metric
2. The mind is a problem solver
3. The mind understands or models the world
4. The mind pursues spiritual development

[03] Evaluating the four candidates for the "human task"

1. What are the key limitations of the "mind as optimizer" paradigm?

• Human cognition does not always follow a process of optimization, as we often satisfice or act contrary to optimization goals. There are many aberrant examples of human behavior that cannot be easily explained by optimization.

2. What is the issue with the "mind as problem-solver" paradigm?

• The mind not only solves problems, but also defines them, which the problem-solving paradigm does not account for. Additionally, rigid problem definitions lead to narrow AI rather than general intelligence.

3. Why is the "mind as world-modeler" paradigm insufficient?

• Modeling the entire universe is intractable, and the mind needs additional cognitive structures to cull the world-model down to what is useful. The mind's desire to learn shapes the world-modeling process, suggesting that motives precede understanding.

4. How does the article characterize the "mind as spiritual development" paradigm?

• Spiritual development is seen as a means of confronting and dealing with the difficult transitions of human existence, rather than the fundamental driver of cognition. It is a reaction to the issues of how the mind interacts with its environment, rather than the interaction itself.

[04] Synthesizing the insights from the four paradigms

1. What common threads or connections does the article identify across the four paradigms?

• The article suggests that each paradigm has some valid insights, which can be combined to form a more coherent picture of the underlying cognitive processes. For example:
  • Goal-orientation and world-modeling are linked by the role of motives in shaping understanding.
  • Problem-solving and world-modeling are connected through the act of building causal plans about the world.
  • Changes in motives link goal-orientation, world-modeling, and spiritual development.

2. What is the key challenge in uniting these paradigms into a comprehensive understanding of the mind?

• Transcending any single paradigm and questioning the common assumptions about interpreting human intelligence is necessary to resolve the contradictions within the four paradigms and arrive at a coherent understanding of the mind.

3. What is the proposed approach for describing the mind, as an alternative to defining a single "purpose"?

• The article suggests describing the mind not as a machine with a single purpose, but as a set of processes and events that cumulatively appear to have multiple purposes. This requires detailing the necessary primitives and their interactions in a way that can be applied to all cases of cognition.