A Formal Theory of Optimal Learning with Experimental Results

1 Introduction

Some believe simplicity is the key to general intelligence [Chaitin, 1966]. Simpler models generalise more often, meaning if either of two programs could have generated data and we have to guess which, it is more likely to be the simpler of the two. This correlation between simplicity and generalisation has profound implications for AI. Yet simplicity is a property of form, not function. The function of software is determined by the hardware that interprets it. This undermines claims regarding the behaviour of the theorised, software superintelligence AIXI [Leike and Hutter, 2015]. In theory there could be no correlation between simplicity and generalisation. In practice, there is. Why?

This question cannot be answered by formalisms like AIXI [Hutter and others, 2024], because they frame AI as a disembodied policy interacting with the environment through an interpreter. In previous work I named this 'computational dualism' [Bennett, 2024a], and argued an alternative is needed. The full formal treatment is lengthy, but greatly simplified version of the argument is as follows. Assume C is a space of software minds like AIXI. I want the optimal f1 ∈ C. I can't assume details like actions or timesteps because that would presuppose aspects of interpretation and undermine any claim I subsequently make [Leike and Hutter, 2015]. Fortunately we only need to know if overall behaviour satisfies goals, not how. Γ is a set of overall behaviours. f2 : C → Γ is a hardware body, like the UTM AIXI uses. It interprets f1. f3 : Γ → {0, 1} is the environment in which goals are pursued. If I constructed f1 for a purpose, then I decide whether it has fulfilled that purpose, and I am part of its environment f3. Goals are satisfied iff f3(f2(f1)) = 1. The performance of f1 in f3 depends on f2. It is pointless to make claims based on f1 alone. That is computational dualism. The problem is resolved by formalising the cognitive system-as-a-whole. I call this formalism Pancomputational Enactivism.

2 Formalism

Pancomputationalism holds all systems are computational [Piccinini, 2015]. Enactivism rejects the agent-environment paradigm, reframing cognition as a part of the environment [Thompson, 2007]. In Pancomputational Enactivism, goals are indistinguishable from the systems that pursue them. Instead, a 'task' represents both goal and computational system. Tasks allow claims to be made regarding subjective complexity of form in relation to objective function. Non-standard definitions of terms like language and extension are used to avoid computational dualism. All environments have at least one 'state', states are defined by their relative differences, and time is difference (so states are mutually exclusive). The justification for all of this is given at length in the thesis [Bennett, 2025a]. With abuse of notation, the formalism is:

• Programs: Φ is the set of states, and P = 2Φ is all possible declarative programs. Each program is a potential point of difference between states. Assume a present state ϕ ∈ Φ, and program f ∈ P is true if ϕ ∈ f .
• Embodied Language: A body is formalised as a finite vocabulary v ⊂ P. The embodied language is Lv = {l ⊆ v : ⋂ l ̸= ∅}. Members of Lv are statements the body can express (state of memory etc). A statement l ⊆ Lv is true when ϕ ∈ ⋃ l. El = {y ∈ Lv : l ⊆ y} is called the extension of l.
• v-Tasks: A v-task α = ⟨Iα,Oα⟩ has inputs Iα ⊂ Lv and outputs Oα ⊂ EIα . Assume a uniform distribution over v-tasks. Let α and ω be v-tasks. If Iα ⊂ Iω, Oα ⊆ Oω, then α is examples of ω.
• Policies: π ∈ Lv is a correct policy 1 for α if EIα ∩ Eπ = Oα. Πα is the set of all correct policies for α. The embodied system learns or generalises to ω by inferring a correct policy from examples α that is also correct for ω, meaning π ∈ Πα∩Πω. 'Experience' is adding inputs and outputs to the examples α. Intelligence is measured by sample and energy efficiency in learning ω from α.
• Heuristics: The weakness of a policy π ∈ Πα is |Eπ|, and its complexity is the cardinality of the smallest correct policy π′ ∈ Πα s.t. Eπ′ = Eπ 2 .

1 Assume only policies that imply past observations, like hypotheses in a Boolean SAT problem [Russell and Norvig., 2021].

2 Basically minimum description length [Rissanen, 1978]. Intu-

3 Results

Proof results. Assume you must choose a policy π ∈ Πα. I show that to maximise the probability of choosing π ∈ Πα ∩ Πω, maximising |Eπ| is necessary and sufficient. It is neither necessary nor sufficient to minimise complexity [Bennett, 2023b; Bennett, 2025c]. I prove an objective upper bound for sample and energy efficiency in adaptation, to complement AIXI's subjective upper bound [Bennett, 2024a]. I show correlation between simplicity and generalisation can be explained by weakness confounding the two [Bennett, 2024b]. These results are summarised as an epistemological razor: "explanations should be no more specific than necessary". See thesis appendices for proofs [Bennett, 2025a].

Experimental results. I implemented an A* search based learning system and had it learn to predict binary strings (4bits input, 4-bits output) [Bennett, 2023b]. The tasks were binary multiplication and addition. I compared weakness and simplicity as heuristics for policy selection. Experiments were run with different numbers of examples to learn from. In each experiment search found a simplest (cmdl) and a weakest (cw) policy to compare. With 6-14 training examples (out of 16 total), choosing weak policies yielded a 110 - 500% improvement in generalisation rate (Figure 1). See GitHub for details [Bennett, 2025a].

Interpretation. This can be seen as an alternative to the minimum description length principle [Rissanen, 1978]. The proofs and experiments show weakness is the better heuristic. This shows how to build more energy and sample efficient AI. If scaled, this approach will yield more reliable systems useful for domains where error-tolerance is low. Due to the enactive frame, this formalism also addresses questions in philosophy, biology and complex systems. Two significant results have already been obtained. The first is a formal explanation of the evolution and function of consciousness [Bennett, 2023a; Bennett et al., 2024]. The second formalises the multilayer architecture of biological self-organisation to show these systems leverage the delegation of control to maximise adaptability [Bennett, 2025c]. This provides a formal foundation for 'soft robotics' [Man and Damasio, 2019] and neuromophic computing [Borghi et al., 2024]. These results all together form a thesis [Bennett, 2025a] at the intersection of AI, philosophy of mind and theoretical neurobiology in pursuit of general intelligence [Goertzel, 2014; Bennett, 2025b].

itively, weakness measures function and complexity measures form.