Consider an embodied agent whose internal state \(y \in Y\), sensor inputs \(s \in S\) and actuator outputs \(a \in A\) vary over time.

We will define a cognitive interpretation \(C\) of the agent as a function which maps the agent’s state \(y\) to a belief \(b \in B\), a desire \(d \in D\) and a fundamental set of immutable assumptions \(\phi \in \Phi\). We will require that the assumptions do not vary with the agent’s state. \[C: Y \to (B \times D \times \Phi) \textrm{ where } \\ \forall y_1, y_2 \in Y {\ \cdot \ }\bigg[ C(y_1) = (b_1, d_1, \phi_1) \ \land \ C(y_2) = (b_2, d_2, \phi_2) \bigg] \implies \phi_1 = \phi_2\]

Given a cognitive interpretation, we can project the agent’s physical internal state into a space of cognitive variables. Consequently, we can observe a “cognitive dynamics”: the changes in the agent’s (attributed) beliefs and desires caused by sensory input (and responsible for actuator output).

By itself, the cognitive interpretation \(C\) is meaningless, since there are no semantics attached to the attributed cognitive content of the agent’s mind. It acquires meaning only when coupled with a theory of rationality \(R\): a normative ideal of how beliefs and desires should be related to sensations and actions.

We will define a theory of rationality \(R\) as a function describing, for an agent with cognitive state \((b, d, \phi) \in B \times D \times \Phi\), faced with sensory input \(s \in S\), what the rational options are for the agent’s next beliefs, desires and actions. \[R: (B \times D \times \Phi \times S) \to (2^B \times 2^D \times 2^A)\] ...where \(2^X\) denotes the powerset of \(X\). Note that more than one belief, desire and / or action may be evaluated as rational by the theory; this is common when the effects of different actions are indistinguishable, and for generality we allow it to be the case for beliefs and desires as well.

We will say that one theory of rationality \(R_1: (B_1 \times D_1 \times \Phi_1 \times S) \to (2^{B_1} \times 2^{D_1} \times 2^A)\) is *subsumed* by another \(R_2: (B_2 \times D_2 \times \Phi_2 \times S) \to (2^{B_2} \times 2^{D_2} \times 2^A)\) if and only if \(R_2\) can compute \(R_1\) under a transformation of the individual variables \(b, d, \phi\). We write this \(R_1 \preceq R_2\):

\[\begin{aligned} & R_1 \preceq R_2 \iff \\ & \forall b \in B_1, \ d \in D_1, \ \phi \in \Phi_1, \ s \in S \\ & \exists f_B: B_1 \to B_2, \ f_D: D_1 \to D_2, \ f_\Phi: \Phi_1 \to \Phi_2 {\ \cdot \ }\\ & R_1(b, d, \phi, s) = R_2\left(f_B(b), f_D(d), f_\phi(\phi), s\right) \end{aligned}\]

Note that the equality in the final expression implies set identity in the components of \(R_1\) and \(R_2\)’s values. It should be obvious that \(\preceq\) constitutes a partial order relation, and hence defines an equivalence relation \(R_1 \sim R_2\) in the case that \(R_1 \preceq R_2 \preceq R_1\).

One very natural way of representing beliefs \(b\) and \(\phi\) mathematically is as subjective probability functions. In this case, we may restrict our theories of rationality to those which are consistent with Bayesian inference.

Assume that a belief \(b\) actually corresponds to a probability function representing a belief about the instantaneous state \(z_t\) of a “phenomenal world” variable \(Z\), and how the world’s next state \(z_{t+1}\) is co-distributed with instantaneous action \(a_t\) and sensation \(s_{t+1}\). \[b : (Z \times Z \times A \times S) \to [0, 1]\] We will denote the underlying probability space as \(\mathbb{B}\).

We can then define a Bayesian theory of rationality \(R\) as one in which beliefs change according to Bayes’ rule, and the “laws of physics” remain constant: \[\begin{aligned} & R(\mathbb{B}, d, \phi, s, a) = (\mathbb{B}^*, d^*, a^*) \\ & \mathbb{B}^*(Z_t=z') = \mathbb{B}(Z_{t+1} = z' {\ | \ }Z_t, S=s, A=a) \\ & \mathbb{B}^*(Z_{t+1} = z' {\ | \ }Z_{t} = z, S=s, A=a) = \mathbb{B}(Z_{t+1} = z' {\ | \ }Z_t = z, S=s, A=a)\end{aligned}\] ...where \(d^*\) and \(a^*\) are still arbitrary sets of rational desires and actions.