preprint,amsmath,amssymb,aps,]revtex4-1

\label{sec:level1} A model of animal behavior

An animal behavior is a dynamical process whose trajectory is defined by the intrinsic dynamics of the animal, the constraints of the environment, and the mutual (closed loop) interaction between the animal and environment that changes the animal’s dynamics.
Here we introduce a formal model for such system. Let assume that the intrinsic (without constrains) dynamic of the animal can be described by a state space \(\mathcal{X}\) and an Ito diffusion equation
\begin{equation} X_{t}=X_{0}+\int_{0}^{t}f(X_{s},s,a_{s})ds+\sigma\int_{0}^{t}F(X_{s},s,a_{s})dW_{s},\\ \end{equation}
where \(f:\mathbb{R}^{n}\times\mathbb{R}\times\mathbb{R}^{d}\to\mathbb{R}^{n}\), \(F:\mathbb{R}^{n}\times\mathbb{R}\times\mathbb{R}^{d}\to\mathbb{R}^{n\times m}\), and \(W_{t}\) is a \(k\)-dimensional Brownian motion. We call \((a_{t})_{t\geq 0}\) the affordance parameter process, or affordance process for short, that represents the dynamic of the parameter and defines what the animal can do.
Let \(C\subset\mathbb{R}^{d}\) be a set that represents the constraints of the dynamics of the animal in the state space. If the dynamics of the animal \(X_{t}\) hits the constraint \(C\) at time \(\tau\) and point \(x_{\tau}\), the dynamics are updated, following the equations
\begin{equation} X_{t}=x_{\tau}+\int_{\tau}^{t}f(X_{s},s,a_{s})ds+\sigma\int_{\tau}^{t}F(X_{s},s,a_{s})dW_{s},\\ \end{equation}
where \(x_{\tau}\) is not necessarily equal to \(X_{\tau}\) and
\begin{equation} a_{t}=a_{0}+\tilde{W}_{t},\\ \end{equation}
where \((\tilde{W}_{t})_{t\geq 0}\) is a \(d\)-dimensional Brownian motion independent of \((W_{t})_{t\geq 0}\). We call the above system of equations Affordance-Diffusion (AD) model. To make the process càdlàg the new value of \(X_{\tau}\) is set to be \(x_{\tau}\).
Let \(\tau_{0}=0\) and \(\tau_{i},i>0\) are the times at which the AD model hits the constraint \(C\) and \(x_{\tau_{i}},i>0\) are the starting points of the dynamics after hitting the constraint, we can rewrite the AD model more compactly as
(Affordance-diffusion process).
The process \((X_{t},a_{t})\) defined by
\begin{align} X_{t} & \label{eq:general1}=\sum_{i=0}^{\infty}\mathbbm{1}_{\{\tau_{i}\leq t<\tau_{i+1}\}}\left\{x_{\tau_{i}}+\int_{\tau_{i}}^{t}f(X_{s},s,a_{s})ds+\sigma\int_{\tau_{i}}^{t}F(X_{s},s,a_{s})dW_{s}\right\}, \\ a_{t} & \label{eq:general2}=a_{0}+\sum_{i=0}^{\infty}\tilde{W}_{\tau_{i}}\mathbbm{1}_{\{\tau_{i}\leq t<\tau_{i+1}\}},\\ \end{align}
is called affordance-diffuion process.
.
It is usual in the diffusion process literature to define a domain \(D\) as a subset of the state space and consider the border of the domain \(\partial D\) as representing the constraint. We define the constraint directly, as it seems more natural from experimental view.
.
The word affordance was introduced by Gibson (REF) to describe the possibilities of an animal behavior given the constraints of the environment. We are modeling the affordance not as a property of the environment alone, but as a property of the animal and its constraint altogether.
Now, our main problem is how to define behaviors for the AD model. If the animal dynamics does not depend on time and the affordance parameter, then the natural way to define behavior is to first partition \(\mathcal{X}\setminus C\) based on some criteria and then define behavior as an element of the partition. The question becomes less trivial when the dynamics depend on time and the affordance parameter. In this case, we can obviously associate a partition for each time and value of the affordance parameter, but we will have to define when the dynamics and the partitions can be considered equivalent. Hence, we will need the following set of definitions:
(Topological conjugacy).
Let
\begin{equation} X_{t}=X_{0}+\int_{0}^{t}f(X_{s},s)ds,\\ \end{equation}
and
\begin{equation} Y_{t}=Y_{0}+\int_{0}^{t}f^{\prime}(Y_{s},s)ds,\\ \end{equation}
be two deterministic dynamics. We say that \(X\) is conjugate to \(Y\) if there is a homeomorphism \(h:\mathcal{X}\to\mathcal{X}\) such that \(f(h(x),t)=h(f^{\prime}(x,t))\) for \(x\in\mathcal{X}\) and \(t\geq 0\).
(Conjugacy between diffusion processes).
Let
\begin{equation} X_{t}=X_{0}+\int_{0}^{t}f(X_{s},s)ds+\sigma\int_{0}^{t}F(X_{s},s)dW_{s},\\ \end{equation}
and
\begin{equation} Y_{t}=Y_{0}+\int_{0}^{t}f^{\prime}(Y_{s},s)ds+\sigma^{\prime}\int_{0}^{t}F^{\prime}(Y_{s},s)dW^{\prime}_{s},\\ \end{equation}
be two diffusion processes. We say that both processes are conjugate if the respective deterministic dynamics (i.e., \(\sigma=\sigma^{\prime}=0\)) are topologically conjugate. Furthermore, given \(P\) a subset of the state space \(\mathcal{X}\), we say that \(X\) and \(Y\) restricted to \(P\) is conjugate if the deterministic dynamics restricted to \(P\) is topologically conjugate.
(Pre-behaviors and pre-holo-behaviors).
A pre-behavior is a restriction of the diffusion process to a subset of \(\mathcal{X}\). A pre-holo-behavior is the set of pre-behaviors associated with a partition of \(\mathcal{X}\).
(Equivalence).
We say that pre-holo-behaviors \(hB_{1}\) and \(hB_{2}\) are equivalent if (1) the respective processes are conjugate and (2) for each pre-behavior in \(hB_{1}\) there exist a conjugate pre-behavior in \(hB_{2}\).
(Behaviors and holo-behaviors).
The equivalence classes of pre-behaviors and pre-holo-behaviors are called behaviors and holo-behaviors.
These definitions are straightforward to understand if we think about a dynamics of an animal behavior. Assume that the phase space that we are considering is the position of the animal. The dynamics is the locomotion of the animal. The constraints are the walls of the space in which the animal is moving. Whenever the animal hits the wall, it changes the dynamics. For example, the animal changes the velocity, or concentrate the locomotion far away from the walls. It is natural to assume that the locomotion before and after hitting the wall are still the same behavior, although their dynamics changed. Using our definitions, we can formalize this natural assumption by saying that the locomotion before and after hitting the wall are in the same equivalence class.
The interesting question is why the affordance is relevant? The answer
(Adaptiveness of a dynamic with respect to the constraint).
The equivalence classes of pre-behaviors and pre-holo-behaviors are called behaviors and holo-behaviors.
Metastable state is about the entire system
Degree of adaptation is about the entire system
Sometimes, the affordance can be exactly the behavior in the case that there is only one behavior.
Affordance is the unit of behavior

Some examples of Affordance-Diffusion model

\label{SS:behavior}
In this section, we will consider some specific examples of animal behaviors that can be modeled by the AD model.
(Locomotion).\label{ex:exploration}
Consider an animal that explores on a circular open field with radius \(c>0\). The position of the animal in the plane is the state space. The constraint \(C=\{(x,y)\in\mathbb{R}^{2}:x^{2}+y^{2}<c)\), represents the borders of the track. We assume \(V(x,y,a_{x},a_{y})=(a_{x}x^{2}+a_{y}y^{2})/2\). Whenever, the animal interacts with the border (e.g., by proprioception or vision), the affordance parameter change and therefore the animal dynamics or, equivalently, the potential changes. The model can be written as
\begin{align} dX_{t} & \label{eq:general1}=\frac{ax}{\mathbbm{1}\{X_{t}\notin C\}}dt+dW_{t}, \\ da_{t} & \label{eq:general2}=d\tilde{W}_{t}\mathbbm{1}\{X_{t}\in\partial C\}.\\ \end{align}
SHOW SIMULATIONS AND DISCUSS SOME IDEAS.
(Foraging).
Consider an animal that forages in a linear track. The state space in this case is the distance between the animal and food source. Let \(d\) be the position of the food with respect to the center of the track. The constraint \(C=(-c-d,c-d)\), for \(c>0\), represents the distance of the borders of the track from the food. We assume \(V(x,a)=ax^{2}/2\).
SHOW SIMULATIONS AND DISCUSS SOME IDEAS.
(Decision making).
Here we consider the classical drift diffusion model. The state space in this case is the position of the animal in the track. The constraint \(C=(-c,c)\), for \(c>0\), represents the borders of the track. We assume \(V(x,a)=ax\).
SHOW SIMULATIONS AND DISCUSS SOME IDEAS.
(Show the transition from one well to two wells).
Two well example. Maybe in 2D.

The 1D Affordance-Diffusion model

\label{SS:model}
We will introduce a model – 1D Affordance-Diffusion model – that captures the main ideas of our framework, but it is simple enough that we can show several of its properties.
Let \((X_{t},a_{t})_{t\geq 0}\) be a stochastic process whose dynamics is defined as follows. We first fix a bounded open set \(C\subset\mathbb{R}\) with border \(\partial C\). We also define the parametrized potential \(V:\mathbb{R}^{2}\to\mathbb{R}\), such that, \(\frac{\partial V(\cdot,a)}{\partial x}\) is continuous for each \(a\in\mathbb{R}\). Let \(X_{0}=0\), \(a_{0}=0\), \(\sigma>0\), the 1D AD model is defined as
\begin{align} dX_{t} & \label{eq:basic1}=-\frac{\partial V(X_{t},a_{t})}{\partial x}\frac{1}{\mathbbm{1}\{X_{t}\notin C\}}dt+\sigma dW_{t}, \\ da_{t} & \label{eq:basic2}=d\tilde{W}_{t}\mathbbm{1}\{X_{t}\in\partial C\}.\\ \end{align}
The process \((X_{t})_{t\geq 0}\) is an overdamped diffusion on \(C\), such that whenever it reaches the boundary \(\partial C\), a small change in the potential happens. The process \((a_{t})_{t\geq 0}\) is a jump process that we call affordance process. \(\mathbb{P}\) is the canonical measure associated with \((X_{t},a_{t})_{t\geq 0}\). We define the set of hitting times \(\mathcal{H}\) to be the (random) set of index \(t\) for which \(X_{t}\) defined by eqs. (\ref{eq:basic1}) and (\ref{eq:basic2}) hits the boundary \(\partial C\). Observe that because of the interaction between \((X_{t})_{t\geq 0}\) and \((a_{t})_{t\geq 0}\), both process are not Markovian separately, although the joint process is Markovian. We also note that AD model is not a hidden model, because \(a_{t}\) is not independent of the past values of \(X_{t}\) given the past values of \(a_{t}\). In other words, the dependence goes in both ways, representing the closed loop interaction between the diffusion dynamics and the affordance.
In our context, the process \((X_{t})_{t\geq 0}\) represents the dynamics of an animal behavior in the state space and \(C\) represents the set of constraints that the dynamics of the animal has to satisfy. The interaction between the animal and the constraint induces changes in the dynamics of the animal, by changing the potential \(V\) parametrized by the affordance process \((a_{t})_{t\geq 0}\).
We also define the auxiliary processes \((Y_{t}^{b})_{t\geq 0}\) where \(Y_{0}^{b}=0\) and
\begin{equation} \label{eq:auxiliary}dY_{t}^{b}&=-\frac{\partial V(Y_{t}^{b},b)}{\partial x}dt+dW_{t}.\\ \end{equation}
These processes are simply the diffusion process with potential \(V\) for a fixed parameter \(b\) and without constraint.

Results

\label{SS:results}
In what follows, we will prove some results related to the AD model.
We will first show that the dynamics of animal behavior is non-stationary and consequently never achieves equilibrium.
.
Let the parametrized potential \(V\) and constraint \(C\) be such that \(\mathbb{P}(Y_{t}^{b}\notin C)>0\) for all \(b\). Then, the set \(\mathcal{H}\) has infinite many elements \(\mathbb{P}\)-almost surely. Therefore the Affordance-Diffusion process is non-stationary. Moreover, \(\lim_{t\rightarrow\infty}a_{t}=\infty\) \(\mathbb{P}\)-a.s.
\noindentProof
. Let \(\mathcal{H}_{t}=\mathcal{H}\cap[0,t]\). We can write the dynamics of \(a_{t}\) as
\begin{equation} a_{t}=\sum_{k\in\mathcal{H}_{t}}\eta_{k},\\ \end{equation}
where \(\eta_{k}\sim\mathcal{N}(0,1)\). Because \(\mathbb{P}(Y_{t}^{b}\notin C)>0\), we conclude that, \(\mathcal{H}\) has infinite many elements \(\mathbb{P}\)-a.s. This implies that, \(\lim_{t\rightarrow\infty}\sum_{k\in\mathcal{H}_{t}}\eta_{k}=\infty\) \(\mathbb{P}\)-a.s. \(\blacksquare\)
.
The exploration model (Example \ref{ex:exploration}) is an example that have \(\mathbb{P}(Y_{t}^{b}\notin C)>0\).
We next investigate the distribution of elements of \(\mathcal{H}\). Let \(k_{i}\) be the \(i\)-th element of \(\mathcal{H}\). For \(i\geq 1\), define the hitting intervals \(T_{i}=k_{i+1}-k_{i}\). We are interested on calculating the distribution of \(T_{i}\)’s.
(—\cite{day1983exponential}—).\label{theo:exponential}
If \(C\) is bounded and if for each \(a\in\mathbb{R}\) it is contained in the basin of attraction of an isolated, asymptotically equilibrium point of the deterministic system \(\dot{x}=V(x,a)\), then we have for any \(i\) and \(x_{0}\in\mathbb{R}\),
\begin{equation} \lim_{\sigma\rightarrow 0}\mathbb{P}(T_{i}>s\mathbb{E})=\exp(-s)\\ \end{equation} \noindentRemark
. Use this to introduce some statistical mechanics concepts like potential, etc.
.
We say that a process \((Y^{b}_{t})_{t\geq 0}\) has scale \(L\) if \(T^{b}=\min\{k\geq 1:Y_{k}^{b}\notin C\}\) satisfies \(\mathbb{E}>L\).
.
A affordance set of scale \(L\) is the set \(\mathcal{A}^{L}=\{b\in\mathbb{R}:(Y^{b}_{t})_{t\geq 0}\;\;\text{has}\;\;\text{scale}\;\;\)L\(\}\).
.
Let \(i\in\mathcal{H}\). If \(T_{i}>L\), we say that \(a_{i}\) is a metastable state of scale \(L\). We call \(\mathcal{M}^{L}\) the set of all metastable states of scale \(L\).
We will show that most of metastable states of scale \(L\) happens when the coefficients of the animal behavior is a subset of affordance set of scale \(L\).
.
We have
\begin{equation} \mathbb{P}(a_{i}\in\mathcal{M}^{sL}\setminus\mathcal{A}^{L})\leq\exp(-s).\\ \end{equation} \noindentProof
. If \(a_{i}\in\mathcal{M}^{sL}\), we have that \(T^{a_{i}}>sL\). By Theorem \ref{theo:exponential} we have
\begin{equation} \mathbb{P}(T^{a_{i}}>s\mathbb{E})\leq\exp(-s).\\ \end{equation}
If \(a_{i}\notin\mathcal{A}^{L}\), then \(\mathbb{E}\leq L\), therefore,
\begin{equation} \mathbb{P}(T^{a_{i}}>sL)\leq\exp(-s),\\ \end{equation}
which ends the proof. \(\blacksquare\)
.
We have
\begin{equation} \mathbb{P}\left(\frac{\sum_{a_{i}\in\mathcal{M}^{L}_{t}}\ind\{a_{i}\notin\mathcal{A}^{sL}\}}{\{a_{i}\in M^{L}_{t}\}}>\epsilon\right)\leq\exp(-\epsilon t/L).\\ \end{equation} \noindentProof
. Concentration of measures. \(\blacksquare\)
These results justify the idea that \(\mathcal{A}\) is a good set to define behaviors, i.e., \(\mathcal{A}^{L}\) is the set of coefficients of the dynamics of animal behavior in which most of the phenomenas that are stable for at least duration \(L\) happens.
.
The set of behaviors is a partition of the state space.
In practice, in several cases the experimenters choose the partition using some criteria of choice. Nevertheless, our theory suggests a natural way to partition. We first define the average potential.
.
The average potential \(\bar{V}(x)\) is given by
\begin{equation} \bar{V}(x)=\frac{1}{\mathbbm{1}\{x\notin C\}}\lim_{t\to\infty}\frac{\int_{0}^{t}V(x,a_{s})ds}{t}.\\ \end{equation}
The marginal probability of \((a_{t})_{t>0}\) is given by \(F\).
(Law of large numbers for Affordance - Diffusion process).
We have
\begin{equation} \lim_{t\to\infty}\frac{\int_{0}^{t}V(x,a_{s})ds}{t}=\int_{\mathbb{R}}V(x,a)F(a)da.\\ \end{equation} \noindentProof
. Law of large numbers. \(\blacksquare\)
Now, given the average potential, we can partition the state space by fixing a scale \(l\) in the following way.
Let \(x^{*}=\arg\max\bar{V}(x)\). The holo-behavior of scale \(l\) is the set \(hB=\{x\in\mathbb{R}:\bar{V}(x*)-\bar{V}(x)>l\}\). A behavior is a subset of \(hB\) that is an interval and it is not contained in other intervals that are subset of \(hB\).
\phr
Give some examples of behaviors. The definition of behavior reflects the variety of ways that an experimenter can define a behavior \phrA partition is always in the parameter space. \phrGive some examples of behaviors.
There exist a behavior at every scale.
.
The process \((a_{t})_{t>0}\) is metric isomorphic to RW\(\otimes\)Po.
\noindentProof
. Let \((T_{i},a_{i})\) be the hitting interval and the respective metastable state. We define the rescaled hitting interval \(\tilde{T}_{i}=\frac{T_{i}}{\mathbb{E}}\). By Theorem \ref{theo:exponential}, we have that \(\tilde{T}_{i}\sim\) Po. Also, by definition, \(a_{i}\sim\) RW. \(\blacksquare\)
(Minimax principle of affordance). \begin{align} \lim_{\sigma\rightarrow 0}\sup_{a\in\mathbb{R}}\log\mathbb{E}= \\ \sup_{a}\inf_{t>0,y\in\partial C}\{J_{[0,t]}(\phi):\phi_{0}=x^{*},\phi_{t}=y\}\\ \end{align}
where
\begin{equation} J_{[0,t]}(\phi)=\frac{1}{2}\int_{0}^{t}\left(\dot{\phi}_{s}-\frac{\partial V}{\partial x}(\phi_{s})\right)^{2}ds\\ \end{equation}
Given two probability measures \(Q\) and \(R\), the total variation distance \(d_{TV}\) between \(Q\) and \(R\) is given by \(d_{TV}(Q,R)=\sup_{A\in\Omega}|Q(A)-P(A)|\).
. \begin{equation} \lim_{\sigma\rightarrow 0}\mathbb{E}.d_{TV}(\tilde{P_{a}},P_{a})=1.\\ \end{equation} .
The dynamics that are most adapted to a constraint \(C\) are the ones in which the mean hitting time is largest.
.
Variational principle for the GTB. Use Wentzell–Freidlin theory
The state space is determined by what the researcher wants to understand (the umwelt of the experimenter).
.
A behavior is a partition of the state space defined by the potential. Therefore, for each affordance parameter, there is an associate partition.
.
The average potential can be used to partition the state space and define an overall set of behaviors. This should be done or each scale of affordance that we choose.
In our model, we consider the internal perspective of the animal and not the position of the external observer . The internal world constructed by the constraints and dynamics giving rise to the specific umwelt of the animal.
With this model, we can discuss the difference between adaptation and optimization . Adaptation is to be regarded as the non - convergence but if the potential converges then it is called optimization (relating it then to simulated annealing) The illusion of adaptation ?! They spontaneous emergence via affordances give the impression that adaptation is the causal force behind shaping ”behavior”.
One possible way to think about this result is that the the interaction between dynamics and constraint is a sequential hypothesis testing process which converges toward minimization of type I and type II error rates.
The difference between dynamics and constraint is just a difference in time-scale. The constraint is the slow portion and the dynamics is the fast portion.
Relationship to simulated annealing , stochastic variational inference
The information geometry of the Gamma distribution (information geometric measures defined on the Riemann manifold)
Discussion on the use of these principles to design artificial agents and thermodynamical framework for artificial intelligence
Feeling of understanding!! By construction because our behavior matches as best of possible the constraint of the world.
\begin{align} dX_{t} & \label{eq:basic1}=-\frac{\partial V(X_{t},a_{t})}{\partial x}\frac{1}{\mathbbm{1}\{X_{t}\notin C\}}dt+dW_{t}, \\ da_{t} & \label{eq:basic2}=\sigma d\tilde{W}_{t}\mathbbm{1}\{X_{t}\in\partial C\}.\\ \end{align}