\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage[round]{natbib}
\let\cite\citep




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[ngerman,english]{babel}









\iflatexml
% \documentclass[%
%reprint,
%superscriptaddress,
%groupedaddress,
%unsortedaddress,
%runinaddress,
%frontmatterverbose, 
preprint,
%showpacs,preprintnumbers,
%nofootinbib,
%nobibnotes,
%bibnotes,
 amsmath,amssymb,
 aps,
%pra,
%prb,
%rmp,
%prstab,
%prstper,
%floatfix,
]{revtex4-1} % Not (yet) supported by LaTeXML.
\fi
\usepackage[]{graphicx}
% Include figure files
\usepackage[]{dcolumn}
% Align table columns on decimal point
\usepackage[]{bm}
% bold math
\usepackage[]{color}

\usepackage[]{bbm}
\newtheorem{remark}{Remark}
\newtheorem{example}{Example}
\newtheorem{lemma}{Lemma}
\newtheorem{hypo}{Hypothesis}
\newtheorem{obs}{Observation}
\newtheorem{fact}{Fact}
\newtheorem{defi}{Definition}
\newtheorem{theo}{Theorem}
% First macro on next line not (yet) supported by LaTeXML: 
% \newtheorem*{theo*}{Theorem}
\newtheorem{prop}{Proposition}
\newtheorem{coro}{Corollary}
\newtheorem{notalert}{Notation alert}
\renewcommand{\E}[][]{\ensuremath{\mathbb{E}}}

\renewcommand{\P}[][]{\ensuremath{\mathbb{P}}}

\newcommand{\R}[][]{\ensuremath{\mathbb{R}}}

\newcommand{\Z}[][]{\ensuremath{\mathbb{Z}}}


\begin{document}

\title{preprint,amsmath,amssymb,aps,{]}revtex4-1 Thermodynamics of spontaneous
animal behavior}



\author[1]{Yasumasa}%
\affil[1]{Affiliation not available}%


\vspace{-1em}



  
  \date{May 24, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup





\selectlanguage{english}
\begin{abstract}
preprint,amsmath,amssymb,aps,{]}revtex4-1 What is a behavior? Defining
and understanding animal behavior is a key problem in biology generally.
We don't have a good general formal theory of behavior. There have been
a tremendous development in methods to analyze and cluster behavior,
into its components, without formulating the mechanism by which behavior
emerges from the interaction of these different elements (clusters or
sub-clusters). The few available attempts consider very specific
examples without formulating a general theory, or when there is a
general theory, it is not formalized enough to allow quantitative
predictions.

In this work, our aim is to introduce a theory that is general enough to
explain basic characteristics of animal behaviors, but precise enough to
allow quantitative predictions about animal behaviors.

First, we will consider a very simple model of a dynamical process on
phase space to derive some basic principles. We call this simple model
''Affordance - Diffusion model'' in which behavior is abstracted as a
particle diffusing over an affordance landscape, which is an energy
landscape dynamically changing in time. Second, we compute the potential
in the case of the particle, in the affordance - diffusion model, is
interacting with some given constraints. Third, we derive two main
general principles of ''behavior'' within the framework of the
affordance - diffusion model: ''minimax principle of affordance'' and
''the variational principle of affordance'' (blurb about the principles
?!). Our study lay the foundation for a thermodynamical theory of
behavior.

Secondary publications and information retrieval purposes.

May be entered using the \textbackslash{}pacs\{\#1\} command.

You may use the description environment to structure your abstract; use
the optional argument of the \textbackslash{}item command to give the
category of each item.%
\end{abstract}%



\sloppy


preprint,amsmath,amssymb,aps,{]}revtex4-1 ++thanks: Pew Latin American
fellowship, Science without Borders fellowship, HHMI

\textbackslash{}email

takahashiyd@gmail.com

\textbackslash{}emailahady@princeton.edu

preprint,amsmath,amssymb,aps,{]}revtex4-1

\section*{Main messages}\label{main-messages}

\begin{itemize}
\tightlist
\item
  What is behavior? Spontaneous behavior have been primarily the focus
  of ethology that aimed to characterize animal behavior. Recently that
  have been an increasing interest in dissecting the different
  components of animals' spontaneous behavior. Less efforts have been
  targeted at formally defining what a behavior is. Animal behavior is
  studied within ecology and within neuroscience . Both fields have
  divergent approaches the former one focus on natural behavior and
  observation and the latter has brain centric view in which the brain
  is regarded as the sole generator of behavior ignoring the fact that
  the animals is embedded in an environment that shapes its behavior.
  Our approach aims to provide a general framework of behavior . This
  framework can also be used to describe global brain activity in
  relation to behavior and the environment in which the animal is
  embedded. The challenge of finding basic principles can be guided by
  theoretical framework like the one we are proposing. The principles
  usually derived are independent from the environment or define a set
  of static or defined changes to the behavior in response to the change
  of the environment. It is more natural to regard the behavior as in a
  constant feedback loop with a changing environment that in order to
  properly studied, one should not study them separately. The concept of
  affordance is able to capture this dynamical interdependent
  relationship between the animal and its environment.
\item
  Elements of the theory = constraint, dynamics, memory, behavior,
  parameters that control the interaction of all the aforementioned (
  (we should have a figure with a conceptual cartoon explaining the
  basic building blocks of our framework : C,D, M, B)
\item
  Is closed loop necessary for behavior? behavior cannot be defined
  appropriately without reference to the environment in which the animal
  is embedded. This environment is in a closed loop interaction with the
  animal giving rise to ''behavior''. (I think it should be removed as a
  highlight from this article and save it to our other ideas concerning
  feedback).
\item
  Affordance-diffusion model (different hidden Markov model)
\item
  Map to thermodynamics quantities (which quantities should we
  calculate: work, entropy, causal entropy, etc..)
\item
  Principles - minimax principle (game theory, Nash equilibrium), min of
  total variation distance. If the minmax principle maps onto game
  theoretic concepts then in this case, our affordance - diffusion model
  can provide a framework for many social cooperation behaviors ?! and
  economical decision making in constrained environments ( relationship
  to bounded rationality).
\item
  Application of theory - Shaevitz, Gordon (main concepts from their
  papers are: studying of stereotype along with delineating it from
  non-stereotypical behavior, there is no mention of the environment but
  they talk about biomechanical limits, describing the behavioural
  states as periodic orbits, the aim of the unsupervised method that
  they developed is to uncover an underlying behavioral program, in
  fruit flies they found that there is a hierarchy of behaviors,
  multiple timescales, non-markovian dynamics )
\item
  Examples
\end{itemize}

preprint,amsmath,amssymb,aps,{]}revtex4-1

\section*{Discussion points}\label{discussion-points}

\begin{itemize}
\tightlist
\item
  In contrast to forcing, introducing an affordance process is a more
  natural way of doing it . The affordance process represents the
  external environment's interaction with the diffusion process. The
  diffusion process thus, encapsulates the internal dynamics of the
  animal.
\item
  Metastability:
\item
  Time scales : there is a lack of theoretical framework that tie
  different temporal scales and their contribution to the emergence of
  behavior.
\item
  Conceptual of model behavior (control theory, self-organizing
  processes, closed loop, dynamical system): A conceptual model defines
  the underlying abstraction used to designate what constitutes a
  behavior. Within a control theoretic framework, there is a target
  value / function to which the system tunes itself (it has some
  teleological soundings). The self organization framework , there is a
  spontaneous emergence of behaviors that are not driven by a control
  signal (contrary to the control theoretic process). Our framework
  belongs to the category of self organization.
\item
  artificial intelligence: Given that artificial agents will also be
  embedded in an environment either ''real'' or ''artificial'', the
  implementation of AI - environment loop (relation) is crucial. One way
  to achieve is via affordances that the AI learns through its
  interaction with the environment. We avoided using terms that are hard
  to define like intelligence and one that , unintentionally, put
  anthropomorphism at the center. Intelligence , with its working
  definition, is already implied in the action of closed loop selection
  of affordances in a particular environment.
\item
  Application in foraging, navigation, vocalization, decision making
  (drift diffusion): In a real world decision making process, the
  underlying drift rate changes continuously, our affordance - diffusion
  model allows us to model the decision making process as a drift -
  diffusion process in which the drift rate is changing ( according to
  the affordances in the environment). Same goes for the foraging that
  is often modeled using random search processes (random walk) and our
  model can extend the understanding of foraging in terms of the
  environmental constraints (where there is an affordance process
  interacting with the environment ) . In spatial navigation tasks, the
  change in the animals' behavior given the spatial constraints of the
  environment is not usually studied. Our model allows us to use
  arbitrary shapes of potential which extends the applicability of the
  system to many different behaviors.
\item
  our affordance - diffusion model clarifies to us why methods used to
  cluster / classify / segment behavior actually work (most famously
  nowadays the work of Josh Shaevitz and Gordon Berman)
\item
  Relation to statistical inference (simulated annealing)
\item
  What is adaptation
\item
  non-equilibrium thermodynamics (thermodynamics of prediction, causal
  entropic forces)
\item
  Discussion about the constraints
\end{itemize}

preprint,amsmath,amssymb,aps,{]}revtex4-1

\section*{Metastability in animal
behavior}\label{metastability-in-animal-behavior}

The usual way to model metastability is to divide the state space into
non-overlapping partitions and associate an exponentially small
transition probability between the partitions. In biology, such picture
is not always accurate. It is common to assume that the metastability
happens in the transition between potentials that are associated with
same state space. An obvious example is given by an animal that forages
in an arena. It is clear from an observer that when the animal interacts
with the wall (\emph{e.g.}, hits the wall), it changes its dynamics,
although still moving within the same arena. An observer would
characterize such change as transition between different animal
behaviors. It is then natural to assume that the transition between
animal behaviors corresponds to transition between metastable states.
Give some example for development and evolution.

Examples of metastability from physics / non linear dynamics and how the
definition differ concretely from biology

It is clear from this picture that there at least four time scales to
consider when trying to understand animal behavior. The fastest time
scale is the sum of several factor that are modeled by random noise. The
dynamics of the animal in the phase space, represents the second fastest
time scale. The third time scale is given by the change in the
parameters that controls the the dynamics of the animal. This change is
driven by the interaction between the animal dynamics and the
constraints of the system that represents the slowest time scale.

The importance of the experimental observation time in ''understanding''
behavior''

Behavior can be regarded as constrained dynamics with memory. Our
proposed model thus take into account constraints, dynamics and memory.
Constraints can be regarded as an abstract geometrical object that
delineate the boundaries spanned by the dynamics. The constraints should
be considered only as reflecting space but within our framework any
constraints (both environmental and biological ) can be mapped onto a
high dimensional geometrical object. The model is out of thermodynamical
equilibrium and non-markovian

The establishment of an Ising - like model for behavior. Ising models
have been successfully used to understand universal properties in many
classes of biological systems from gene networks to neuronal networks. A
similar approach with behavior is lacking that focuses on elucidating
universal principles of behavior.

Drawing from Cybernetic principles , behavior can be broadly defined /
understood as a feedback control system. In our case, it is the feedback
between the dynamics and constraints that shapes behavior to navigate
across different and select appropriate affordances. Affordances can be
defined as a potential action that is made possible to an agent by the
environment around it. While affordances are defined with respect to an
agent's individual capabilities (e.g., a tree branch might have a `
walkability affordance for a monkey, not necessarily for a sedentary
man), they are objective in the sense that they do not depend on whether
the agent perceives them, attends to them, or chooses to act on them,
and can be recognized by anyone familiar with the agent's motor
repertoire. The affordance landscape in our model is the set of all
possible actions defined over the constraints on which the particle /
agent is acting.

Contrast with energy landscape , energy landscape is static but the
affordance landscape is dynamic owing to the existence of a feedback
loop.

What is called affordance in the literature (Gibson, Jakob Van Uekell,
\ldots{}) refers to all action possibilities in the environment in
relation to the agent. In our model , the parameter space of a
represents the affordances (which control the possible behavior of the
animal in the environment in a closed loop fashion ) .

There is recent resurgence interest in closed loop approaches to
understanding the brain not only as an experimental approach but as a
framework to understand behavior.

preprint,amsmath,amssymb,aps,{]}revtex4-1

\section*{A model of animal behavior}\label{a-model-of-animal-behavior}

An animal behavior is a dynamical process whose trajectory is defined by
the intrinsic dynamics of the animal, the constraints of the
environment, and the mutual (closed loop) interaction between the animal
and environment that changes the animal's dynamics.

Here we introduce a formal model for such system. Let assume that the
intrinsic (without constrains) dynamic of the animal can be described by
a state space
\textbackslash{}(\textbackslash{}mathcal\{X\}\textbackslash{}) and an
Ito diffusion equation

\begin{equation}
X_{t}=X_{0}+\int_{0}^{t}f(X_{s},s,a_{s})ds+\sigma\int_{0}^{t}F(X_{s},s,a_{s})dW_{s},\\
 \end{equation}

where
\textbackslash{}(f:\textbackslash{}mathbb\{R\}\^{}\{n\}\textbackslash{}times\textbackslash{}mathbb\{R\}\textbackslash{}times\textbackslash{}mathbb\{R\}\^{}\{d\}\textbackslash{}to\textbackslash{}mathbb\{R\}\^{}\{n\}\textbackslash{}),
\textbackslash{}(F:\textbackslash{}mathbb\{R\}\^{}\{n\}\textbackslash{}times\textbackslash{}mathbb\{R\}\textbackslash{}times\textbackslash{}mathbb\{R\}\^{}\{d\}\textbackslash{}to\textbackslash{}mathbb\{R\}\^{}\{n\textbackslash{}times
m\}\textbackslash{}), and \textbackslash{}(W\_\{t\}\textbackslash{}) is
a \textbackslash{}(k\textbackslash{})-dimensional Brownian motion. We
call \textbackslash{}((a\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}) the affordance parameter process, or affordance
process for short, that represents the dynamic of the parameter and
defines what the animal can do.

Let
\textbackslash{}(C\textbackslash{}subset\textbackslash{}mathbb\{R\}\^{}\{d\}\textbackslash{})
be a set that represents the constraints of the dynamics of the animal
in the state space. If the dynamics of the animal
\textbackslash{}(X\_\{t\}\textbackslash{}) hits the constraint
\textbackslash{}(C\textbackslash{}) at time
\textbackslash{}(\textbackslash{}tau\textbackslash{}) and point
\textbackslash{}(x\_\{\textbackslash{}tau\}\textbackslash{}), the
dynamics are updated, following the equations

\begin{equation}
X_{t}=x_{\tau}+\int_{\tau}^{t}f(X_{s},s,a_{s})ds+\sigma\int_{\tau}^{t}F(X_{s},s,a_{s})dW_{s},\\
 \end{equation}

where \textbackslash{}(x\_\{\textbackslash{}tau\}\textbackslash{}) is
not necessarily equal to
\textbackslash{}(X\_\{\textbackslash{}tau\}\textbackslash{}) and

\begin{equation}
a_{t}=a_{0}+\tilde{W}_{t},\\
 \end{equation}

where
\textbackslash{}((\textbackslash{}tilde\{W\}\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}) is a
\textbackslash{}(d\textbackslash{})-dimensional Brownian motion
independent of \textbackslash{}((W\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}). We call the above system of equations
Affordance-Diffusion (AD) model. To make the process c\selectlanguage{ngerman}àdlàg the new
value of \textbackslash{}(X\_\{\textbackslash{}tau\}\textbackslash{}) is
set to be \textbackslash{}(x\_\{\textbackslash{}tau\}\textbackslash{}).

Let \textbackslash{}(\textbackslash{}tau\_\{0\}=0\textbackslash{}) and
\textbackslash{}(\textbackslash{}tau\_\{i\},i\textgreater{}0\textbackslash{})
are the times at which the AD model hits the constraint
\textbackslash{}(C\textbackslash{}) and
\textbackslash{}(x\_\{\textbackslash{}tau\_\{i\}\},i\textgreater{}0\textbackslash{})
are the starting points of the dynamics after hitting the constraint, we
can rewrite the AD model more compactly as

\textbf{(Affordance-diffusion process).}

\emph{The process \textbackslash{}((X\_\{t\},a\_\{t\})\textbackslash{})
defined by}

\begin{align}
 X_{t} &amp; =\sum_{i=0}^{\infty}\mathbbm{1}_{\{\tau_{i}\leq t<\tau_{i+1}\}}\left\{x_{\tau_{i}}+\int_{\tau_{i}}^{t}f(X_{s},s,a_{s})ds+\sigma\int_{\tau_{i}}^{t}F(X_{s},s,a_{s})dW_{s}\right\}, \\ 
 a_{t} &amp; =a_{0}+\sum_{i=0}^{\infty}\tilde{W}_{\tau_{i}}\mathbbm{1}_{\{\tau_{i}\leq t<\tau_{i+1}\}},\\
 \end{align}

\emph{is called affordance-diffuion process.}

\textbf{.}

\emph{It is usual in the diffusion process literature to define a domain
\textbackslash{}(D\textbackslash{}) as a subset of the state space and
consider the border of the domain
\textbackslash{}(\textbackslash{}partial D\textbackslash{}) as
representing the constraint. We define the constraint directly, as it
seems more natural from experimental view.}

\textbf{.}

\emph{The word affordance was introduced by Gibson (REF) to describe the
possibilities of an animal behavior given the constraints of the
environment. We are modeling the affordance not as a property of the
environment alone, but as a property of the animal and its constraint
altogether.}

Now, our main problem is how to define behaviors for the AD model. If
the animal dynamics does not depend on time and the affordance
parameter, then the natural way to define behavior is to first partition
\textbackslash{}(\textbackslash{}mathcal\{X\}\textbackslash{}setminus
C\textbackslash{}) based on some criteria and then define behavior as an
element of the partition. The question becomes less trivial when the
dynamics depend on time and the affordance parameter. In this case, we
can obviously associate a partition for each time and value of the
affordance parameter, but we will have to define when the dynamics and
the partitions can be considered equivalent. Hence, we will need the
following set of definitions:

\textbf{(Topological conjugacy).}

\emph{Let}

\begin{equation}
X_{t}=X_{0}+\int_{0}^{t}f(X_{s},s)ds,\\
 \end{equation}

\emph{and}

\begin{equation}
Y_{t}=Y_{0}+\int_{0}^{t}f^{\prime}(Y_{s},s)ds,\\
 \end{equation}

\emph{be two deterministic dynamics. We say that
\textbackslash{}(X\textbackslash{}) is conjugate to
\textbackslash{}(Y\textbackslash{}) if there is a homeomorphism
\textbackslash{}(h:\textbackslash{}mathcal\{X\}\textbackslash{}to\textbackslash{}mathcal\{X\}\textbackslash{})
such that
\textbackslash{}(f(h(x),t)=h(f\^{}\{\textbackslash{}prime\}(x,t))\textbackslash{})
for
\textbackslash{}(x\textbackslash{}in\textbackslash{}mathcal\{X\}\textbackslash{})
and \textbackslash{}(t\textbackslash{}geq 0\textbackslash{}).}

\textbf{(Conjugacy between diffusion processes).}

\emph{Let}

\begin{equation}
X_{t}=X_{0}+\int_{0}^{t}f(X_{s},s)ds+\sigma\int_{0}^{t}F(X_{s},s)dW_{s},\\
 \end{equation}

\emph{and}

\begin{equation}
Y_{t}=Y_{0}+\int_{0}^{t}f^{\prime}(Y_{s},s)ds+\sigma^{\prime}\int_{0}^{t}F^{\prime}(Y_{s},s)dW^{\prime}_{s},\\
 \end{equation}

\emph{be two diffusion processes. We say that both processes are
conjugate if the respective deterministic dynamics (i.e.,
\textbackslash{}(\textbackslash{}sigma=\textbackslash{}sigma\^{}\{\textbackslash{}prime\}=0\textbackslash{}))
are topologically conjugate. Furthermore, given
\textbackslash{}(P\textbackslash{}) a subset of the state space
\textbackslash{}(\textbackslash{}mathcal\{X\}\textbackslash{}), we say
that \textbackslash{}(X\textbackslash{}) and
\textbackslash{}(Y\textbackslash{}) restricted to
\textbackslash{}(P\textbackslash{}) is conjugate if the deterministic
dynamics restricted to \textbackslash{}(P\textbackslash{}) is
topologically conjugate.}

\textbf{(Pre-behaviors and pre-holo-behaviors).}

\emph{A pre-behavior is a restriction of the diffusion process to a
subset of
\textbackslash{}(\textbackslash{}mathcal\{X\}\textbackslash{}). A
pre-holo-behavior is the set of pre-behaviors associated with a
partition of
\textbackslash{}(\textbackslash{}mathcal\{X\}\textbackslash{}).}

\textbf{(Equivalence).}

\emph{We say that pre-holo-behaviors
\textbackslash{}(hB\_\{1\}\textbackslash{}) and
\textbackslash{}(hB\_\{2\}\textbackslash{}) are equivalent if (1) the
respective processes are conjugate and (2) for each pre-behavior in
\textbackslash{}(hB\_\{1\}\textbackslash{}) there exist a conjugate
pre-behavior in \textbackslash{}(hB\_\{2\}\textbackslash{}).}

\textbf{(Behaviors and holo-behaviors).}

\emph{The equivalence classes of pre-behaviors and pre-holo-behaviors
are called behaviors and holo-behaviors.}

These definitions are straightforward to understand if we think about a
dynamics of an animal behavior. Assume that the phase space that we are
considering is the position of the animal. The dynamics is the
locomotion of the animal. The constraints are the walls of the space in
which the animal is moving. Whenever the animal hits the wall, it
changes the dynamics. For example, the animal changes the velocity, or
concentrate the locomotion far away from the walls. It is natural to
assume that the locomotion before and after hitting the wall are still
the same behavior, although their dynamics changed. Using our
definitions, we can formalize this natural assumption by saying that the
locomotion before and after hitting the wall are in the same equivalence
class.

The interesting question is why the affordance is relevant? The answer

\textbf{(Adaptiveness of a dynamic with respect to the constraint).}

\emph{The equivalence classes of pre-behaviors and pre-holo-behaviors
are called behaviors and holo-behaviors.}

Metastable state is about the entire system

Degree of adaptation is about the entire system

Sometimes, the affordance can be exactly the behavior in the case that
there is only one behavior.

Affordance is the unit of behavior

\subsection*{Some examples of Affordance-Diffusion
model}\label{some-examples-of-affordance-diffusion-model}

\par\null

In this section, we will consider some specific examples of animal
behaviors that can be modeled by the AD model.

\textbf{(Locomotion).}

\emph{Consider an animal that explores on a circular open field with
radius \textbackslash{}(c\textgreater{}0\textbackslash{}). The position
of the animal in the plane is the state space. The constraint
\textbackslash{}(C=\textbackslash{}\{(x,y)\textbackslash{}in\textbackslash{}mathbb\{R\}\^{}\{2\}:x\^{}\{2\}+y\^{}\{2\}\textless{}c)\textbackslash{}),
represents the borders of the track. We assume
\textbackslash{}(V(x,y,a\_\{x\},a\_\{y\})=(a\_\{x\}x\^{}\{2\}+a\_\{y\}y\^{}\{2\})/2\textbackslash{}).
Whenever, the animal interacts with the border (e.g., by proprioception
or vision), the affordance parameter change and therefore the animal
dynamics or, equivalently, the potential changes. The model can be
written as}

\begin{align}
 dX_{t} &amp; =\frac{ax}{\mathbbm{1}\{X_{t}\notin C\}}dt+dW_{t}, \\ 
 da_{t} &amp; =d\tilde{W}_{t}\mathbbm{1}\{X_{t}\in\partial C\}.\\
 \end{align}

SHOW SIMULATIONS AND DISCUSS SOME IDEAS.

\textbf{(Foraging).}

\emph{Consider an animal that forages in a linear track. The state space
in this case is the distance between the animal and food source. Let
\textbackslash{}(d\textbackslash{}) be the position of the food with
respect to the center of the track. The constraint
\textbackslash{}(C=(-c-d,c-d)\textbackslash{}), for
\textbackslash{}(c\textgreater{}0\textbackslash{}), represents the
distance of the borders of the track from the food. We assume
\textbackslash{}(V(x,a)=ax\^{}\{2\}/2\textbackslash{}).}

SHOW SIMULATIONS AND DISCUSS SOME IDEAS.

\textbf{(Decision making).}

\emph{Here we consider the classical drift diffusion model. The state
space in this case is the position of the animal in the track. The
constraint \textbackslash{}(C=(-c,c)\textbackslash{}), for
\textbackslash{}(c\textgreater{}0\textbackslash{}), represents the
borders of the track. We assume
\textbackslash{}(V(x,a)=ax\textbackslash{}).}

SHOW SIMULATIONS AND DISCUSS SOME IDEAS.

\textbf{(Show the transition from one well to two wells).}

\emph{Two well example. Maybe in 2D.}

\subsection*{The 1D Affordance-Diffusion
model}\label{the-1d-affordance-diffusion-model}

\par\null

We will introduce a model -- 1D Affordance-Diffusion model -- that
captures the main ideas of our framework, but it is simple enough that
we can show several of its properties.

Let \textbackslash{}((X\_\{t\},a\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}) be a stochastic process whose dynamics is defined
as follows. We first fix a bounded open set
\textbackslash{}(C\textbackslash{}subset\textbackslash{}mathbb\{R\}\textbackslash{})
with border \textbackslash{}(\textbackslash{}partial C\textbackslash{}).
We also define the parametrized potential
\textbackslash{}(V:\textbackslash{}mathbb\{R\}\^{}\{2\}\textbackslash{}to\textbackslash{}mathbb\{R\}\textbackslash{}),
such that,
\textbackslash{}(\textbackslash{}frac\{\textbackslash{}partial
V(\textbackslash{}cdot,a)\}\{\textbackslash{}partial
x\}\textbackslash{}) is continuous for each
\textbackslash{}(a\textbackslash{}in\textbackslash{}mathbb\{R\}\textbackslash{}).
Let \textbackslash{}(X\_\{0\}=0\textbackslash{}),
\textbackslash{}(a\_\{0\}=0\textbackslash{}),
\textbackslash{}(\textbackslash{}sigma\textgreater{}0\textbackslash{}),
the 1D AD model is defined as

\begin{align}
 dX_{t} &amp; =-\frac{\partial V(X_{t},a_{t})}{\partial x}\frac{1}{\mathbbm{1}\{X_{t}\notin C\}}dt+\sigma dW_{t}, \\ 
 da_{t} &amp; =d\tilde{W}_{t}\mathbbm{1}\{X_{t}\in\partial C\}.\\
 \end{align}

The process \textbackslash{}((X\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}) is an overdamped diffusion on
\textbackslash{}(C\textbackslash{}), such that whenever it reaches the
boundary \textbackslash{}(\textbackslash{}partial C\textbackslash{}), a
small change in the potential happens. The process
\textbackslash{}((a\_\{t\})\_\{t\textbackslash{}geq 0\}\textbackslash{})
is a jump process that we call affordance process.
\textbackslash{}(\textbackslash{}mathbb\{P\}\textbackslash{}) is the
canonical measure associated with
\textbackslash{}((X\_\{t\},a\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}). We define the set of hitting times
\textbackslash{}(\textbackslash{}mathcal\{H\}\textbackslash{}) to be the
(random) set of index \textbackslash{}(t\textbackslash{}) for which
\textbackslash{}(X\_\{t\}\textbackslash{}) defined by eqs.
(\ref{eq:basic1}) and
(\ref{eq:basic2}) hits the boundary
\textbackslash{}(\textbackslash{}partial C\textbackslash{}). Observe
that because of the interaction between
\textbackslash{}((X\_\{t\})\_\{t\textbackslash{}geq 0\}\textbackslash{})
and \textbackslash{}((a\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}), both process are not Markovian separately,
although the joint process is Markovian. We also note that AD model is
not a hidden model, because \textbackslash{}(a\_\{t\}\textbackslash{})
is not independent of the past values of
\textbackslash{}(X\_\{t\}\textbackslash{}) given the past values of
\textbackslash{}(a\_\{t\}\textbackslash{}). In other words, the
dependence goes in both ways, representing the closed loop interaction
between the diffusion dynamics and the affordance.

In our context, the process
\textbackslash{}((X\_\{t\})\_\{t\textbackslash{}geq 0\}\textbackslash{})
represents the dynamics of an animal behavior in the state space and
\textbackslash{}(C\textbackslash{}) represents the set of constraints
that the dynamics of the animal has to satisfy. The interaction between
the animal and the constraint induces changes in the dynamics of the
animal, by changing the potential \textbackslash{}(V\textbackslash{})
parametrized by the affordance process
\textbackslash{}((a\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}).

We also define the auxiliary processes
\textbackslash{}((Y\_\{t\}\^{}\{b\})\_\{t\textbackslash{}geq
0\}\textbackslash{}) where
\textbackslash{}(Y\_\{0\}\^{}\{b\}=0\textbackslash{}) and

\begin{equation}
dY_{t}^{b}&amp;=-\frac{\partial V(Y_{t}^{b},b)}{\partial x}dt+dW_{t}.\\
 \end{equation}

These processes are simply the diffusion process with potential
\textbackslash{}(V\textbackslash{}) for a fixed parameter
\textbackslash{}(b\textbackslash{}) and without constraint.

\subsection*{Results}\label{results}

\par\null

In what follows, we will prove some results related to the AD model.

We will first show that the dynamics of animal behavior is
non-stationary and consequently never achieves equilibrium.

\textbf{.}

\emph{Let the parametrized potential \textbackslash{}(V\textbackslash{})
and constraint \textbackslash{}(C\textbackslash{}) be such that
\textbackslash{}(\textbackslash{}mathbb\{P\}(Y\_\{t\}\^{}\{b\}\textbackslash{}notin
C)\textgreater{}0\textbackslash{}) for all
\textbackslash{}(b\textbackslash{}). Then, the set
\textbackslash{}(\textbackslash{}mathcal\{H\}\textbackslash{}) has
infinite many elements
\textbackslash{}(\textbackslash{}mathbb\{P\}\textbackslash{})-almost
surely. Therefore the Affordance-Diffusion process is non-stationary.
Moreover,
\textbackslash{}(\textbackslash{}lim\_\{t\textbackslash{}rightarrow\textbackslash{}infty\}a\_\{t\}=\textbackslash{}infty\textbackslash{})
\textbackslash{}(\textbackslash{}mathbb\{P\}\textbackslash{})-a.s.}

\textbackslash{}noindentProof

. Let
\textbackslash{}(\textbackslash{}mathcal\{H\}\_\{t\}=\textbackslash{}mathcal\{H\}\textbackslash{}cap{[}0,t{]}\textbackslash{}).
We can write the dynamics of \textbackslash{}(a\_\{t\}\textbackslash{})
as

\begin{equation}
a_{t}=\sum_{k\in\mathcal{H}_{t}}\eta_{k},\\
 \end{equation}

where
\textbackslash{}(\textbackslash{}eta\_\{k\}\textbackslash{}sim\textbackslash{}mathcal\{N\}(0,1)\textbackslash{}).
Because
\textbackslash{}(\textbackslash{}mathbb\{P\}(Y\_\{t\}\^{}\{b\}\textbackslash{}notin
C)\textgreater{}0\textbackslash{}), we conclude that,
\textbackslash{}(\textbackslash{}mathcal\{H\}\textbackslash{}) has
infinite many elements
\textbackslash{}(\textbackslash{}mathbb\{P\}\textbackslash{})-a.s. This
implies that,
\textbackslash{}(\textbackslash{}lim\_\{t\textbackslash{}rightarrow\textbackslash{}infty\}\textbackslash{}sum\_\{k\textbackslash{}in\textbackslash{}mathcal\{H\}\_\{t\}\}\textbackslash{}eta\_\{k\}=\textbackslash{}infty\textbackslash{})
\textbackslash{}(\textbackslash{}mathbb\{P\}\textbackslash{})-a.s.
\textbackslash{}(\textbackslash{}blacksquare\textbackslash{})

\textbf{.}

\emph{The exploration model (Example
\ref{ex:exploration}) is an example that have
\textbackslash{}(\textbackslash{}mathbb\{P\}(Y\_\{t\}\^{}\{b\}\textbackslash{}notin
C)\textgreater{}0\textbackslash{}).}

We next investigate the distribution of elements of
\textbackslash{}(\textbackslash{}mathcal\{H\}\textbackslash{}). Let
\textbackslash{}(k\_\{i\}\textbackslash{}) be the
\textbackslash{}(i\textbackslash{})-th element of
\textbackslash{}(\textbackslash{}mathcal\{H\}\textbackslash{}). For
\textbackslash{}(i\textbackslash{}geq 1\textbackslash{}), define the
hitting intervals
\textbackslash{}(T\_\{i\}=k\_\{i+1\}-k\_\{i\}\textbackslash{}). We are
interested on calculating the distribution of
\textbackslash{}(T\_\{i\}\textbackslash{})'s.

\textbf{(---\cite{day1983exponential}---).}

\emph{If \textbackslash{}(C\textbackslash{}) is bounded and if for each
\textbackslash{}(a\textbackslash{}in\textbackslash{}mathbb\{R\}\textbackslash{})
it is contained in the basin of attraction of an isolated,
asymptotically equilibrium point of the deterministic system
\textbackslash{}(\textbackslash{}dot\{x\}=V(x,a)\textbackslash{}), then
we have for any \textbackslash{}(i\textbackslash{}) and
\textbackslash{}(x\_\{0\}\textbackslash{}in\textbackslash{}mathbb\{R\}\textbackslash{}),}

\begin{equation}
\lim_{\sigma\rightarrow 0}\mathbb{P}(T_{i}>s\mathbb{E})=\exp(-s)\\
 \end{equation} \textbackslash{}noindentRemark

. Use this to introduce some statistical mechanics concepts like
potential, etc.

\textbf{.}

\emph{We say that a process
\textbackslash{}((Y\^{}\{b\}\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{}) has scale \textbackslash{}(L\textbackslash{}) if
\textbackslash{}(T\^{}\{b\}=\textbackslash{}min\textbackslash{}\{k\textbackslash{}geq
1:Y\_\{k\}\^{}\{b\}\textbackslash{}notin
C\textbackslash{}\}\textbackslash{}) satisfies
\textbackslash{}(\textbackslash{}mathbb\{E\}\textgreater{}L\textbackslash{}).}

\textbf{.}

\emph{A affordance set of scale \textbackslash{}(L\textbackslash{}) is
the set
\textbackslash{}(\textbackslash{}mathcal\{A\}\^{}\{L\}=\textbackslash{}\{b\textbackslash{}in\textbackslash{}mathbb\{R\}:(Y\^{}\{b\}\_\{t\})\_\{t\textbackslash{}geq
0\}\textbackslash{};\textbackslash{};\textbackslash{}text\{has\}\textbackslash{};\textbackslash{};\textbackslash{}text\{scale\}\textbackslash{};\textbackslash{};\textbackslash{})L\textbackslash{}(\textbackslash{}\}\textbackslash{}).}

\textbf{.}

\emph{Let
\textbackslash{}(i\textbackslash{}in\textbackslash{}mathcal\{H\}\textbackslash{}).
If \textbackslash{}(T\_\{i\}\textgreater{}L\textbackslash{}), we say
that \textbackslash{}(a\_\{i\}\textbackslash{}) is a metastable state of
scale \textbackslash{}(L\textbackslash{}). We call
\textbackslash{}(\textbackslash{}mathcal\{M\}\^{}\{L\}\textbackslash{})
the set of all metastable states of scale
\textbackslash{}(L\textbackslash{}).}

We will show that most of metastable states of scale
\textbackslash{}(L\textbackslash{}) happens when the coefficients of the
animal behavior is a subset of affordance set of scale
\textbackslash{}(L\textbackslash{}).

\textbf{.}

\emph{We have}

\begin{equation}
\mathbb{P}(a_{i}\in\mathcal{M}^{sL}\setminus\mathcal{A}^{L})\leq\exp(-s).\\
 \end{equation} \textbackslash{}noindentProof

. If
\textbackslash{}(a\_\{i\}\textbackslash{}in\textbackslash{}mathcal\{M\}\^{}\{sL\}\textbackslash{}),
we have that
\textbackslash{}(T\^{}\{a\_\{i\}\}\textgreater{}sL\textbackslash{}). By
Theorem \ref{theo:exponential} we have

\begin{equation}
\mathbb{P}(T^{a_{i}}>s\mathbb{E})\leq\exp(-s).\\
 \end{equation}

If
\textbackslash{}(a\_\{i\}\textbackslash{}notin\textbackslash{}mathcal\{A\}\^{}\{L\}\textbackslash{}),
then \textbackslash{}(\textbackslash{}mathbb\{E\}\textbackslash{}leq
L\textbackslash{}), therefore,

\begin{equation}
\mathbb{P}(T^{a_{i}}>sL)\leq\exp(-s),\\
 \end{equation}

which ends the proof.
\textbackslash{}(\textbackslash{}blacksquare\textbackslash{})

\textbf{.}

\emph{We have}

\begin{equation}
\mathbb{P}\left(\frac{\sum_{a_{i}\in\mathcal{M}^{L}_{t}}\ind\{a_{i}\notin\mathcal{A}^{sL}\}}{\{a_{i}\in M^{L}_{t}\}}>\epsilon\right)\leq\exp(-\epsilon t/L).\\
 \end{equation} \textbackslash{}noindentProof

. Concentration of measures.
\textbackslash{}(\textbackslash{}blacksquare\textbackslash{})

These results justify the idea that
\textbackslash{}(\textbackslash{}mathcal\{A\}\textbackslash{}) is a good
set to define behaviors, \emph{i.e.},
\textbackslash{}(\textbackslash{}mathcal\{A\}\^{}\{L\}\textbackslash{})
is the set of coefficients of the dynamics of animal behavior in which
most of the phenomenas that are stable for at least duration
\textbackslash{}(L\textbackslash{}) happens.

\textbf{.}

\emph{The set of behaviors is a partition of the state space.}

In practice, in several cases the experimenters choose the partition
using some criteria of choice. Nevertheless, our theory suggests a
natural way to partition. We first define the average potential.

\textbf{.}

\emph{The average potential
\textbackslash{}(\textbackslash{}bar\{V\}(x)\textbackslash{}) is given
by}

\begin{equation}
\bar{V}(x)=\frac{1}{\mathbbm{1}\{x\notin C\}}\lim_{t\to\infty}\frac{\int_{0}^{t}V(x,a_{s})ds}{t}.\\
 \end{equation}

The marginal probability of
\textbackslash{}((a\_\{t\})\_\{t\textgreater{}0\}\textbackslash{}) is
given by \textbackslash{}(F\textbackslash{}).

\textbf{(Law of large numbers for Affordance - Diffusion process).}

\emph{We have}

\begin{equation}
\lim_{t\to\infty}\frac{\int_{0}^{t}V(x,a_{s})ds}{t}=\int_{\mathbb{R}}V(x,a)F(a)da.\\
 \end{equation} \textbackslash{}noindentProof

. Law of large numbers.
\textbackslash{}(\textbackslash{}blacksquare\textbackslash{})

Now, given the average potential, we can partition the state space by
fixing a scale \textbackslash{}(l\textbackslash{}) in the following way.

Let
\textbackslash{}(x\^{}\{*\}=\textbackslash{}arg\textbackslash{}max\textbackslash{}bar\{V\}(x)\textbackslash{}).
The holo-behavior of scale \textbackslash{}(l\textbackslash{}) is the
set
\textbackslash{}(hB=\textbackslash{}\{x\textbackslash{}in\textbackslash{}mathbb\{R\}:\textbackslash{}bar\{V\}(x*)-\textbackslash{}bar\{V\}(x)\textgreater{}l\textbackslash{}\}\textbackslash{}).
A behavior is a subset of \textbackslash{}(hB\textbackslash{}) that is
an interval and it is not contained in other intervals that are subset
of \textbackslash{}(hB\textbackslash{}).

\textbackslash{}phr

Give some examples of behaviors. The definition of behavior reflects the
variety of ways that an experimenter can define a behavior
\textbackslash{}phrA partition is always in the parameter space.
\textbackslash{}phrGive some examples of behaviors.

There exist a behavior at every scale.

\textbf{.}

\emph{The process
\textbackslash{}((a\_\{t\})\_\{t\textgreater{}0\}\textbackslash{}) is
metric isomorphic to
RW\textbackslash{}(\textbackslash{}otimes\textbackslash{})Po.}

\textbackslash{}noindentProof

. Let \textbackslash{}((T\_\{i\},a\_\{i\})\textbackslash{}) be the
hitting interval and the respective metastable state. We define the
rescaled hitting interval
\textbackslash{}(\textbackslash{}tilde\{T\}\_\{i\}=\textbackslash{}frac\{T\_\{i\}\}\{\textbackslash{}mathbb\{E\}\}\textbackslash{}).
By Theorem \ref{theo:exponential}, we have that
\textbackslash{}(\textbackslash{}tilde\{T\}\_\{i\}\textbackslash{}sim\textbackslash{})
Po. Also, by definition,
\textbackslash{}(a\_\{i\}\textbackslash{}sim\textbackslash{}) RW.
\textbackslash{}(\textbackslash{}blacksquare\textbackslash{})

\textbf{(Minimax principle of affordance).} \begin{align}
\lim_{\sigma\rightarrow 0}\sup_{a\in\mathbb{R}}\log\mathbb{E}= \\ 
\sup_{a}\inf_{t>0,y\in\partial C}\{J_{[0,t]}(\phi):\phi_{0}=x^{*},\phi_{t}=y\}\\
 \end{align}

\emph{where}

\begin{equation}
J_{[0,t]}(\phi)=\frac{1}{2}\int_{0}^{t}\left(\dot{\phi}_{s}-\frac{\partial V}{\partial x}(\phi_{s})\right)^{2}ds\\
 \end{equation}

Given two probability measures \textbackslash{}(Q\textbackslash{}) and
\textbackslash{}(R\textbackslash{}), the total variation distance
\textbackslash{}(d\_\{TV\}\textbackslash{}) between
\textbackslash{}(Q\textbackslash{}) and
\textbackslash{}(R\textbackslash{}) is given by
\textbackslash{}(d\_\{TV\}(Q,R)=\textbackslash{}sup\_\{A\textbackslash{}in\textbackslash{}Omega\}\textbar{}Q(A)-P(A)\textbar{}\textbackslash{}).

\textbf{.} \begin{equation}
\lim_{\sigma\rightarrow 0}\mathbb{E}.d_{TV}(\tilde{P_{a}},P_{a})=1.\\
 \end{equation} \textbf{.}

\emph{The dynamics that are most adapted to a constraint
\textbackslash{}(C\textbackslash{}) are the ones in which the mean
hitting time is largest.}

\textbf{.}

\emph{Variational principle for the GTB. Use Wentzell--Freidlin theory}

The state space is determined by what the researcher wants to understand
(the umwelt of the experimenter).

\textbf{.}

\emph{A behavior is a partition of the state space defined by the
potential. Therefore, for each affordance parameter, there is an
associate partition.}

\textbf{.}

\emph{The average potential can be used to partition the state space and
define an overall set of behaviors. This should be done or each scale of
affordance that we choose.}

In our model, we consider the internal perspective of the animal and not
the position of the external observer . The internal world constructed
by the constraints and dynamics giving rise to the specific umwelt of
the animal.

With this model, we can discuss the difference between adaptation and
optimization . Adaptation is to be regarded as the non - convergence but
if the potential converges then it is called optimization (relating it
then to simulated annealing) The illusion of adaptation ?! They
spontaneous emergence via affordances give the impression that
adaptation is the causal force behind shaping ''behavior''.

One possible way to think about this result is that the the interaction
between dynamics and constraint is a sequential hypothesis testing
process which converges toward minimization of type I and type II error
rates.

The difference between dynamics and constraint is just a difference in
time-scale. The constraint is the slow portion and the dynamics is the
fast portion.

Relationship to simulated annealing , stochastic variational inference

The information geometry of the Gamma distribution (information
geometric measures defined on the Riemann manifold)

Discussion on the use of these principles to design artificial agents
and thermodynamical framework for artificial intelligence

Feeling of understanding!! By construction because our behavior matches
as best of possible the constraint of the world.

\begin{align}
 dX_{t} &amp; =-\frac{\partial V(X_{t},a_{t})}{\partial x}\frac{1}{\mathbbm{1}\{X_{t}\notin C\}}dt+dW_{t}, \\ 
 da_{t} &amp; =\sigma d\tilde{W}_{t}\mathbbm{1}\{X_{t}\in\partial C\}.\\
 \end{align}

\selectlanguage{english}
\FloatBarrier
\end{document}

