\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage{natbib}




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}



\usepackage{float}








\begin{document}

\title{Machine Learning in Mathematica! - Learning a map}



\author[1]{Midhun Parakkal Unni}%
\affil[1]{University of Exeter}%


\vspace{-1em}



  
  \date{May 26, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup








\sloppy


Mathematica newly introduced machine learning to its kitty. And I was
excited to see its capabilities! So I decided to test its RNN
abilities..to begin with (Really RNN!).

So here is a simple problem for Mathematica's RNN chain to learn - A
simple map 2x Mod 1.

\text{You start with a number say $x$ the next number in the series is $2x~ mod~ 1$. That multiplies the number by 2 and take only the decimal parts (remove the integer part)}

\texttt{Ex.\ x=1.2\ ,\ 2x\ mod\ 1\ =\ 0.4}

The question I tried to ask was that does the RNN learn the map after
some iterations?

\par\null

Here is the code for doing the training:

f {[}x\_{]} := Mod{[}2*x, 1{]};

\texttt{data\ =\ NestList{[}f,\ 0.1,\ 100{]};}

\texttt{n\ =\ 1;}

\texttt{training\ =}

\texttt{RandomSample{[}}

\texttt{List\ /@\ Most{[}\#{]}\ -\textgreater{}\ List@Last{[}\#{]}\ \&\ /@\ (Partition{[}data,\ n\ +\ 1,\ 1{]}){]};}

\texttt{net\ =\ NetChain{[}\{GatedRecurrentLayer{[}40{]},\ GatedRecurrentLayer{[}40{]},}

\texttt{GatedRecurrentLayer{[}40{]},\ GatedRecurrentLayer{[}40{]},}

\texttt{GatedRecurrentLayer{[}20{]},\ GatedRecurrentLayer{[}20{]},}

\texttt{GatedRecurrentLayer{[}20{]},\ GatedRecurrentLayer{[}20{]},\ LinearLayer{[}1{]}\},}

\texttt{"Input"\ -\textgreater{}\ \{n,\ 1\},\ "Output"\ -\textgreater{}\ 1{]}}

\texttt{trained\ =\ NetTrain{[}net,\ training,\ TargetDevice\ -\textgreater{}\ "CPU"{]}}

\texttt{trainedmap\ {[}x\_{]}\ :=\ Flatten{[}Append{[}Rest{[}x{]},\ trained{[}x{]}{]}{]};}

\par\null

Here is the plot of the result where predicted is in blue and ground
truth in yellow. I gave RNN an initial condition then its output was
used over and over again.

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/rnnsol/rnnsol}
\caption{{The predicted (in blue) the ground truth (in yellow). The solution
diverges as iterations pass.
{\label{930482}}%
}}
\end{center}
\end{figure}

And Now the deeper question did it actually learn the function?

\par\null\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/rnnand-original/rnnand-original}
\caption{{The True function in red and the learned one in blue.
{\label{247472}}%
}}
\end{center}
\end{figure}

The Red is the original function and the blue what RNN learns.

Can you make it better and learn the complete function!? That's for some
other time.

\selectlanguage{english}
\FloatBarrier
\end{document}

