\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage[round]{natbib}
\let\cite\citep




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}



\usepackage{float}






\usepackage{listings}
\usepackage{color}
 
\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}
 
\lstdefinestyle{mystyle}{
    backgroundcolor=\color{backcolour},   
    commentstyle=\color{codegreen},
    keywordstyle=\color{magenta},
    numberstyle=\tiny\color{codegray},
    stringstyle=\color{codepurple},
    breakatwhitespace=false,         
    breaklines=true,                 
    captionpos=b,                    
    keepspaces=true,                 
    numbers=left,                    
    numbersep=15pt,                  
    showspaces=false,                
    showstringspaces=false,
    showtabs=false,                  
    tabsize=2,
    language=Python,
    basicstyle=\linespread{0.5},
}
 
\lstset{style=mystyle}

\begin{document}

\title{German Traffic Sign Classifier -- CarND assigment}



\author[1]{Alberto Escarlate}%
\affil[1]{Udacity Self Driving Car Nanodegree}%


\vspace{-1em}



  
  \date{May 08, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup





\selectlanguage{english}
\begin{abstract}
This is the submission for the~Traffic Sign Classifier project part of
Udacity Self-Drive Engineer Nanodegree. The assignment was to train a
Convolutional Neural Network (CNN) model to classify traffic signs from
the~\href{http://benchmark.ini.rub.de/?section=gtsrb\&subsection=dataset}{German
Traffic Sign Dataset}.~

Using a model based on the LeNet architecture enabled my classifier to
achieve the required 93\% training accuracy.~%
\end{abstract}%



\sloppy


\section*{Goals}\label{goals}

The goals of the project were:

\begin{itemize}
\tightlist
\item
  Load the data set (see below for links to the project data set)
\item
  Explore, summarize and visualize the data set
\item
  Design, train and test a model architecture
\item
  Use the model to make predictions on new images
\item
  Analyze the softmax probabilities of the new images
\end{itemize}

\section*{Dataset Exploration}\label{dataset-exploration}

\subsection*{Data structrure}\label{data-structrure}

The data, from the
\href{http://benchmark.ini.rub.de/?section=gtsrb\&subsection=dataset}{German
Traffic Sign} dataset,is a pickled dictionary with 4 key/value pairs:

\begin{itemize}
\tightlist
\item
  \texttt{features}: 4D array containing raw pixel data of the traffic
  sign images, (num examples, width, height, channels).
\item
  \texttt{labels}: 1D array containing the label/class id of the traffic
  sign. The file signnames.csv contains id -\textgreater{} name mappings
  for each id.
\item
  \texttt{sizes}: list containing tuples, (width, height) representing
  the original width and height the image.
\item
  \texttt{coords}: list containing tuples, (x1, y1, x2, y2) representing
  coordinates of a bounding box around the sign in the image.
\end{itemize}

\subsection*{Dataset contents}\label{dataset-contents}

The dataset provided for this assignment already contained a subset of
images cropped and resized to 32 x 32 pixels so we had to consider only
\texttt{features} and \texttt{labels}.

The dataset provided for this assignment already contained a subset of
images cropped and resized to 32 x 32 pixels so we had to consider only
\texttt{features} and \texttt{labels}.

Examining the data we could find the following numbers:

\begin{itemize}
\tightlist
\item
  Number of training examples: 34,799
\item
  Number of testing examples: 12,630\\
\item
  Image data shape: (32, 32, 3)
\item
  Number of classes: 43
\end{itemize}

\subsection*{Classes}\label{classes}

Another file provided was a list of classes with their respective names.
Here we display 5 rows randomly picked.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Screenshot-2017-08-21-22-44-561/Screenshot-2017-08-21-22-44-561}
\caption{{This is a caption
{\label{231257}}%
}}
\end{center}
\end{figure}

\pagebreak

And here are some training images for each of the 43 classes.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/figure2/figure2}
\caption{{Display of all classes with a sample training image.
{\label{430524}}%
}}
\end{center}
\end{figure}

Inspecting the training dataset we notice, in the histogram image below, that the classes aren't equally balanced. With a considerable number of classes having less than 500 samples the model can have its testing accuracy impacted.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/figure3/figure3}
\caption{{Distribution of the training samples
{\label{527151}}{\label{527151}}%
}}
\end{center}
\end{figure}

\pagebreak

\section*{Model Architecture}\label{model-architecture}

\subsection*{CarND model vs.~LeNet}\label{carnd-model-vs.lenet}

The model architecture I implemented \emph{CarND} was similar to the the
\emph{LeNet} architecture \cite{lecun1998gradient} with a slight modification.
I included dropout layers after each of the Fully Connected layers and
avoid overfitting.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/figure5/figure5}
\caption{{Comparing CarND model with LeNet
{\label{355794}}%
}}
\end{center}
\end{figure}

As a last step, I implemented a few image processing techniques and test
whether these would make the model increase its accuracy. The techniques
were:

\begin{itemize}
\tightlist
\item
  Image normalization
\item
  Histogram equalization
\item
  Gaussian blur
\item
  Image sharpening
\item
  Downsampling to grayscale
\end{itemize}

Overall the transformations didn't change the accuracy significantly, so
I decided to proceed with only downsampling the images to grayscale.
That was enough to get to the required 93\% accuracy.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.98\columnwidth]{figures/Screenshot-2017-08-21-23-18-19/Screenshot-2017-08-21-23-18-19}
\caption{{Comparing validation accuracy when using image transformations in the
data set
{\label{828650}}%
}}
\end{center}
\end{figure}



Starting with LeNet already put me in a good position to iterate the
model and adjust the hyperparameters. From previous experience in the
\textbf{Deep Learning Foundation nanodegree} I knew I should play with
adding new layers and dropout after the fully connected layers. Then I
had to iterate the hyperparameters to fine tune the results. One
hyperparameter that got me stuck for some time was the batch size, I
assumed that a large batch would work better and only when I decreased
from 256 to 128 and then to 64 that I got favorable accuracies.

\subsection*{Hyperparameters}\label{hyperparameters}

\begin{itemize}
\tightlist
\item
  Epochs = 100
\item
  Batch size = 64
\item
  Dropout keep probability = 0.50
\item
  Learning rate = 0.0005
\end{itemize}

\subsubsection*{Weight initialization}\label{weight-initialization}

\begin{itemize}
\tightlist
\item
  \(\mu\) = 0
\item
  \(\sigma\) = 0.1
\end{itemize}

\subsection*{Achieved Accuracy}\label{achieved-accuracy}

\begin{itemize}
\tightlist
\item
  Validation Accuracy = 0.942
\item
  Test Accuracy = 0.925
\item
  Training Accuracy = 1.000
\end{itemize}

\section*{Testing the model with new
images}\label{testing-the-model-with-new-images}

Once the model was stable, I found a set of images of German traffic
signs on the Internet to test how it would predict the traffic sign
classes.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=1.00\columnwidth]{figures/Screenshot-2017-08-21-22-52-55/Screenshot-2017-08-21-22-52-55}
\caption{{Signs found on the Internet
{\label{283701}}%
}}
\end{center}
\end{figure}

The signs downloaded from the Internet (figure \ref{283701}) were then
tested against the trained model. Running the prediction had an accuracy
of 0.833. The images I chose are likely to have better results as they
are well centered and have no background that could mislead the model.
The only prediction that failed to predict was very similar to the
actual image (a triangular sign with red border and white interior).\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.84\columnwidth]{figures/Screenshot-2017-08-22-00-26-43/Screenshot-2017-08-22-15.53.50}
\caption{{Prediction for images found on the Internet
{\label{479347}}{\label{479347}}%
}}
\end{center}
\end{figure}

Calculating the top 5 Softmax probabilities for each image the model
strongly predicted the top choices with probabilities getting almost all
close to 100\%. The model accuracy in the small set of five samples got
to 83.3\% compared to 92.5\% for the test set accuracy.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=1.00\columnwidth]{figures/Screenshot-2017-08-22-00-39-37/download}
\caption{{First column shows the correct class. Columns 2-6 show the predicted
class sorted by probability.
{\label{553169}}%
}}
\end{center}
\end{figure}

In the test showed above the two incorrect predictions had the correct
class in the top 5. Besides the top choice is quite similar visually to
the correct class.

\selectlanguage{english}
\FloatBarrier
\bibliographystyle{plainnat}
\bibliography{bibliography/converted_to_latex.bib%
}

\end{document}

