To study how plants respond to their environment researchers use imaging phenotyping technologies. The use of image-based phenotyping has enabled researchers to analyse plants and produce data at a large scale. However, this large influx of data has created a 'big data' problem to emerge causing researchers to search for new innovative ways to tackle the challenges of processing their data in a reasonable timeframe. To address such issues, deep learning and data science techniques are being used to perform a comprehensive analysis. Here we use a Plant Screen™ compact system to image a series of barley plants using two different imaging sensors. This compact system contains an RGB top and side view camera and a hyperspectral visible near infrared (VNIR) camera. To streamline the processing and analysis of RGB and hyperspectral imaging, we are building a pipeline using a lightweight implementation of the U-Net architecture to improve the accuracy of semantic segmentation based on the raw images captured via the compact system. Several models were designed and developed, each of which was tailored to either the type of imaging sensor being used or the angle for which the images been provided were taken (e.g., top-down, side-view). Results showed that each model regardless of sensor or perspective produced an accuracy greater than 90% and could accurately segment cereal crops regardless of their size, shape or colour. These results demonstrate the feasibility of using DL models to semantically segment cereal crops imaged using either RGB or hyperspectral imaging sensors.