Mapping fluvial hydromorphology is an important part of defining river habitat. Mapping via field sampling or hydraulic modelling is however time consuming, and mapping hydromorphology directly from remote sensing data may offer an efficient solution. Here we present a system for automated classification of fluvial hydromorphology based on a deep learning classification scheme applied to aerial orthophotos. Using selected rivers in Norway, we show how surface flow patterns (smooth or rippled surfaces versus standing waves) can be classified in imagery using a trained convolutional neural network. We show how integration of these classified surface flow patterns with information on channel gradient, obtained from airborne topographic LiDAR data, can be used to compartmentalize the rivers into hydromorphological units that represent the dominant flow features. Automated classifications were consistent with those produced manually. They were found to be discharge-dependent, showing the temporally dynamic aspect of hydromorphology. The proposed system is quick, flexible, generalizable, and free from researcher-subjectivity. The deep learning approach used here can be customized to provide more detailed information on flow features, such as delineating between standing waves and advective diffusion of air bubbles/foam, to provide a more refined classification of surface flow patterns, and the classification approach can be further advanced by inclusion of additional remote sensing methods that provide further information on hydromorphological features.