Soil erosion decreases soil productivity and threats global food security. Various relationships between crop yield and soil erosion were found for different regions, but a comprehensive global synthesis of the relationship is lacking. To quantify crop yield and soil erosion, we conducted a meta-analysis with a hierarchical mixed-effects model based on global-scale studies, and incorporating erosion depth, crop yield, grain type, soil type, measure, and other factors. The results confirmed that crop yield was negatively affected by soil erosion, but the decreasing trend of crop yield with erosion thickness was different. Crop yields did not decreased significantly when erosion depth was < 5 cm. When erosion depth was > 20 cm, crop yield clearly decreased no matter what soil management measures were adopted, which indicated that the degradation of soil productivity could not be restored. In addition, the effect degree of erosion was significantly different between 5 - 10 cm and 10 - 15 cm erosion depth intervals. We conclude that a widely used linear relationship does not well describe the relationships between crop yield and soil erosion, and we suggested that the crop yield response curve was concave when erosion depth was > 5 cm.