The study of segmenting breast lesions from ultrasound images is crucial to breast cancer diagnosis and treatment. In this study, we propose the Inception-Multi-Scale-Attention U-Net (IMSA-Net) method to address the challenges associated with breast cancer imaging artifacts, tumor morphological alterations, and blurred borders. The IMSA-Net method incorporates the inception module into the downsampling feature extraction section of the U-Net model. This can extract advanced features from the input image and enhance the model’s capability to express complex features. Furthermore, during the feature fusion stage, a multi-scale attention structure is introduced to focus on capturing detailed information during the segmentation process. This enhancement contributes to improving the precision of the segmentation results. To enhance the algorithm’s non-linear factor and mitigate the gradient disappearance problem during training, the Gaussian Error Linear Unit (GELU) activation function is employed, replacing the traditional Rectified Linear Unit (ReLU) activation function. This modification allows for better feature representation and overall model performance. The experimental results demonstrate that the IMSA-Net model’s evaluation metrics of mIoU, Dice, Acc, Precision, and Recall on the breast ultrasound images dataset reached 74.18%, 77.98%, 96.86%, 89.23%, and 66.68%, respectively. These results outperform those of the U-Net model by 2.41%, 2.81%, 0.09%, 0.06%, and 2.82%. Overall, the IMSA-Net method tackles the difficulties related to breast lesion segmentation, assisting in subsequent breast disease diagnosis.