Cancer is one of the most prevalent diseases worldwide. The most prevalent condition in women when aberrant cells develop out of control is breast cancer. Breast cancer detection and classification are exceedingly difficult tasks. As a result, several computational techniques, including k-nearest neighbor (KNN), support vector machine (SVM), multilayer perceptron (MLP), decision tree (DT), and genetic algorithms, have been applied in the current computing world for the diagnosis and classification of breast cancer. However, each method has its own limitations to how accurately it can be utilized. A novel convolutional neural network (CNN) model based on the Visual Geometry Group network (VGGNet) was also suggested in this study. The 16 layers in the current VGGNet-16 model lead to overfitting on the training and test data. We, thus, propose the VGGNet-12 model for breast cancer classification. The VGGNet-16 model has the problem of overfitting the breast cancer classification dataset. Based on the overfitting issues in the existing model, this research reduced the number of different layers in the VGGNet-16 model to solve the overfitting problem in this model. Because various models of the VGGNet, such as VGGNet-13 and VGGNet-19, were developed, this study proposed a new version of the VGGNet model, that is, the VGGNet-12 model. The performance of this model is checked using the breast cancer dataset, as compared to the CNN and LeNet models. From the simulation result, it can be seen that the proposed VGGNet-12 model enhances the simulation result as compared to the model used in this study. Overall, the experimental findings indicate that the suggested VGGNet-12 model did well in classifying breast cancer in terms of several characteristics.