Accurate fault detection and defect severity classification in photovoltaic (PV) modules are essential for predictive maintenance and reliable energy production. Electroluminescence (EL) imaging provides a fast, non-contact, and effective approach for identifying internal defects in PV cells, including cracks, inactive regions, and structural abnormalities. However, automatic severity classification remains challenging due to class imbalance and subtle visual differences between intermediate defect levels. This study investigated transfer learning using several convolutional neural network (CNN) architectures, including ResNet18, ResNet34, VGG16, VGG19, and EfficientNet-B0, for multi-class PV defect severity classification using publicly available EL images within a unified experimental framework. Among the individual models, VGG16 achieved the best test performance with an accuracy of 74.54% and a macro-F1 score of 0.56, indicating that single CNN models remain insufficiently robust for reliable multi-class severity classification, particularly under class imbalance and overlapping severity levels. To address these limitations, ensemble learning was employed to combine complementary CNN models, significantly improving classification performance to approximately 90% accuracy and 0.84 macro-F1. In addition, gradient-weighted class activation mapping analysis was used to provide qualitative insight into the regions of the EL images that influenced model predictions. The ensemble models produced visually consolidated and interpretable attention patterns across the representative samples compared with the individual models. Overall, the proposed ensemble-based framework provides a practical and computationally efficient approach for improving PV defect severity classification using EL imaging.