Congenital heart disease (CHD) has been one of the most serious problems in newborns. For fetal heart health care, 3D modeling and printing technology has been adopted in diagnosis of CHD during antenatal care. However, the development of 3D printing techniques and their applications in clinic have been hindered by the manually processing of ultrasound (US) volume data in clinical practice. To overcome this problem, we present an interactive semi-automatic method based on deep learning that uses manually processing results from expert sonographers for training. The accuracy, interpretability, and variability of the performances were evaluated on the validation set demonstrated that, compared with a physician with three years’ experience, Faster-RCNN-based Threshold (FRT) had achieved better performance on outflow tracts view (OTV) and three-vessel view (TVV). No significant difference was found among the clinical evaluation values, in proportion, measured from the model rebuilt by FRT and that from US volume data. Furthermore, reconstruction time of fetal heart blood pool model was reduced from approximately 5 hours to 5 minutes. Our results showed that deep learning has the ability to process US data accurately, representing an important step towards the reconstruction of the fetal heart digital model, which may make huge progress in clinical diagnosis and treatment of CHD during pregnancy.