In the rapid advancement in informatics, there is a study of generating a caption for an input image. In this study, we investigate the applicability of such method to auto-generate a radiological diagnosis report in Japanese from a medical image. We built a deep-learning report-generating model, consisting of convolutional layers and recurrent layers. For dataset, 5,000 reports previously diagnosed from 2015 through 2018, embedded with key-images are collected. In the collection, pairs of diagnosis statements and first key-image are acquired for training and validation test-set. After running 50 epochs of training, loss and convergence indicator has stayed around 0.46. With variation in diagnosis statement size and vocabulary size the loss has not changed. From our results, it is clear that the training has not converged with 5,000 reports.