Semiconductor quantum dots (QDs) are a promising host for quantum computers because of their scalability. However, as the number of QDs grows, the time required to tune the potential increases, hampering scaling up. Machine learning is a promising approach to automate and expedite this tuning process. We propose to use model-based reinforcement learning (MBRL) for auto-tuning QDs. MBRL is expected to offer more generality because it models the environment and can divert the constructed model for other tasks. However, it remains to be seen whether the environment model can be constructed properly despite the sparse characteristic of QDs. In this work, we investigate the applicability of MBRL in this regard by emulating auto-tuning of a QD device to a single QD condition using MBRL on pre-measured data.