11:00 AM - 1:00 PM
[MGI34-P04] The importance of the measurement process in super-resolution and a toy model analysis of the back-projection
Keywords:back-projection, super-resolution, machine learning
In recent years, there has been a lot of research on super-resolution (SR) methods for images using machine learning techniques such as dictionary learning and sparse coding. In the framework of the algorithm, an operation called back-projection has been proposed to improve the reproducibility of SR.
During the machine-learning SR process, it is necessary to utilize the limited information in the low-resolution (LR) input data as much as possible. Back-projection is suggested to evaluate the validity of the information amplified in this process and to improve the reproducibility of the SR. In general, the image data we can obtain in a practical manner are degraded by measurement processes. Once we obtain an SR image, we can create a new LR image by down-sampling the SR image. The created LR image is expected to be identical to the LR input when the SR image is identical to the original high-resolution (HR) image, which we cannot actually hold in our hands. Therefore, back-projection operation updates the SR image to compensate for the difference between the newly obtained LR image and the LR input. However, we need to make some assumptions on down-sampling when it is difficult to accurately estimate the degrading effects of the measurement process, such as SR of topographical images. In this study, we evaluated the effect of difference between the actual degradation by the measurement process (I) and assumed down-sampling (II) during the update of the back-projection process with a toy model analysis.
For simplicity, we used one-dimensional synthetic data. The synthetic data were downgraded by two different methods (degrading process I), and the downgraded data were up-sampled to the original resolution by cubic interpolation (SR process). Subsequently, the down-sampling operators were applied to the up-sampled datasets again (degrading process II). The above process gave us four different pairs of LR data: two pairs where process I and process II are identical, and the two other pairs where they are different. By comparing the root-mean-square error (RMSE) between these updated HR data and the original data, we examined how the behaviour of the back-projection differs depending on the two degradation processes.
When the same degrading operator was applied in processes I and II, back-projection decreased the RMSE by around 5% compared to SR without back-projection. In contrast, when the different operators were applied, the RMSE improvement was less than 1%. This result confirms that it is important to estimate the accurate measurement processes to make the most of the back-projection updating. If process I is difficult to estimate, the problem on back-projection will come down to the model selection of measurement processes.
Future work includes (1) development of effective estimation methods for hyperparameter estimation used in back-projection and (2) evaluation by artificial model analysis using more complex models.
During the machine-learning SR process, it is necessary to utilize the limited information in the low-resolution (LR) input data as much as possible. Back-projection is suggested to evaluate the validity of the information amplified in this process and to improve the reproducibility of the SR. In general, the image data we can obtain in a practical manner are degraded by measurement processes. Once we obtain an SR image, we can create a new LR image by down-sampling the SR image. The created LR image is expected to be identical to the LR input when the SR image is identical to the original high-resolution (HR) image, which we cannot actually hold in our hands. Therefore, back-projection operation updates the SR image to compensate for the difference between the newly obtained LR image and the LR input. However, we need to make some assumptions on down-sampling when it is difficult to accurately estimate the degrading effects of the measurement process, such as SR of topographical images. In this study, we evaluated the effect of difference between the actual degradation by the measurement process (I) and assumed down-sampling (II) during the update of the back-projection process with a toy model analysis.
For simplicity, we used one-dimensional synthetic data. The synthetic data were downgraded by two different methods (degrading process I), and the downgraded data were up-sampled to the original resolution by cubic interpolation (SR process). Subsequently, the down-sampling operators were applied to the up-sampled datasets again (degrading process II). The above process gave us four different pairs of LR data: two pairs where process I and process II are identical, and the two other pairs where they are different. By comparing the root-mean-square error (RMSE) between these updated HR data and the original data, we examined how the behaviour of the back-projection differs depending on the two degradation processes.
When the same degrading operator was applied in processes I and II, back-projection decreased the RMSE by around 5% compared to SR without back-projection. In contrast, when the different operators were applied, the RMSE improvement was less than 1%. This result confirms that it is important to estimate the accurate measurement processes to make the most of the back-projection updating. If process I is difficult to estimate, the problem on back-projection will come down to the model selection of measurement processes.
Future work includes (1) development of effective estimation methods for hyperparameter estimation used in back-projection and (2) evaluation by artificial model analysis using more complex models.