[MGI30-P02] An experiment of high-bandwidth virtual remote storage (HbVRS) system on long fat network (LFN)
Science cloud is a cloud system specialized for data intensive/centric science, which is based on a concept of the fourth paradigm proposed by Jim Gray in 2009. However, only a few science cloud systems have ever yielded tremendous scientific results so far. High-bandwidth storage I/O is one of the important issues to be overcome for big data sciences. In the study, we propose a high-bandwidth virtual remote storage (HbVRS) tool using a distributed file system (Gfarm) and a UDP-based data transfer protocol (HpFP) [1]. The tool is based on our examination of parallel HpFP data transfer in 10 Gbps using a long-distance 10G network (long fat network: LFN) between Japan and USA crossing the Pacific. We installed an application to draw a set of time sequential graphic files using the tool on the NICT Science Cloud. We successfully read data files in order of time sequence from a virtual storage as fast as more than 20 Gbps. The present results suggest that client hosts connected with a long fat network will be able to access to big data stored in cloud storage wherever over the world it is located. An application is demonstrated using the HbVRS [1].
[1] Ken T. Murata, P. Pavarangkoon, K. Yamamoto, , Y. Nagaya, K. Muranaga, T. Mizuhara, A. Takaki, O. Tatebe, and E. Kimura, “Multiple streams of UDT and HpFP protocols for high-bandwidth remote storage system in long fat network,” in Proc. 7th IEEE Annu. Information Technology, Electronics and Mobile Communication Conf. (IEMCON), 2016. doi: 10.1109/IEMCON.2016.7746276
[1] Ken T. Murata, P. Pavarangkoon, K. Yamamoto, , Y. Nagaya, K. Muranaga, T. Mizuhara, A. Takaki, O. Tatebe, and E. Kimura, “Multiple streams of UDT and HpFP protocols for high-bandwidth remote storage system in long fat network,” in Proc. 7th IEEE Annu. Information Technology, Electronics and Mobile Communication Conf. (IEMCON), 2016. doi: 10.1109/IEMCON.2016.7746276