[M-GI30] [JJ] Earth and planetary informatics with huge data management

[MGI30-P02] An experiment of high-bandwidth virtual remote storage (HbVRS) system on long fat network (LFN)

*Kazuya Muranaga¹, Kazunori Yamamoto², Ken T. Murata², Takamichi Mizuhara³, Osamu Tatebe⁴, Masahiro Tanaka⁴, Eizen Kimura⁵ (1.Systems Engineering Consultants Co., LTD., 2.National Institute of Information and Communications Technology, 3.CLEALINKTECHNOLOGY Co.,Ltd., 4.University of Tsukuba, 5.Department of Medical Informatics Ehime Univ.)

Science cloud is a cloud system specialized for data intensive/centric science, which is based on a concept of the fourth paradigm proposed by Jim Gray in 2009. However, only a few science cloud systems have ever yielded tremendous scientific results so far. High-bandwidth storage I/O is one of the important issues to be overcome for big data sciences. In the study, we propose a high-bandwidth virtual remote storage (HbVRS) tool using a distributed file system (Gfarm) and a UDP-based data transfer protocol (HpFP) [1]. The tool is based on our examination of parallel HpFP data transfer in 10 Gbps using a long-distance 10G network (long fat network: LFN) between Japan and USA crossing the Pacific. We installed an application to draw a set of time sequential graphic files using the tool on the NICT Science Cloud. We successfully read data files in order of time sequence from a virtual storage as fast as more than 20 Gbps. The present results suggest that client hosts connected with a long fat network will be able to access to big data stored in cloud storage wherever over the world it is located. An application is demonstrated using the HbVRS [1].
[1] Ken T. Murata, P. Pavarangkoon, K. Yamamoto, , Y. Nagaya, K. Muranaga, T. Mizuhara, A. Takaki, O. Tatebe, and E. Kimura, “Multiple streams of UDT and HpFP protocols for high-bandwidth remote storage system in long fat network,” in Proc. 7th IEEE Annu. Information Technology, Electronics and Mobile Communication Conf. (IEMCON), 2016. doi: 10.1109/IEMCON.2016.7746276