Japan Geoscience Union Meeting 2019

Presentation information

[J] Poster

M (Multidisciplinary and Interdisciplinary) » M-GI General Geosciences, Information Geosciences & Simulations

[M-GI37] Earth and planetary informatics with huge data management

Sun. May 26, 2019 10:45 AM - 12:15 PM Poster Hall (International Exhibition Hall8, Makuhari Messe)

convener:Ken T. Murata(National Institute of Information and Communications Technology), Rie Honda(Department of Science and Technology, System of Natual Science, Kochi University), Susumu Nonogaki(Geological Survey of Japan, National Institute of Advanced Industrial Science and Technology), Takeshi Horinouchi(Faculty of Environmental Earth Science, Hokkaido University)

[MGI37-P10] Experimental Evaluation of High-Speed File Transfer over JHPCN

*Praphan Pavarangkoon1, Ken T. Murata1, Kazunori Yamamoto1, Takamichi Mizuhara2, Yuya Kagebayashi2, Ayahiro Takaki2, Kazuya Muranaga3, Keiichiro Fukazawa4 (1.National Institute of Information and Communications Technology, Tokyo, Japan, 2.CLEALINK TECHNOLOGY Co., Ltd., Kyoto, Japan, 3.Systems Engineering Consultants Co., Ltd., Tokyo, Japan, 4.Academic Center for Computing and Media Studies, Kyoto University, Kyoto, Japan)

Keywords:High-Speed File Transfer, Big Data, JHPCN, HpFP

With the rapid growth of data amount, big data processing imposes a substantial burden on computation, storage, and communication in data centers. The file transfer tool is used to provide the users with the ease of transferring files across the network, e.g., between data centers. In practical long distance networks, the amount of packet loss is not negligible to seriously affecting the file transfer. In general, the file transfer tool is based on transmission control protocol (TCP) and user datagram protocol (UDP). To the best of our knowledge, however, no file transfer tools based on TCP and UDP have succeeded in higher throughput than 10 Gbps on long distance networks with packet loss. In this research, we introduce a high-speed data transfer protocol for inter-data center transport network, namely high-performance and flexible protocol 2 (HpFP2). The HpFP2 is an improved version of the HpFP and is more suitable for real network environments. To accommodate different usage scenarios, we complete four operating modes of HpFP2: aggressive, fair, fast-start, and modest modes. The aggressive mode is to maximize its own throughput without regard to fairness or network stability. The fair mode is to maintain the fairness among all network connections and balance the speed of each network connection by gradually increasing the amount of data transmitted until it finds the network’s maximum carrying capacity. The fast-start mode is to improve the properties of fair mode by providing a fast and stable experience. The modest mode is to improve the estimation of the fair transmission rate and prevent the rate oscillation which is occurred by the aggressive mode. Based on HpFP2, we implement a file transfer tool, called high-performance copy (HCP), to improve file transfer performance over Japan high performance computing and networking (JHPCN). The performance of our file transfer tool is evaluated using real datasets collected from supercomputer resources. The results show that the HCP achieves notably better performance than the conventional tool for file transfer over JHPCN.