日本地球惑星科学連合2019年大会

講演情報

[J] ポスター発表

セッション記号 M (領域外・複数領域) » M-GI 地球科学一般・情報地球科学

[M-GI37] 情報地球惑星科学と大量データ処理

2019年5月26日(日) 10:45 〜 12:15 ポスター会場 (幕張メッセ国際展示場 8ホール)

コンビーナ:村田 健史(情報通信研究機構)、本田 理恵(高知大学自然科学系理工学部門)、野々垣 進(国立研究開発法人 産業技術総合研究所 地質情報研究部門 情報地質研究グループ)、堀之内 武(北海道大学地球環境科学研究院)

[MGI37-P10] Experimental Evaluation of High-Speed File Transfer over JHPCN

*Praphan Pavarangkoon1Ken T. Murata1Kazunori Yamamoto1Takamichi Mizuhara2Yuya Kagebayashi2Ayahiro Takaki2Kazuya Muranaga3Keiichiro Fukazawa4 (1.National Institute of Information and Communications Technology, Tokyo, Japan、2.CLEALINK TECHNOLOGY Co., Ltd., Kyoto, Japan、3.Systems Engineering Consultants Co., Ltd., Tokyo, Japan、4.Academic Center for Computing and Media Studies, Kyoto University, Kyoto, Japan)

キーワード:High-Speed File Transfer, Big Data, JHPCN, HpFP

With the rapid growth of data amount, big data processing imposes a substantial burden on computation, storage, and communication in data centers. The file transfer tool is used to provide the users with the ease of transferring files across the network, e.g., between data centers. In practical long distance networks, the amount of packet loss is not negligible to seriously affecting the file transfer. In general, the file transfer tool is based on transmission control protocol (TCP) and user datagram protocol (UDP). To the best of our knowledge, however, no file transfer tools based on TCP and UDP have succeeded in higher throughput than 10 Gbps on long distance networks with packet loss. In this research, we introduce a high-speed data transfer protocol for inter-data center transport network, namely high-performance and flexible protocol 2 (HpFP2). The HpFP2 is an improved version of the HpFP and is more suitable for real network environments. To accommodate different usage scenarios, we complete four operating modes of HpFP2: aggressive, fair, fast-start, and modest modes. The aggressive mode is to maximize its own throughput without regard to fairness or network stability. The fair mode is to maintain the fairness among all network connections and balance the speed of each network connection by gradually increasing the amount of data transmitted until it finds the network’s maximum carrying capacity. The fast-start mode is to improve the properties of fair mode by providing a fast and stable experience. The modest mode is to improve the estimation of the fair transmission rate and prevent the rate oscillation which is occurred by the aggressive mode. Based on HpFP2, we implement a file transfer tool, called high-performance copy (HCP), to improve file transfer performance over Japan high performance computing and networking (JHPCN). The performance of our file transfer tool is evaluated using real datasets collected from supercomputer resources. The results show that the HCP achieves notably better performance than the conventional tool for file transfer over JHPCN.