[M-GI26] Earth and planetary informatics with huge data management

[MGI26-03] Performance Enhancement of High-Speed Data Transfer in JHPCN

*Praphan Pavarangkoon¹, Ken T. Murata¹, Kazunori Yamamoto¹, Takamichi Mizuhara², Yuya Kagebayashi², Ayahiro Takaki², Kazuya Muranaga³, Eizen Kimura⁴ (1.National Institute of Information and Communications Technology, 2.CLEALINK TECHNOLOGY Co., Ltd., 3.Systems Engineering Consultants Co., Ltd., 4.Ehime University)

Keywords:High-Speed Data Transfer, Big Data, JHPCN, HpFP

Nowadays, a variety of social and academic fields such as remote sensing systems, satellites for disaster resiliency, and 4K/8K material transmission system require high-speed data transfer over long distance networks. In many projects, the data transfer application is occasionally allowed to occupy a specified wide-band network without any congestions. In many practical long distance networks, the amount of packet loss is not negligible to seriously affecting the data transfer. Tolerance to both latency and packet loss is an expected ability to the protocol rather than congestion controls. Several data transfer protocols, which are based on either transmission control protocol (TCP) or user datagram protocol (UDP), have been introduced for high bandwidth networks. However, only a few protocols have ever succeeded in higher throughput than 10 Gbps on long distance networks with packet loss, thus many effective applications have not yet introduced in these fields. To achieve high throughput even under the condition of latency and packet loss, this research presents a novel transport protocol based on the UDP, namely high-performance and flexible protocol (HpFP). The HpFP puts more focus on latency and packet loss tolerances than fairness and friendliness, and provides an open source socket library working on user-land for multi-OS environments. We first introduce the fundamental concepts and control algorithms to realize the HpFP. Then, we develop a high-speed file transfer tool based on the HpFP, namely high-speed copy tool (HCP). The performance of our tool is evaluated on Japan high performance computing and networking (JHPCN). The results show that the HCP achieves better performance than the traditional tools in JHPCN. We believe that this tool becomes useful to improve the end-to-end high-speed networking data transfer capabilities in collaborative big data science among eight centers of JHPCN.