论文标题
通过透明的Infiniband支持加速基于Netty的应用程序
Accelerating netty-based applications through transparent InfiniBand support
论文作者
论文摘要
许多大数据框架都用Java编写，例如Apache Spark，Flink和Cassandra。这些系统使用基于Java Nio的网络框架Netty。尽管这允许在传统以太网网络上进行快速的网络，但它不能完全利用现代互连的整个性能，例如Infiniband，提供100 GBIT/S等的带宽。在本文中，我们建议对Java库Hadronio的Netty支持，为基于Nio的Java应用程序提供透明的Infiniband支持。 Hadronio基于UCX，该UCX支持包括Infiniband在内的几个互连。我们提出了支持Netty的Hadronio扩展和优化。涵盖单线程和多线程的微分析的评估表明，Netty应用程序有可能达到低至5 US的往返时间，并完全利用高速NIC的100 GBIT/S带宽，而无需更改应用程序的源代码。我们还将Hadronio与传统插座以及LIBVMA进行了比较，结果表明，Hadronio对普通插座有很大的改进，并且在几种情况下可以超越表现性LIBVMA。
Many big-data frameworks are written in Java, e.g. Apache Spark, Flink and Cassandra. These systems use the networking framework netty which is based on Java NIO. While this allows for fast networking on traditional Ethernet networks, it cannot fully exploit the whole performance of modern interconnects, like InfiniBand, providing bandwidths of 100 Gbit/s and more. In this paper we propose netty support for hadroNIO, a Java library, providing transparent InfiniBand support for Java applications based on NIO. hadroNIO is based on UCX, which supports several interconnects, including InfiniBand. We present hadroNIO extensions and optimizations for supporting netty. The evaluations with microbenchmarks, covering single- and multi-threaded scenarios, show that it is possible for netty applications to reach round-trip times as low as 5 us and fully utilize the 100 Gbit/s bandwidth of high-speed NICs, without changing the application's source code. We also compare hadroNIO with traditional sockets, as well as libvma and the results show, that hadroNIO offers a substantial improvement over plain sockets and can outperform libvma in several scenarios.
