论文标题

Kiwipy:强大的,大量的,大数据和计算科学工作流程的消息传递

kiwiPy: Robust, high-volume, messaging for big-data and computational science workflows

论文作者

Uhrin, Martin, Huber, Sebastiaan P.

论文摘要

在这项工作中,我们介绍了Kiwipy,这是一个python库Kiwipy,旨在支持基于强大的消息的高通量,大数据,应用程序,同时足够一般,可以在需要以可预测的方式进行通信的任何地方进行通用。 Kiwipy依靠RabbitMQ协议,这是一个行业标准消息代理,同时提供了一个简单而直观的界面,可以在基于多线程和基于Coroutine的应用程序中使用。为了演示Kiwipy的一些功能,我们从Aiida(高通量模拟平台Aiida)提供了示例,其中Kiwipy用作工作流引擎的关键组成部分。

In this work we present kiwiPy, a Python library designed to support robust message based communication for high-throughput, big-data, applications while being general enough to be useful wherever high-volumes of messages need to be communicated in a predictable manner. KiwiPy relies on the RabbitMQ protocol, an industry standard message broker, while providing a simple and intuitive interface that can be used in both multithreaded and coroutine based applications. To demonstrate some of kiwiPy's functionality we give examples from AiiDA, a high-throughput simulation platform, where kiwiPy is used as a key component of the workflow engine.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源