Compute node pipeline monitor and control interface

pipelinefs

Layout of pipelinefs filesystem
Usage

The monitor and control interface to the compute node pipelines is based on the control interface described in the section called “Control interface”. The interface on the head node has been enhanced somewhat from the interface described in the section called “Control interface” to provide greater simplicity as well as a single control path from the head node to all compute node pipelines. This interface effectively takes the form of a sort of “device browser”, which is, unlike the usual device browser, integrated with the operating system on the head node through a special filesystem. The advantages of this design are that the message passing interface to the compute node pipelines is hidden by an interface that instead uses reading and writing files and directories on the head node, and that the interface is readily available to all users (human and otherwise) on the head node by virtue of its integration with the operating system. By this design, the API used by monitor and control programs on the head node is a subset of the standard libc library functions, such as read, write, creat, rmdir, etc.

The program that implements the interface is named pipelinefs, which is also the name of the filesystem that the program creates. pipelinefs is based on the FUSE library, which is used to implement filesystems in userspace under Linux (and other operating systems).[1] pipelinefs itself is written in the Python programming language, which was chosen mainly for reasons of expediency; the program could be reimplemented in another language, if that became desirable, without requiring any changes in programs that use the pipelinefs filesystem interface.



[1] FUSE has been adopted by the official Linux kernel starting at version 2.6.14. If you are using an earlier kernel version, FUSE likely will work (see the FUSE web site for details about usable kernel versions), but you will need to download, build and install it yourself. Note that FUSE depends on a loadable kernel module, which probably requires superuser access to install on your system.