Last February, SNIA hosted its one-day Persistent Memory Summit in San Jose; it was my pleasure to be invited to participate by delivering a presentation on behalf of the OpenFabrics Alliance. The day long program was chock full of deeply technical, detailed information about the state of the art in persistent memory technology coupled with previews of some possible future directions this exciting technology could conceivably take. The Summit played to a completely packed house, including an auxiliary room equipped with a remote video feed. Quite the event!
But why would the OpenFabrics Alliance (the OFA) be offering a presentation at a Persistent Memory (PM) Summit, you ask? Fabrics! Which just happens to be the OFA’s forte.
For several years now, SNIA’s NVM Programming Model Technical Working Group (NVMP TWG) has been describing programming models designed to deliver high availability, the primary thesis for which is simply stated – data isn’t truly ‘highly available’ until it is stored persistently in at least two places. Hence the need to access remote persistent memory via a fabric in a highly efficient, and performant, manner. And that’s where the OFA comes in.
For those unfamiliar with us, the OFA concerns itself with developing open source network software to allow applications to get the most performance possible from the network. Historically, that has meant that the OFA has developed libraries and kernel modules that conform to the Verbs specification as defined in the InfiniBand Architecture specifications. Over time, the suite has expanded to include software for derivative specifications such as RoCE (RDMA over Converged Ethernet) and iWARP (RDMA over TCP/IP). In today’s world, much of the work of maintaining the Verbs API has been assumed by the open source community itself. Success!
Several years ago, the OFA began an effort called the OpenFabrics Interfaces project to define a network API now known as ‘libfabric’. This API complements the Verbs API; Verbs continues into the foreseeable future as the API of choice for verbs-based fabrics such as InfiniBand. The idea was that the libfabric API would be driven mainly by the unique requirements of the consumers of network services. The result would be networking solutions that are transport independent and that meet the needs of application and middleware developers through a freely available open source API.
So, what does all this have to do with persistent memory? A great deal!
By now, we have come to the realization that remote, fabric attached persistent memory, while very much like local memory in many respects, has some unique characteristics. To state the obvious, it has persistence characteristics akin to those found in classical file systems, but it has the potential to be accessed using fast memory semantics instead of conventional file-based POSIX semantics. Accomplishing this implies a need for new features exposed to consumers through the API, giving the consumer greater control over the persistence of data written to the remote memory. Fortunately, the libfabric framework was designed from the outset for flexibility, which should make it straightforward to define the new structures needed to support Persistent Memory accesses over a high performance, switched fabric.
My presentation at the Persistent Memory Summit had two main goals; the first was to introduce the OpenFabrics Alliance’s approach to API development. The second was to begin the discussion of API requirements to support Persistent Memory. For example, during the talk we drew a distinction between ‘Data Storage’ (what it means to access conventional storage) versus ‘Data Access’ (accessing persistent memory over a fabric). As the slides in the presentation make clear, these are two very different use cases, and yet the enhanced libfabric API must support both equally well. At the end of the presentation, we presented some ideas for what a converged I/O stack designed to support both use cases might look like. Naturally, this is just the beginning of the story.
There is much work to do. The work on the libfabric API is now underway in earnest in the OpenFabrics Interfaces Working Group (OFIWG – https://www.openfabrics.org/index.php/working-groups-overview.html#ofiwg), and as with the original libfabric work, we are beginning with a requirement gathering phase. We want to be sure that the resulting enhancements to the libfabric API meet the needs of applications accessing remote persistent memory.
Being an open source organization, the OFA welcomes input from all interested parties in our efforts to support the emergence of this exciting new technology. For more information on how to get involved, please go to the OpenFabrics website (https://www.openfabrics.org) to find information about regular working group meetings and how you can get involved. Or, feel free to reach out to me directly for more information – grun@cray.com
