From f6e151dd81dd097bea71cc5f50b5d9e4fe782720 Mon Sep 17 00:00:00 2001 From: Sean Hefty Date: Thu, 25 Oct 2012 00:11:56 -0700 Subject: [PATCH] refresh --- meta | 7 ++-- patches/refresh-temp | 10 ------ patches/rs-iomap | 79 +++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 81 insertions(+), 15 deletions(-) delete mode 100644 patches/refresh-temp diff --git a/meta b/meta index 5a0bbf60..8112ba0f 100644 --- a/meta +++ b/meta @@ -1,9 +1,8 @@ Version: 1 -Previous: 547f64604b2ca57ed551357484ec05a86f3b3000 -Head: 8d119518a58362736417c623784f3ac0309d594b +Previous: ea39febef7030f34141385fc938bdb77c8150020 +Head: 5a6fcf8af7162d16274556981d27be74f8a43295 Applied: - rs-iomap: e5a8d2c61f31c60bf37aa3ad41219bdf62e08e57 - refresh-temp: 8d119518a58362736417c623784f3ac0309d594b + rs-iomap: 5a6fcf8af7162d16274556981d27be74f8a43295 Unapplied: riostream: 1ba9685f351cc59e13a02f4328cc78dbe00e0505 iom-dbg: 88434072d07f8edc58f454ac954d78bd39441eed diff --git a/patches/refresh-temp b/patches/refresh-temp deleted file mode 100644 index a567563e..00000000 --- a/patches/refresh-temp +++ /dev/null @@ -1,10 +0,0 @@ -Bottom: 915e0ee4505ffa5dc789b46752172eee702adae3 -Top: 915e0ee4505ffa5dc789b46752172eee702adae3 -Author: Sean Hefty -Date: 2012-10-24 23:25:26 -0700 - -Refresh of rs-iomap - ---- - - diff --git a/patches/rs-iomap b/patches/rs-iomap index a24bc0ef..88af3469 100644 --- a/patches/rs-iomap +++ b/patches/rs-iomap @@ -3,7 +3,84 @@ Top: 915e0ee4505ffa5dc789b46752172eee702adae3 Author: Sean Hefty Date: 2012-10-21 14:16:03 -0700 -rsocket: Add direct data placement +rsocket: Add APIs for direct data placement + +We introduce rsocket extensions for supporting direct +data placement (also known as zero copy). Direct data +placement avoids data copies into network buffers when +sending or receiving data. This patch implements zero +copies on the receive side, but adds some basic framework for +supporting it on the sending side. + +Integrating zero copy support into the existing socket APIs +is difficult to achieve when the sockets are set as +nonblocking. Any such implementation is likely to be unusable +in practice. The problem stems from the fact that socket +operations are synchronous in nature. Support for asynchronous +operations is limited to connection establishment. + +Therefore we introduce new calls to handle direct data placement. +The use of the new calls is optional and does not affect the +use of the existing calls. An attempt is made to have the new +routines integrate naturally with the existing APIs. The new +functions are: riomap, riounmap, and riowrite. The basic operation +can be described as follows: + +1. App A calls riomap to register a data buffer with the local + RDMA device. Riomap returns an off_t offset value that + corresponds to the registered data buffer. The app may + select the offset value. +2. Rsockets will transmit an internal message to the remote + peer with information about the registration. This exchange + is hidden from the applications. +3. App A sends a notification message to app B indicating that + the remote iomapped buffer is now available to receive data. +4. App B calls riowrite to transmit data directly into the + riomapped data buffer. +5. App B sends a notification message to app A indicating that + data is available in the mapped buffer. +6. After all transfers are complete, app A calls riounmap to + deregister its data buffer. + +Riomap and riounmap are functionally equivalent to RDMA +memory registration and deregistration routines. They are loosely +based on the mmap and munmap APIs. + +off_t riomap(int socket, void *buf, size_t len, + int prot, int flags, off_t offset) + +Riomap registers an application buffer with the RDMA hardware +associated with an rsocket. The buffer is registered either for +local only access (PROT_NONE) or for remote write access (PROT_WRITE). +When registered for remote access, the buffer is mapped to a given +offset. The offset is either provided by the user, or if the user +selects -1 for the offset, rsockets selects one. The remote peer may +access an iomapped buffer directly by specifying the correct offset. +The mapping is not guaranteed to be available until after the remote +peer receives a data transfer initiated after riomap has completed. + +int riounmap(int socket, void *buf, size_t len) + +Riounmap removes the mapping between a buffer and an rsocket. + +size_t riowrite(int socket, const void *buf, size_t count, + off_t offset, int flags) + +Riowrite allows an application to transfer data over an rsocket +directly into a remotely iomapped buffer. The remote buffer is specified +through an offset parameter, which corresponds to a remote iomapped buffer. +From the sender's perspective, riowrite behaves similar to rwrite. From +a receiver's view, riowrite transfers are silently redirected into a pre- +determined data buffer. Data is received automatically, and the receiver +is not informed of the transfer. However, iowrite data is still considered +part of the data stream, such that iowrite data will be written before a +subsequent transfer is received. A message sent immediately after +initiating an iowrite may be used to notify the receiver of the iowrite. + +It should be noted that the current implementation primarily focused +on being functional for evaluation purposes. Some checks have been +deferred for subsequent patches, and performance is currently limited +by linear lookups. Signed-off-by: Sean Hefty -- 2.46.0