From 228b9061f234cbe1d75c855ead450e17fcc607f8 Mon Sep 17 00:00:00 2001 From: Sean Hefty Date: Wed, 24 Oct 2012 11:02:13 -0700 Subject: [PATCH] refresh --- meta | 7 ++- patches/refresh-temp | 118 ------------------------------------------- patches/rs-iomap | 95 +++++++++++++++++++++++++++++++--- 3 files changed, 92 insertions(+), 128 deletions(-) delete mode 100644 patches/refresh-temp diff --git a/meta b/meta index 5913593b..2b14af23 100644 --- a/meta +++ b/meta @@ -1,9 +1,8 @@ Version: 1 -Previous: 578ed148d2c1ae4bd63865722d0cbb954247c247 -Head: b8def33f803bdfba5a07e11f7989f739a1e829f2 +Previous: 20949f6e32bd90af55dff86a1545f4ec489a648e +Head: d25c8115399392055d5c4fac68cc089a3d0cd330 Applied: - rs-iomap: aa5ec58a7c5a4849b45c9e97f319cd6cfcca3d06 - refresh-temp: b8def33f803bdfba5a07e11f7989f739a1e829f2 + rs-iomap: d25c8115399392055d5c4fac68cc089a3d0cd330 Unapplied: riostream: b833a6940dd0b3a4ce3f0d8fe4ddc6d145cc4c68 resv-rs-len: 7b6ff5c4894f54b221d877adcd709795dffb2fe9 diff --git a/patches/refresh-temp b/patches/refresh-temp deleted file mode 100644 index 75a34106..00000000 --- a/patches/refresh-temp +++ /dev/null @@ -1,118 +0,0 @@ -Bottom: f5779eb35742a3071e97dd98c5f5d30f75687733 -Top: 93fea4a0133ee9be5198c904501a510e1624c0d0 -Author: Sean Hefty -Date: 2012-10-24 11:02:12 -0700 - -Refresh of rs-iomap - ---- - -diff --git a/docs/rsocket b/docs/rsocket -index 5399f6c..1484f65 100644 ---- a/docs/rsocket -+++ b/docs/rsocket -@@ -110,11 +110,11 @@ Bits Message Meaning of - 31:29 Type Bits 28:0 - 000 Data Transfer bytes transfered - 001 reserved --010 reserved -+010 reserved - used internally, available for future use - 011 reserved - 100 Credit Update received credits granted - 101 reserved --110 reserved -+110 Iomap Updated index of updated entry - 111 Control control message type - - Data Transfer -@@ -133,6 +133,12 @@ care not to modify a remote target SGL while it may be in use. This is done - by tracking when a receive buffer referenced by a remote target SGL has been - filled. - -+Iomap Updated -+Used to indicate that a remote iomap entry was updated. The updated entry -+contains the offset value associated with an address, length, and rkey. Once -+an iomap has been updated, the local application can issue directed IO -+transfers against the corresponding remote buffer. -+ - Control Message - DISCONNECT - Indicates that the rsocket connection has been fully disconnected and will no - longer send or receive data. Data received before the disconnect message was -@@ -142,3 +148,44 @@ Control Message - SHUTDOWN - Indicates that the remote rsocket has shutdown the send side of its - connection. The recipient of a shutdown message will no longer accept - incoming data, but may still transfer outbound data. -+ -+ -+Iomapped Buffers -+---------------- -+Rsockets allows for zero-copy transfers using what it refers to as iomapped -+buffers. Iomapping and direct data placement (zero-copy) transfers are done -+using rsocket specific extensions. The general operation is similar to -+that used for normal data transfers described above. -+ -+ host A host B -+ remote iomap -+ target iomap <----------- [ ] -+ [ ] ------ -+ [ ] -- ------ iomapped buffer(s) -+ -- -----> +--+ -+ -- | | -+ -- | | -+ -- | | -+ -- +--+ -+ -- -+ ---> +--+ -+ | | -+ | | -+ +--+ -+ -+The remote iomap contains the address, size, and rkey of the target iomap. As -+the applicaton maps buffers host B to a given rsocket, rsockets will issue an RDMA -+write against one of the entries in the target iomap on host A. The -+updated entry will reference an available iomapped buffer. Immediate data -+included with the RDMA write will indicate to host A that a target iomap -+has been updated. -+ -+When host A wishes to transfer directly into an iomapped buffer, it will check -+its target iomap for an offset corresponding to a remotely mapped buffer. A -+matching iomap entry will contain the address, size, and rkey of the target -+buffer on host B. Host A will then issue an RDMA operation against the -+registered remote data buffer. -+ -+From host A's perspective, the transfer appears as a normal send/write -+operation, with the data stream redirected directly into the receiving -+application's buffer. -diff --git a/man/rsocket.7 b/man/rsocket.7 -index 6a9ea95..a1171fd 100644 ---- a/man/rsocket.7 -+++ b/man/rsocket.7 -@@ -73,7 +73,14 @@ size_t riowrite(int socket, const void *buf, size_t count, off_t offset, int fla - .TP - Riowrite allows an application to transfer data over an rsocket - directly into a remotely iomapped buffer. The remote buffer is specified --through an offset parameter, which corresponds to a remote iomapped buffer. -+through an offset parameter, which corresponds to a remote iomapped buffer. -+From the sender's perspective, riowrite behaves similar to rwrite. From -+a receiver's view, riowrite transfers are silently redirected into a pre- -+determined data buffer. Data is received automatically, and the receiver -+is not informed of the transfer. However, iowrite data is still considered -+part of the data stream, such that iowrite data will be written before a -+subsequent transfer is received. A message sent immediately after initiating -+an iowrite may be used to notify the receiver of the iowrite. - .P - In addition to standard socket options, rsockets supports options - specific to RDMA devices and protocols. These options are accessible -diff --git a/src/rsocket.c b/src/rsocket.c -index 0173555..c534c61 100644 ---- a/src/rsocket.c -+++ b/src/rsocket.c -@@ -89,7 +89,7 @@ static uint32_t polling_time = 10; - enum { - RS_OP_DATA, - RS_OP_RSVD_DATA_MORE, -- RS_OP_WRITE, -+ RS_OP_WRITE, /* opcode is not transmitted over the network */ - RS_OP_RSVD_DRA_MORE, - RS_OP_SGL, - RS_OP_RSVD, diff --git a/patches/rs-iomap b/patches/rs-iomap index 011b336f..c7237b7d 100644 --- a/patches/rs-iomap +++ b/patches/rs-iomap @@ -1,5 +1,5 @@ Bottom: daf53db464152f40dc8d6f2c99844510b03f8567 -Top: f5779eb35742a3071e97dd98c5f5d30f75687733 +Top: 93fea4a0133ee9be5198c904501a510e1624c0d0 Author: Sean Hefty Date: 2012-10-21 14:16:03 -0700 @@ -10,6 +10,82 @@ Signed-off-by: Sean Hefty --- +diff --git a/docs/rsocket b/docs/rsocket +index 5399f6c..1484f65 100644 +--- a/docs/rsocket ++++ b/docs/rsocket +@@ -110,11 +110,11 @@ Bits Message Meaning of + 31:29 Type Bits 28:0 + 000 Data Transfer bytes transfered + 001 reserved +-010 reserved ++010 reserved - used internally, available for future use + 011 reserved + 100 Credit Update received credits granted + 101 reserved +-110 reserved ++110 Iomap Updated index of updated entry + 111 Control control message type + + Data Transfer +@@ -133,6 +133,12 @@ care not to modify a remote target SGL while it may be in use. This is done + by tracking when a receive buffer referenced by a remote target SGL has been + filled. + ++Iomap Updated ++Used to indicate that a remote iomap entry was updated. The updated entry ++contains the offset value associated with an address, length, and rkey. Once ++an iomap has been updated, the local application can issue directed IO ++transfers against the corresponding remote buffer. ++ + Control Message - DISCONNECT + Indicates that the rsocket connection has been fully disconnected and will no + longer send or receive data. Data received before the disconnect message was +@@ -142,3 +148,44 @@ Control Message - SHUTDOWN + Indicates that the remote rsocket has shutdown the send side of its + connection. The recipient of a shutdown message will no longer accept + incoming data, but may still transfer outbound data. ++ ++ ++Iomapped Buffers ++---------------- ++Rsockets allows for zero-copy transfers using what it refers to as iomapped ++buffers. Iomapping and direct data placement (zero-copy) transfers are done ++using rsocket specific extensions. The general operation is similar to ++that used for normal data transfers described above. ++ ++ host A host B ++ remote iomap ++ target iomap <----------- [ ] ++ [ ] ------ ++ [ ] -- ------ iomapped buffer(s) ++ -- -----> +--+ ++ -- | | ++ -- | | ++ -- | | ++ -- +--+ ++ -- ++ ---> +--+ ++ | | ++ | | ++ +--+ ++ ++The remote iomap contains the address, size, and rkey of the target iomap. As ++the applicaton maps buffers host B to a given rsocket, rsockets will issue an RDMA ++write against one of the entries in the target iomap on host A. The ++updated entry will reference an available iomapped buffer. Immediate data ++included with the RDMA write will indicate to host A that a target iomap ++has been updated. ++ ++When host A wishes to transfer directly into an iomapped buffer, it will check ++its target iomap for an offset corresponding to a remotely mapped buffer. A ++matching iomap entry will contain the address, size, and rkey of the target ++buffer on host B. Host A will then issue an RDMA operation against the ++registered remote data buffer. ++ ++From host A's perspective, the transfer appears as a normal send/write ++operation, with the data stream redirected directly into the receiving ++application's buffer. diff --git a/include/rdma/rsocket.h b/include/rdma/rsocket.h index 65feda9..f220c13 100644 --- a/include/rdma/rsocket.h @@ -51,7 +127,7 @@ index 65feda9..f220c13 100644 } #endif diff --git a/man/rsocket.7 b/man/rsocket.7 -index 2ed5ca4..6a9ea95 100644 +index 2ed5ca4..a1171fd 100644 --- a/man/rsocket.7 +++ b/man/rsocket.7 @@ -6,7 +6,7 @@ rsocket \- RDMA socket API @@ -72,7 +148,7 @@ index 2ed5ca4..6a9ea95 100644 follow capabilities and flags are supported at this time: .P PF_INET, PF_INET6, SOCK_STREAM, IPPROTO_TCP, TCP_MAXSEG -@@ -41,6 +41,40 @@ SO_REUSEADDR, TCP_NODELAY, SO_ERROR, SO_SNDBUF, SO_RCVBUF +@@ -41,6 +41,47 @@ SO_REUSEADDR, TCP_NODELAY, SO_ERROR, SO_SNDBUF, SO_RCVBUF .P O_NONBLOCK .P @@ -108,7 +184,14 @@ index 2ed5ca4..6a9ea95 100644 +.TP +Riowrite allows an application to transfer data over an rsocket +directly into a remotely iomapped buffer. The remote buffer is specified -+through an offset parameter, which corresponds to a remote iomapped buffer. ++through an offset parameter, which corresponds to a remote iomapped buffer. ++From the sender's perspective, riowrite behaves similar to rwrite. From ++a receiver's view, riowrite transfers are silently redirected into a pre- ++determined data buffer. Data is received automatically, and the receiver ++is not informed of the transfer. However, iowrite data is still considered ++part of the data stream, such that iowrite data will be written before a ++subsequent transfer is received. A message sent immediately after initiating ++an iowrite may be used to notify the receiver of the iowrite. +.P In addition to standard socket options, rsockets supports options specific to RDMA devices and protocols. These options are accessible @@ -172,7 +255,7 @@ index 26e7f98..0c5f388 100644 + +#endif /* INDEXER_H */ diff --git a/src/rsocket.c b/src/rsocket.c -index cc5effe..0173555 100644 +index cc5effe..c534c61 100644 --- a/src/rsocket.c +++ b/src/rsocket.c @@ -55,6 +55,7 @@ @@ -208,7 +291,7 @@ index cc5effe..0173555 100644 RS_OP_DATA, RS_OP_RSVD_DATA_MORE, - RS_OP_RSVD_DRA, -+ RS_OP_WRITE, ++ RS_OP_WRITE, /* opcode is not transmitted over the network */ RS_OP_RSVD_DRA_MORE, RS_OP_SGL, RS_OP_RSVD, -- 2.41.0