From: Tziporet Koren Date: Thu, 31 May 2007 08:27:46 +0000 (+0300) Subject: New docs for RDS X-Git-Url: https://openfabrics.org/gitweb/?a=commitdiff_plain;h=b78b0785be1236ec9f323c5b0709772d422b5565;p=compat-rdma%2Fdocs.git New docs for RDS Updated HOWTO.build_ofed --- diff --git a/HOWTO.build_ofed b/HOWTO.build_ofed index 2f7496a..b5f33ff 100644 --- a/HOWTO.build_ofed +++ b/HOWTO.build_ofed @@ -18,10 +18,6 @@ The script "build_ofed.sh" is used to build the OFED package based on the OpenFabrics project and InfiniBand git tree. The package is built under the current working directory. -The OFED package includes InfiniBand kernel modules, userspace libraries, -diagnostic tools, performance benchmarks, firmware burning tools, Open MPI and -OSU MPI. - See OFED_release_notes.txt for more details. ============================================================================== @@ -29,88 +25,95 @@ See OFED_release_notes.txt for more details. ============================================================================== The build script for the OFED package can be downloaded from: - https://openib.org/svn/gen2/branches/1.2/ofed/build + git://git.openfabrics.org/~vlad/ofabuild + branch: ofed_1_2 Name: build_ofed.sh - -Usage: build_ofed.sh --ver|-v --git|-g - [--svnrev|-r ] + Usage: build_ofed.sh --ver|-v [--tmpdir ] - [--without-makedist] - [--userspace|-u ] [--ofed-scripts ] [--ofed-docs ] [--mpidir|-m ] - [--extrasdir|-e ] Required: - --ver Determines the name of the OFED version that is built - --git Path to a local GIT tree (directory). The tree must - have previously been created by one of the methods - provided in the "Requirements" section below. + --ver -- Determines the name of the OFED version that is built Optional: - --svnrev The svn revision for extraction of the userspace - component (default: most recent) - - --tmpdir Directory to use as a work area (default: /tmp ) - - --without-makedist Do not execute "make dist" for the userspace - component (default: do "make dist") - - --userspace If you have already checked out the userspace - component, you may use this option to request - that the userspace component be taken from the - given directory. Otherwise, the userspace_URL - (see below) will be used. - - --ofed_scripts If you have already checked out the scripts - component, you may use this option to request that - the scripts component be taken from the given - directory. Otherwise, the ofed_scripts_URL (see - below) will be used. - - --ofed_docs If you have already checked out the docs component, - you may use this option to request that the docs - component be taken from the given directory. - Otherwise, the ofed_docs_URL (see below) will be - used. + --tmpdir -- Directory to use as a work area. (default: /tmp ) + --ofed-scripts -- If you have already checked out the scripts + component, you may use this option to request that + the scripts component be taken from the given + directory. Otherwise, the ofed_scripts_URL (below) + will be used. + --ofed-docs -- If you have already checked out the docs component, + you may use this option to request that the docs + component be taken from the given directory. + Otherwise, the ofed_docs_URL (below) will be used. + --mpidir -- If you have already checked out the mpi component, + you may use this option to request that the mpi + component be taken from the given directory. + Otherwise, the mpi_URL (below) will be used. - --mpidir If you have already checked out the mpi component, - you may use this option to request that the mpi - component be taken from the given directory. - Otherwise, the mpi_URL (see below) will be used. - - --extrasdir If you have already checked out the extras - component, you may use this option to request that - the extras component be taken from the given - directory. Otherwise, the extras_URL (see below) - will be used. - - -Sources are extracted by default from the following locations: - userspace_URL: - https://openib.org/svn/gen2/branches/1.2/src/userspace - openib_scripts_URL: - https://openib.org/svn/gen2/branches/1.2/ofed/openib/scripts - ofed_scripts_URL: - https://openib.org/svn/gen2/branches/1.2/ofed/scripts - ofed_docs_URL: - https://openib.org/svn/gen2/branches/1.2/ofed/docs - mpi_URL: - https://openib.org/svn/gen2/branches/1.2/ofed/mpi - extras_URL: - https://openib.org/svn/gen2/branches/1.2/ofed/extras Example: - ./build_ofed.sh --ver 1.2-rc6 --git /local/git/ofed_1_1/ + ./build_ofed.sh --ver 1.2-rc6 This command will create a package (i.e., subtree) called OFED-1.2-rc6 - in the current working direcory. The git tree "/local/git/ofed_1_1/" - in this example is a local InfiniBand git tree which was created using - one of the methods in the "Requirements" section below. + in the current working directory. + +Sources are extracted by default from the following locations: + libibverbs: + git://git.openfabrics.org/ofed_1_2/libibverbs.git ofed_1_2 + libmthca: + git://git.openfabrics.org/ofed_1_2/libmthca.git ofed_1_2 + libehca: + git://git.openfabrics.org/ofed_1_2/libehca.git ofed_1_2 + libipathverbs: + git://git.openfabrics.org/ofed_1_2/libipathverbs.git ofed_1_2 + tvflash: + git://git.openfabrics.org/ofed_1_2/tvflash.git ofed_1_2 + libibcm: + git://git.openfabrics.org/ofed_1_2/libibcm.git ofed_1_2 + libsdp: + git://git.openfabrics.org/ofed_1_2/libsdp.git ofed_1_2 + mstflint: + git://git.openfabrics.org/ofed_1_2/mstflint.git ofed_1_2 + perftest: + git://git.openfabrics.org/ofed_1_2/perftest.git ofed_1_2 + srptools: + git://git.openfabrics.org/ofed_1_2/srptools.git ofed_1_2 + ipoibtools: + git://git.openfabrics.org/ofed_1_2/ipoibtools.git ofed_1_2 + librdmacm: + git://git.openfabrics.org/ofed_1_2/librdmacm.git ofed_1_2 + dapl: + git://git.openfabrics.org/ofed_1_2/dapl.git ofed_1_2 + imgen: + git://git.openfabrics.org/ofed_1_2/imgen.git ofed_1_2 + management: + git://git.openfabrics.org/ofed_1_2/management.git ofed_1_2 + libcxgb3: + git://git.openfabrics.org/ofed_1_2/libcxgb3.git ofed_1_2 + qlvnictools: + git://git.openfabrics.org/ofed_1_2/qlvnictools.git ofed_1_2 + sdpnetstat: + git://git.openfabrics.org/ofed_1_2/sdpnetstat.git ofed_1_2 + rds-tools: + http://www.openfabrics.org/~vlad/ofed_1_2/rds-tools + + mvapich: + http://www.openfabrics.org/~pasha/ofed_1_2/mvapich + openmpi: + http://www.openfabrics.org/~jsquyres/ofed_1_2 + mvapich2: + http://www.openfabrics.org/~rowland/ofed_1_2 + mpitests: + http://www.openfabrics.org/~pasha/ofed_1_2/mpitests + mpi-selector: + git://git.openfabrics.org/~jsquyres/mpi-selector.git ofed_1_2 + ============================================================================== 3. Requirements @@ -118,33 +121,9 @@ Example: 1. Git: Can be downloaded from: - http://www.kernel.org/pub/software/scm/git/git-1.4.2.tar.gz + http://www.kernel.org/pub/software/scm/git/git-1.5.2.tar.gz -2. Subversion: - Can be downloaded from: - http://subversion.tigris.org - -3. InfiniBand Git tree: - There are two ways to get the infiniband git tree: - - The faster way: - mkdir gitdir - cd gitdir - git clone --bare \ - git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git \ - .git - git fetch git://www.mellanox.co.il/~git/infiniband ofed_1_1 \ - ofed_addons cma_branch ehca_branch mst_sdp - - - The slower way: - mkdir gitdir - cd gitdir - git clone -s --bare git://www.mellanox.co.il/~git/infiniband .git - git checkout ofed_1_1 `git-ls-tree -r --name-only ofed_1_1 \ - include/rdma include/scsi/srp.h drivers/infiniband \ - Documentation/infiniband ofed_scripts kernel_patches` - echo 'ref: refs/heads/ofed_1_1' > .git/HEAD - -4. Autotools: +2. Autotools: libtool-1.5.20 or higher autoconf-2.59 or higher diff --git a/RDS_README.txt b/RDS_README.txt new file mode 100755 index 0000000..41d5c11 --- /dev/null +++ b/RDS_README.txt @@ -0,0 +1,218 @@ +RDS(7) Linux Programmer’s Manual RDS(7) + + + +NAME + rds - RDS socket API + +SYNOPSIS + #include + #include + #define RDS_CANCEL_SENT_TO 1 + #define RDS_SNDBUF 2 + + rds_socket = socket(pf_rds, SOCK_SEQPACKET, 0); + +DESCRIPTION + This is an implementation of the RDS socket API. It provides reliable, + in-order datagram delivery between sockets over a variety of trans- + ports. + + +SOCKET CREATION + RDS is still in development and as such does not have a reserved proto- + col family constant. Applications must read the string representation + of the protocol family value from the pf_rds sysctl parameter file + described below. + + +BINDING + A new RDS socket has no local address when it is first returned from + socket(2). It must be bound to a local address by calling bind(2) + before any messages can be sent or received. RDS sockets do not sup- + port connecting to remote endpoints with connect(2). An RDS socket can + only be bound to one address and only one socket can be bound to a + given address. If no port is specified in the binding address then an + unbound port is selected at random. + + RDS has the notion of associating a socket to an underlying transport. + The transport for a socket is decided based on the local address that + is bound. From that point on the socket can only reach destinations + which are available through the this transport. + + +MESSAGE TRANSMISSION + Messages may be sent using sendmsg(2) once the RDS socket is bound. + Message length cannot exceed 4 gigabytes as the wire protocol uses an + unsigned 32 bit integer to express the message length. + + RDS does not support out of band data. + + A successful sendmsg(2) call puts the message in the socket’s transmit + queue where it will remain until either the destination acknowledges + that the message is no longer in the network or the application removes + the message from the send queue. Messages are removed from the send + queue with the RDS_CANCEL_SENT_TO socket option described below. + + A given RDS socket has limited transmit buffer space for each destina- + tion address. While a message is in the transmit queue its payload + bytes are accounted for. If an attempt is made to send a message to a + destination whose buffer does not have room for the new message then + the sender will block or EAGAIN will be returned depending on MSG_DONT- + WAIT message flag. The SO_SNDTIMEO socket option dictates how long the + send will wait for buffer. + + The size of the send buffer for a given destination is governed by the + RDS_SNDBUF socket option and sysctl parameters described below. The + SO_SNDBUF socket option is ignored. + + A message sent with no payload bytes will not consume any space in the + destination’s send buffer but will result in a message receipt on the + destination. The receiver will not get any payload data but will be + able to see the sender’s address. + + +MESSAGE RECEIPT + Messages may be received with recvmsg(2) on an RDS socket once it is + bound to a source address. The MSG_DONTWAIT message flag determines if + the receive will block waiting for message arrival and the SO_RCVTIMEO + socket option dictates how long the receive will wait. The MSG_PEEK + flag stops the message from being removed from the receive queue. + + The memory consumed by messages waiting for delivery does not limit the + number of messages that can be queued for receive. Senders must be + careful not to overwhelm the receiver by sizing their send buffers + appropriately. The SO_RCVBUF socket option is ignored. + + If the length of the message exceeds the size of the buffer provided to + recvmsg(2) then the remainder of the bytes in the message are discarded + and the MSG_TRUNC flag is set in the msg_flags field. In this truncat- + ing case recvmsg(2) will still return the number of bytes copied, not + the length of entire messge. If MSG_TRUNC is set in the flags argument + to recvmsg(2) then it will return the number of bytes in the entire + message. Thus one can examine the size of the next message in the + receive queue without incuring a copying overhead by providing a zero + length buffer and setting MSG_PEEK and MSG_TRUNC in the flags argument. + + The sending address of a zero-length message will still be provided in + the msg_name field. + + +POLL + RDS supports a limited poll(2) API. POLLIN is returned when there is a + message waiting in the socket’s receive queue. POLLOUT is always + returned, it is up to the application to back off if poll is used to + trigger sends. + + +RELIABILITY + If sendmsg(2) succeeds then RDS guarantees that the message will be + visible to recvmsg(2) on a socket bound to the destination address as + long as that destination socket remains open. + + If there is no socket bound on the destination than the message is + silently dropped. If the sending RDS can’t be sure that there is no + socket bound then it will try to send the message indefinitely until it + can be sure or the sent message is canceled. + + If a socket is closed then all pending sent messages on the socket are + canceled and may or may not be seen by the receiver. + + The RDS_CANCEL_SENT_TO socket option can be used to cancel all pending + messages to a given destination. + + If a receiving socket is closed with pending messages then the sender + considers those messages as having left the network and will not + retransmit them. + + A message will only be seen by recvmsg(2) without MSG_PEEK once. Once + the message has been delivered it is removed from the sending socket’s + transmit queue. + + All messages sent from the same socket to the same destination will be + delivered in the order they’re sent. Messages sent from different + sockets, or to different destinations, may be delivered in any order. + + +ADDRESS FORMATS + RDS uses sockaddr_in as described in ip(7) to describe addresses, + including setting sin_family to AF_INET . RDS only supports unicast + communication -- broadcast and multicast addresses are not supported. + + +SOCKET OPTIONS + The following RDS specific socket options are available when the + sol_rds sysctl parameter is read and used as the level with getsock- + opt(2) or setsockopt(2) + + + RDS_SNDBUF + This determines the total number of bytes that may be queued in + the transmit queue for a given destination. Changing this does + not have an immediate effect on pending transmission, it is + intended to be set early and infrequently. The default, mini- + mum, and maximum values of this option are governed by the snd- + buf_* sysctl parameters described below. + + + RDS_CANCEL_SENT_TO + Setting this option is used to cancel messages sent to a spe- + cific destination. The destination address is specified by + passing a sockaddr pointer and length as the optval and optlen + arguments to setsockopt(2) . Errors are only returned if the + socket is not yet bound or if sockaddr is malformed. No error + is returned if there are no messages queued for the given desti- + nation. getsockopt(2) is not supported on this option and will + return ENOPROTOOPT . + + +SYSCTL + These parameteres may only be accessed through their files in + /proc/sys/net/rds/ . Access through sysctl(2) is not supported. + + + pf_rds This file contains the string representation of the protocol + family constant passed to socket(2) to create a new RDS socket. + + + sol_rds + This file contains the string representation of the socket level + parameter that is passed to getsockopt(2) and setsockopt(2) to + manipulate RDS socket options. + + + sndbuf_default_bytes + This parameter determines the initial value of RDS_SNDBUF on a + newly created socket. New values written to this file must not + be less than sndbuf_min_bytes and not greater than snd- + buf_max_bytes + + sndbuf_max_bytes + This parameter determines the maximum value of the snd- + buf_default_bytes and sndbuf_min_bytes parameters. It can not + be greater than the number of bytes represented in an unsigned + 32bit integer (4 gigabytes). + + sndbuf_min_bytes + This parameter determines the minimum value of the snd- + buf_default_bytes and sndbuf_max_bytes parameters. It can not + be less than 0. + + reconnect_delay_min_ms + This parameter determines the minimum amount of time that will + pass before attempting to reconnect to a peer after a failed + connect attempt. + + reconnect_delay_max_ms + This parameter determines the maximum amount of time that will + seperate reconnect attempts. The reconnect delay approaches + this by exponentially increasing the minimum delay. + + +SEE ALSO + socket(2), bind(2), sendmsg(2), recvmsg(2), getsockopt(2). setsock- + opt(2). + + + +Linux Man Page RDS(7) diff --git a/rds_release_notes.txt b/rds_release_notes.txt new file mode 100755 index 0000000..3659868 --- /dev/null +++ b/rds_release_notes.txt @@ -0,0 +1,36 @@ + Open Fabrics Enterprise Distribution (OFED) + RDS in OFED 1.2 Release Notes + + May 2007 + + +=============================================================================== +Table of Contents +=============================================================================== +1. Overview +2. Supported Platforms +3. Known Issues + +=============================================================================== +1. Overview +=============================================================================== +RDS socket API. It provides reliable, in-order datagram delivery between +sockets over a variety of transports. +For details see RDS.README.txt. +Also available at: http://oss.oracle.com/projects/rds/dist/documentation/rds.7.txt + +=============================================================================== +2. supported platforms +=============================================================================== + + RHEL4.0 Update 3,4,5 + RHEL5.0 + SLES 10 + +=============================================================================== +3. Known Issues +=============================================================================== +1. RDS HA fails + + +