From: Tziporet Koren Date: Sun, 3 Jun 2007 14:56:33 +0000 (+0300) Subject: Update from Ishai X-Git-Url: https://openfabrics.org/gitweb/?a=commitdiff_plain;h=0b412d86c1c7be4a9dde5704517710f23a9a0e05;p=~ardavis%2Fofed_docs%2F.git Update from Ishai --- diff --git a/srp_release_notes.txt b/srp_release_notes.txt index d83d731..67d9923 100644 --- a/srp_release_notes.txt +++ b/srp_release_notes.txt @@ -2,7 +2,7 @@ Open Fabrics Enterprise Distribution (OFED) SRP in OFED 1.2 Release Notes - May 2007 + June 2007 ============================================================================== @@ -32,6 +32,19 @@ for transferring commands and data between a SCSI initiator port and a SCSI target port using RDMA communication service. +============================================================================== +1. Changes and Bug Fixes +============================================================================== +* Support HA in Red Hat +* Add a default rules file (/etc/srp_daemon.conf) and it is possible to supply + attributes of the connection in the rules file (max_cmd_per_lun, max_sect). +* Handle a rare race condition. +* Decrease log size in ./var/log/messages +* correct line in error messages in rule files +* fix a potential memory corruption +* support CM dgid redirection +* Fix some memory leaks found + ============================================================================== 2. Software Dependencies ============================================================================== @@ -52,7 +65,7 @@ The SRP Initiator supports: (www.t10.org/ftp/t10/drafts/spc3/spc3r21b.pdf) - Basic SCSI Block Commands -2 (SBC-2) (www.t10.org/ftp/t10/drafts/sbc2/sbc2r16.pdf) -- Basic functionalities, task management and limited error handling +- Basic functionality, task management and limited error handling ============================================================================== 4. Loading SRP Initiator @@ -82,7 +95,7 @@ automatically. - To establish a connection with an SRP Target and create SRP (SCSI) device(s) for that target under /dev, use the following command: - echo id_ext=[GUID value],ioc_guid=[GUID value],dgid=[port GID value],\ + echo -n id_ext=[GUID value],ioc_guid=[GUID value],dgid=[port GID value],\ pkey=ffff,service_id=[service[0] value] > \ /sys/class/infiniband_srp/srp-mthca[hca number]-[port number]/add_target @@ -100,10 +113,10 @@ automatically. - To list the new SCSI devices that have been added by the echo command, you may use either of the following two methods: - a. Execute "fdisk -l". This commands lists all devices; the new devices are + a. Execute "fdisk -l". This command lists all devices; the new devices are included in this listing. - b. Execute "dmesg" or look at /var/log/messages to find messages with the names - of the new devices. + b. Execute "dmesg" or look at /var/log/messages to find messages with the + names of the new devices. ============================================================================== @@ -171,7 +184,7 @@ ibsrpdm usage b. To establish a connection with an SRP Target (Section 5) using the output from the "libsrpdm -c" example above, execute the following command: - echo id_ext=200400A0B81146A1,ioc_guid=0002c90200402bd4, + echo -n id_ext=200400A0B81146A1,ioc_guid=0002c90200402bd4, dgid=fe800000000000000002c90200402bd5,pkey=ffff,service_id=200400a0b81146a1 > /sys/class/infiniband_srp/srp-mthca0-1/add_target @@ -189,12 +202,16 @@ In addition to the ibsrpdm functionality described above, srp_daemon can also: - Discover reachable SRP targets given an infiniband HCA name and port, rather than just by /dev/umad where is a digit - Enable High Availability operation (together with Device-Mapper Multipath) +- Have a configuration file that can decide to which targets to connect. a. srp_daemon commands equivalent to ibsrpdm: "srp_daemon -a -o" is equivalent to "ibsrpdm" "srp_daemon -c -a -o" is equivalent to "ibsrpdm -c" +Note: These srp_daemon commands can behave differently than the equivalent + ibsrpdm command when /etc/srp_daemon.conf is not empty. + b. srp_daemon extensions to ibsrpdm - To discover SRP Targets reachable from HCA device , @@ -207,7 +224,18 @@ b. srp_daemon extensions to ibsrpdm - Executing srp_daemon without -a option will display only the reachable Targets to which the initiator is not connected (via the port upon which - srp_daemon was activated). + srp_daemon was activated). When executing with the -e option it is better + to omit -a. + + - It is recommended to use the -n option. This option adds the initiator_ext + to the connecting string. (See Section 8 for more details). + + - One can set a configuration file for srp_daemon. The default + configuration file is /etc/srp_daemon.conf but it is possible to supply + another configuration file using the -f option. The configuration file + configure to which targets srp_daemon is allowed to connect. The + configuration file can be used also to set values for additional + parameters (e.g., max_cmd_per_lun, max_sect). - Continuous background (daemon) operation, providing automatic ongoing detection and connection capability -- see the next section. @@ -221,7 +249,11 @@ b. srp_daemon extensions to ibsrpdm - To connect to all the existing Targets in the fabric, execute srp_daemon -e -o. This utility will scan the fabric once, connect to - every Target it detects, and then exit. + every Target it detects, and then exit. + + - Note that srp_daemon will follow the configuration it finds in + /etc/srp_daemon.conf. It can ignore a target if this target is disallowed + by the configuration file. - To connect to all the existing Targets in the fabric and to connect to new targets that will join the fabric, execute srp_daemon -e. This utility @@ -278,10 +310,6 @@ values according to this convention. For example: 9. High Availability (HA) ============================================================================== - Note: This is a Beta release of the High Availability feature for the - SCSI RDMA Protocol (SRP) Initiator. - It is intended for development use, not as a complete product. - High Availability Overview -------------------------- @@ -291,8 +319,7 @@ SRP daemon. Each initiator is connected to the same target from several ports/HCAs. The DM multipath is responsible for joining together different paths to the same target and for fail-over between paths when one of them goes offline. -Rules were added to udev that will execute multipath on newly joined SCSI -devices. +Multipath will be execute on newly joined SCSI devices. Each initiator should execute several instances of the SRP daemon, one for each port. At startup, each SRP daemon detects the SRP targets in the fabric and @@ -311,30 +338,32 @@ the scsi_host is removed, multipath switches to another path to this target When the failed path recovers, it will be detected by the SRP daemon. The SRP daemon will then request ib_srp to connect to this target. Once the connection -is up, there will be a new scsi_host for this target. The udev rule will then -execute multipath on the devices of this host, and we will return to the -original state (before the path failed). +is up, there will be a new scsi_host for this target. Multipath will be +executed on the devices of this host, and we will return to the original state +(before the path failed). High Availability Prerequisites ------------------------------- Installation for RHEL4: (Execute once) - - Verify that the standard device-mapper-multipath rpm is installed. If not, install it from the RHEL distribution. + - Verify that the standard device-mapper-multipath rpm is installed. If not, + install it from the RHEL distribution. Installation for SLES10: (Execute once) - - Verify that multipath is installed. If not, it is possible to download it - from http://christophe.varoqui.free.fr/multipath-tools/multipath-tools-0.4.7.tar.bz2 - and then compile and install it. + - Verify that multipath is installed. If not, install it from the + installation. It is also possible to download it from + http://christophe.varoqui.free.fr/multipath-tools/multipath-tools-0.4.7.tar.bz2 + and then compile and install it. - - Update udev: (Execute once - for manual activation of High Availability only) + - Update udev: (Execute once - for manual activation of High Availability only) - - Add a file to /etc/udev/rules.d/ (you can call it 91-srp.rules) - This file should have one line: - ACTION=="add", KERNEL=="sd*[!0-9]", RUN+="/sbin/multipath %M:%m" + - Add a file to /etc/udev/rules.d/ (you can call it 91-srp.rules) + This file should have one line: + ACTION=="add", KERNEL=="sd*[!0-9]", RUN+="/sbin/multipath %M:%m" - Note that when SRPHA_ENABLE is set to "yes" (see below in Automatic activation - of High Availability subsection), this file is created upon each boot of - the driver and deleted when the driver is unloaded. + Note that when SRPHA_ENABLE is set to "yes" (see below in Automatic + activation of High Availability subsection), this file is created upon each + boot of the driver and deleted when the driver is unloaded. Manual Activation of High Availability @@ -347,7 +376,7 @@ Initialization: (Execute after each boot of the driver) as described above 4) Execute for each port and each HCA: srp_daemon -c -e -R 300 -i -p - (You can use another value for -R. See in Known Issues section the + (You can use another value for -R. See in Known Issues section the workaround for the rare race condition.) This step can be performed by executing srp_daemon.sh, which sends @@ -356,7 +385,8 @@ Initialization: (Execute after each boot of the driver) Now it is possible to access the SRP LUNs on /dev/mapper/. NOTE: It is possible that regular (not SRP) LUNs may also be present; - the SRP LUNs may be identified by their name. + the SRP LUNs may be identified by their name. you may want to + configure the /etc/multipath.conf file to change multipath behavior. Automatic Activation of High Availability @@ -419,19 +449,15 @@ should make sure this will not occur. One solution may be to stop "haldaemon" 11. Known Issues ============================================================================== -- SRP is not supported on a 32-bit operating system running on a 64-bit - platform. - - The SCSI device is sent offline when a link goes down for several seconds, when the subnet manager goes down for a long time, or when a disk is removed from a target during run-time. - There is a very rare race condition which can cause the SRP daemon to miss a - target that joins the fabric. The race can occur if a target that left the - fabric rejoins it after the ib_srp module has decided to remove this target, - but before the scsi_host has been removed. As a result, when the SRP daemon - checks if this target is already connected, it will receive a positive - response and will therefore not reconnect to this target. + target that joins the fabric. The race can occur if a target join and leaves + the fabric several times in a short time. (This can happen if the cable is + not connected well). In such a case the SM may ignore this quick change of + state and may not send an InformInfo to the srp_daemon. Workaround: Execute the srp_daemon command with the -R option. This option causes the SRP daemon to perform a full rescan of the fabric every @@ -445,10 +471,6 @@ should make sure this will not occur. One solution may be to stop "haldaemon" all LUNs on all connections may take some time (several minutes). However, it is possible to start working while this process is in progress. -- If SRP High Availability is enabled, disconnections while OFED is booting, or - simultaneous disconnections and connections during normal operation, may lead - to what seems as a deadlock between multipath instances. - - Stopping the driver while SRP High Availability is enabled kills all multipath processes. Consider appropriate actions in case multipath is used for other purposes. @@ -468,11 +490,16 @@ should make sure this will not occur. One solution may be to stop "haldaemon" on each path. See Section 8, "Multiple Connections from Initiator IB Port to the Target" for details. +- The srp_daemon tool reads by default the configuration file + /etc/srp_daemon.conf. In case this configuration file disallows to connect + to a certain target, srp_daemon will ignore this target. If you find out + that srp_daemon ignores a target, please check /etc/srp_daemon.conf file. + ============================================================================== 12. Vendor Specific Notes ============================================================================== -Hosts connected to Silverstorm SRP Targets must perform one of the following +Hosts connected to Qlogic SRP Targets must perform one of the following steps after upgrading to OFED 1.2 to continue accessing their storage successfully: @@ -498,8 +525,8 @@ successfully: /sys/class/inifiniband_srp/srp-mthca0-1/add_target -2. Change the SRP map on the Silverstorm SRP Target to set the expected +2. Change the SRP map on the Qlogic SRP Target to set the expected initiator extension to 0. For details on how to change the SRP map on a - Silverstorm SRP Target, please refer to product documentation. + Qlogic SRP Target, please refer to product documentation.