From: Vladimir Sokolovsky Date: Mon, 6 Aug 2012 14:26:13 +0000 (+0300) Subject: OFED_tips.txt: Fixed examples to match OFED-3.5 X-Git-Url: https://openfabrics.org/gitweb/?a=commitdiff_plain;h=97e3c99f62259041d96526e1e3594dbaa94cf260;p=compat-rdma%2Fdocs.git OFED_tips.txt: Fixed examples to match OFED-3.5 Signed-off-by: Vladimir Sokolovsky --- diff --git a/OFED_tips.txt b/OFED_tips.txt index 9e74123..78fe0c0 100644 --- a/OFED_tips.txt +++ b/OFED_tips.txt @@ -1,7 +1,7 @@ Open Fabrics Enterprise Distribution (OFED) - Tips for Working with OFED 1.5.x + Tips for Working with OFED 3.5 - December 2011 + August 2012 =============================================================================== Table of Contents @@ -9,7 +9,7 @@ Table of Contents 1. OFED Utilities 2. Debug HOWTOs 3. Pinning (Locking) User Memory Pages -4. External Module Compilation Over OFED-1.5.x +4. External Module Compilation Over OFED-3.5 5. Adding/Deleting a patch to OFED package 6. Adding vendor specific actions to the installation of OFED 7. How to compile OFED sources manually @@ -60,25 +60,25 @@ b. ibstat Examples: ibstat -l # list all IB devices - ibstat mthca0 2 # stat port 2 of mthca0 + ibstat mlx4_0 2 # stat port 2 of mlx4_0 c. Using sysfs file system The driver supports the sysfs file system under: /sys/class/infiniband Examples: - > ls /sys/class/infiniband/mthca0/ + > ls /sys/class/infiniband/mlx4_0/ board_id device fw_ver hca_type hw_rev node_desc node_guid node_type ports sys_image_guid - > cat /sys/class/infiniband/mthca0/board_id - MT_0200000001 + > cat /sys/class/infiniband/mlx4_0/board_id + MT_1090110019 - > ls /sys/class/infiniband/mthca0/ports/1/ + > ls /sys/class/infiniband/mlx4_0/ports/1/ cap_mask counters gids lid lid_mask_count phys_state pkeys rate sm_lid sm_sl state - > cat /sys/class/infiniband/mthca0/ports/1/state + > cat /sys/class/infiniband/mlx4_0/ports/1/state 4: ACTIVE 1.2 Performance Tests @@ -190,100 +190,89 @@ or not). This file is a part of the ofed-docs RPM and installed under The same information can be obtained by executing the 'ofed_info' command. For example: -> ofed_info -OFED-1.5.1: +# ofed_info -s +OFED-3.5-rc1: -compat-dapl: -http://www.openfabrics.org/downloads/dapl/compat-dapl-1.2.16.tar.gz +# ofed_info +OFED-3.5-rc1: + +ofed-scripts: +git://git.openfabrics.org/~vlad/ofed_scripts.git compat-rdma +ar.gz + +compat-rdma: +git://git.openfabrics.org/compat-rdma/compat-rdma.git master +commit 2036874cf4c072c04b96b69e5b965a788fc77a6d dapl: -http://www.openfabrics.org/downloads/dapl/dapl-2.0.27.tar.gz +http://www.openfabrics.org/downloads/dapl/dapl-2.0.36.tar.gz ib-bonding: -http://www.openfabrics.org/~monis/ofed_1_5/ib-bonding-0.9.0-42.src.rpm +http://www.openfabrics.org/downloads/ib-bonding/ib-bonding-0.9.0-43.src.rpm + +ibacm: +http://www.openfabrics.org/downloads/rdmacm/ofed/ibacm-1.0.7.tar.gz ibsim: http://www.openfabrics.org/downloads/ibsim/ibsim-0.5-0.1.g327c3d8.tar.gz ibutils: -http://www.openfabrics.org/downloads/ibutils/ibutils-1.5.4-0.1.g0464fe6.tar.gz +http://www.openfabrics.org/downloads/ibutils/ibutils-1.5.7-0.1.g05a9d1a.tar.gz infiniband-diags: -http://www.openfabrics.org/downloads/management/infiniband-diags-1.5.5.tar.gz +http://www.openfabrics.org/downloads/management/infiniband-diags-1.6.1.tar.gz + +infinipath-psm: +http://www.openfabrics.org/downloads/infinipath-psm/infinipath-psm-2.9-926.1005_open.tar.gz libcxgb3: -http://www.openfabrics.org/downloads/cxgb3/libcxgb3-1.2.5.tar.gz +http://www.openfabrics.org/downloads/cxgb3/libcxgb3-1.3.1.tar.gz + +libcxgb4: +http://www.openfabrics.org/downloads/cxgb4/libcxgb4-1.2.0.tar.gz libehca: -http://www.openfabrics.org/downloads/libehca/libehca-1.2.1-0.1.g0a82a52.tar.gz +http://www.openfabrics.org/downloads/libehca/libehca-1.2.2-0.1.g69e1a88.tar.gz libibcm: http://www.openfabrics.org/downloads/rdmacm/libibcm-1.0.5.tar.gz libibmad: -http://www.openfabrics.org/downloads/management/libibmad-1.3.4.tar.gz +http://www.openfabrics.org/downloads/management/libibmad-1.3.9.tar.gz libibumad: -http://www.openfabrics.org/downloads/management/libibumad-1.3.4.tar.gz +http://www.openfabrics.org/downloads/management/libibumad-1.3.8.tar.gz libibverbs: -http://www.openfabrics.org/downloads/libibverbs/libibverbs-1.1.3-0.6.g932f1a2.tar.gz +http://www.openfabrics.org/downloads/verbs/libibverbs-1.1.6.tar.gz libipathverbs: http://www.openfabrics.org/downloads/libipathverbs/libipathverbs-1.2.tar.gz libmlx4: -http://www.openfabrics.org/downloads/libmlx4/libmlx4-1.0-0.7.g2432360.tar.gz +http://www.openfabrics.org/downloads/mlx4/libmlx4-1.0.4.tar.gz libmthca: http://www.openfabrics.org/downloads/libmthca/libmthca-1.0.5-0.1.gbe5eef3.tar.gz libnes: -http://www.openfabrics.org/downloads/nes/libnes-1.0.1.tar.gz +http://www.openfabrics.org/downloads/nes/libnes-1.1.3.tar.gz librdmacm: -http://www.openfabrics.org/downloads/rdmacm/librdmacm-1.0.11.tar.gz - -libsdp: -http://www.openfabrics.org/downloads/libsdp/libsdp-1.1.100-0.1.g920ea31.tar.gz - -mpi-selector: -http://www.openfabrics.org/downloads/mpi-selector/mpi-selector-1.0.3-1.src.rpm - -mpitests: -http://www.openfabrics.org/~pasha/ofed_1_5/mpitests/mpitests-3.2-916.src.rpm +http://www.openfabrics.org/downloads/rdmacm/librdmacm-1.0.16.tar.gz mstflint: -http://www.openfabrics.org/downloads/mstflint/mstflint-1.4-0.2.g0fccc0f.tar.gz - -mvapich: -http://www.openfabrics.org/~pasha/ofed_1_5_1/mvapich/mvapich-1.2.0-3635.src.rpm - -mvapich2: -http://www.openfabrics.org/~perkinjo/ofed_1_5/mvapich2-1.4.1-1.src.rpm - -ofa_kernel: -git://git.openfabrics.org/ofed_1_5/linux-2.6.git ofed_kernel_1_5 -commit 17badd753b40fb1046dc2e5474739357a921fb86 +http://www.openfabrics.org/downloads/mstflint/mstflint-1.4-1.18.g1adcfbf.tar.gz ofed-docs: -git://git.openfabrics.org/~tziporet/docs.git ofed_1_5 -commit 137b089f71570fed2d5b9080410fa0b8f35d7fd5 - -open-iscsi-generic1: -http://www.openfabrics.org/downloads/iscsi/open-iscsi-generic-2.0-754.1.src.rpm - -open-iscsi-generic2: -http://www.openfabrics.org/downloads/iscsi/open-iscsi-generic-2.0-869.2.src.rpm - -openmpi: -http://www.openfabrics.org/~jsquyres/ofed_1_5/openmpi-1.4.1-2ofed.src.rpm +git://git.openfabrics.org/compat-rdma/docs.git ofed_3_2 +commit 6dff336cfb197351c9fa0b50c0715b59a4d822cf opensm: -http://www.openfabrics.org/downloads/management/opensm-3.3.5.tar.gz +http://www.openfabrics.org/downloads/management/opensm-3.3.15.tar.gz perftest: -http://www.openfabrics.org/downloads/perftest/perftest-1.2.3-0.10.g90b10d8.tar.gz +http://www.openfabrics.org/downloads/perftest/perftest-1.4.0-0.66.g9074687.tar.gz qlvnictools: http://www.openfabrics.org/downloads/qlvnictools/qlvnictools-0.0.1-0.1.ge27eef7.tar.gz @@ -292,23 +281,11 @@ qperf: http://www.openfabrics.org/downloads/qperf/qperf-0.4.6-0.1.gb81434e.tar.gz rds-tools: -http://www.openfabrics.org/~vlad/ofed_1_5/rds-tools/rds-tools-1.5-1.src.rpm - -rnfs-utils: -http://www.openfabrics.org/~swise/ofed_1_5/rnfs-utils/rnfs-utils-1.1.5-10.OFED.src.rpm - -sdpnetstat: -http://www.openfabrics.org/downloads/sdpnetstat/sdpnetstat-1.60-0.2.g8844f04.tar.gz +http://www.openfabrics.org/downloads/rds-tools/rds-tools-2.0.4.tar.gz srptools: http://www.openfabrics.org/downloads/srptools/srptools-0.0.4-0.1.gce1f64c.tar.gz -tgt-generic: -http://www.openfabrics.org/downloads/iscsi/tgt-generic-0.1-20080828.src.rpm - -ofed-scripts: -git://git.openfabrics.org/~vlad/ofed_scripts.git ofed_1_5 -commit f3359b78fbe6cd9e3f78d2c39498546270b809ad 2.2 Installed OFED Components ------------------------------- @@ -317,12 +294,11 @@ on the machine. For example: - > /etc/infiniband/info +# /etc/infiniband/info prefix=/usr -Kernel=2.6.18-164.el5 - -Configure options: --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-mthca-mod --with-mlx4-mod --with-mlx4_en-mod --with-ipoib-mod --with-sdp-mod --with-rds-mod +Kernel=2.6.32-220.el6.x86_64 +Configure options: --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-mthca-mod --with-mlx4-mod --with-mlx4_en-mod --with-cxgb3-mod --with-cxgb4-mod --with-nes-mod --with-qib-mod --with-ipoib-mod --with-srp-mod 2.3 Building/Installing InfiniBand (IB) Modules With Debug Information ---------------------------------------------------------------------- @@ -488,25 +464,22 @@ meaning without building the RPMs and without using the install.pl script. 7.1 Compiling the kernel modules -------------------------------- -1. tar xzf OFED-1.5.1.tgz -2. rpm -ihv OFED-1.5.1/SRPMS/ofa_kernel-1.5.1-ofed1.5.1.src.rpm +1. tar xzf OFED-3.5.tgz +2. rpm -ihv OFED-3.5/SRPMS/compat-rdma-3.5-OFED.3.5.src.rpm 3. cd /usr/src/redhat/SOURCES -4. tar xzvf ofa_kernel-1.5.1.tgz -5. cd ofa_kernel-1.5.1 +4. tar xzvf compat-rdma-3.5.tgz +5. cd compat-rdma-3.5 6. configure: run ./configure --help for a list of options. basic invocation is: - ./configure --with-core-mod --with-ipoib-mod --with-mthca-mod --with-mlx4_core-mod --with-mlx4_inf-mod + ./configure --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-mlx4-mod --with-mlx4_en-mod --with-cxgb3-mod --with-cxgb4-mod --with-nes-mod --with-qib-mod --with-ipoib-mod --with-srp-mod 7. make make install NOTES: - 1. configure applies the patches to the source code according to the current - kernel. If you wish to rerun configure it is recommend to untar the source - code tree from the beginning and start with a clean state. - An alternative is to pass the option: --without-patch to the configure invocation. - - 2. The modules select for install are written to configure.mk.kernel + 1. The modules select for install are written to configure.mk.kernel + 2. INSTALL_MOD_DIR = "extra/ofa_kernel" for RedHat base Distros + INSTALL_MOD_DIR = "updates" for other Distros 7.2 Compiling the user space libraries -------------------------------------- @@ -515,13 +488,13 @@ do the following: Example for libibverbs: -1. tar xzf OFED-1.5.1.tgz -2. rpm -ihv SRPMS/libibverbs-1.1.3-0.6.g932f1a2.src.rpm +1. tar xzf OFED-3.5.tgz +2. rpm -ihv SRPMS/libibverbs-1.1.6-1.src.rpm 3. cd /usr/src/redhat/SOURCES (for RedHat) or cd /usr/src/packages/SOURCES (for SuSE) -4. tar xzf libibverbs-1.1.3.tgz -5. cd libibverbs-1.1.3 +4. tar xzf libibverbs-1.1.6.tgz +5. cd libibverbs-1.1.6 6. ./configure (specify parameters, if required) 7. make 8. make install